Jan Schlegel – Blog

Does Catastrophic Forgetting Happen in Tiny Subspaces?

Python

Machine Learning

Deep Learning

Continual Learning

We investigate whether catastrophic forgetting in continual learning is geometrically similar to ordinary learning: concentrated in the low-curvature bulk subspace of the loss Hessian rather than its high-curvature dominant directions. By projecting gradient updates into either the bulk or dominant subspace and training on Permuted MNIST, Split-CIFAR10, and Split-CIFAR100, we confirm that learning occurs primarily in the bulk. In the Domain-IL setting, forgetting appears to reside there as well. Task-IL results are more ambiguous, motivating larger-scale experiments.

Detecting Pneumonia from Chest X-Rays: CNNs, Integrated Gradients, and Grad-CAM

Python

Data Science

Machine Learning

Deep Learning

Healthcare

We train a ResNet-50-based convolutional neural network to classify chest X-ray images as healthy or pneumonia. To understand what the model actually learns, we apply two attribution methods, Integrated Gradients and Grad-CAM, and compare the regions they highlight. A data randomization test confirms that both methods produce genuinely model-dependent explanations rather than artifacts. The attribution maps reveal that the CNN focuses on clinically sensible lung regions, with broader activation patterns for pneumonia cases and more localized attention for healthy patients.

Detecting Heart Attacks from ECGs: Classical ML, Deep Learning, and Representation Learning

Python

Data Science

Machine Learning

Deep Learning

Healthcare

We classify electrocardiogram recordings as healthy or indicative of myocardial infarction using a progression of increasingly powerful methods. Starting from logistic regression and gradient boosting on raw signals, we show that feature engineering closes much of the gap to deep learning. We then train LSTMs, CNNs, and a transformer, and use attention maps to visualize which parts of the ECG signal drive predictions. Finally, we explore whether contrastive representation learning can produce embeddings that transfer across datasets and outperform supervised pretraining.

Predicting Heart Disease: When Simple Models Rival Deep Learning

Python

Data Science

Machine Learning

Deep Learning

Healthcare

We train a Lasso logistic regression, a multi-layer perceptron, and a neural additive model to predict heart disease from routine clinical data. All three achieve high and comparable accuracy. Using SHAP values, learned shape functions, and coefficient analysis to interpret each model, we find that they independently identify the same clinical features as most important, consistent with established cardiology literature.

Data-Driven Tactics: Analyzing Spain’s Playing Style in Euro 2024 Qualifiers

Python

Data Science

Data Analysis

Visualization

Sports Analytics

We examine Spain’s characteristic ‘Tiki-Taka’ playing style through multiple analytical lenses: offside patterns, passing networks, set-piece execution, formation dynamics, and defensive line positioning. Our analysis reveals how Spain maintains possession through short passing sequences involving defenders, utilizes full pitch width when in possession, and employs a high defensive line that reflects their controlling philosophy. We also identify tactical vulnerabilities, particularly Spain’s susceptibility to offsides and counter-attacks, providing actionable insights for opposition teams.

A Visual Introduction to Copulas

Statistics

Finance

Python

This post offers a concise and intuitive introduction to copula theory, explaining what copulas are, why they are useful, and how they are applied in finance. We’ll skip the heavy math and focus on the core ideas.

Comparing Univariate and Multivariate Models for Value at Risk Forecasting

Simulation

R

This post explores the effectiveness of univariate and multivariate GARCH-based models in forecasting Value at Risk (VaR) for a long equity portfolio. While multivariate models generally perform better in backtests, univariate models often fall short. However, neither model type consistently outperforms the other in predictive accuracy, highlighting the trade-offs between simplicity and complexity in risk forecasting.