Selected work across probabilistic machine learning, high-dimensional statistics, and their applications to climate and health.
Master's Thesis2025
Mechanistic Interpretability in Text-to-Image Diffusion Models
Jan Schlegel
M.Sc. Thesis · ETH Zurich · Grade 6.00 / 6.00
Despite the widespread success of text-to-image diffusion models, their internal mechanisms remain largely opaque. In this work, we mechanistically analyze Stable Diffusion v2.1 by applying Sparse Autoencoders (SAEs) to decompose the dense activations of its cross-attention blocks into a sparse, interpretable feature basis. Through time-stratified causal interventions, we reveal a clear temporal hierarchy in the generative process, demonstrating that early timesteps are critical for defining an image's global composition and style, the mid-stage refines the established structure and blends concepts, while later stages perform textural refinement. Moreover, we find that the learned feature dictionary exhibits remarkable cross-resolution generalization, maintaining intervention and reconstruction efficacy across different image sizes, and suggesting that SAEs can be trained more efficiently on lower-resolution data without sacrificing their analytical power. Moving beyond single-feature analysis, we introduce a novel framework for discovering temporal computational subgraphs in diffusion models. By adapting gradient-based attribution techniques, we trace causal dependencies between features across early denoising steps. By identifying these causal computational pathways that link sequences of these interpretable features, our work provides a deeper mechanistic understanding of the generative algorithm, paving the way for more controllable and reliable generative models.
Extrapolation and Distributional Robustness for Climate Downscaling
Gianfranco Basile*, Jan Schlegel*, Maybritt Schillinger, Nicolai Meinshausen
* Equal contribution · Seminar for Statistics, ETH Zurich
Climate downscaling translates coarse global climate model (GCM) outputs into high-resolution predictions that inform local decision-making. This work investigates the robustness and transferability of statistical downscaling methods applied to unseen GCMs. Building on generative frameworks such as Coarse-from-Super and Corrector Diffusion models, we evaluate their performance under cross-GCM extrapolation. Our experiments show that stochastic methods outperform deterministic baselines in energy score while maintaining comparable mean squared error, highlighting the potential of generative downscaling for modeling the variability inherent in climate systems.
Does Catastrophic Forgetting Happen in Tiny Subspaces?
Rufat Asadli, Armin Begic, Jan Schlegel, Philemon Thalmann
Deep Learning course project · ETH Zurich
Catastrophic forgetting remains a central challenge in continual learning, where adapting to new tasks disrupts previously acquired knowledge. Recent work suggests that non-continual learning occurs primarily within the low-curvature bulk subspace of the loss Hessian. We investigate how constraining gradient updates to either the bulk or dominant subspace affects learning and forgetting. Across Permuted MNIST, Split-CIFAR10, and Split-CIFAR100, we confirm that task-specific learning occurs in the bulk subspace, and find evidence that forgetting may predominantly occur there as well — pointing toward efficient algorithms to counter forgetting.
Portfolio Value at Risk Forecasting with Copula-GARCH Models
Jan Schlegel
B.A. Thesis · University of Zurich · Grade 6.00 / 6.00
This thesis examines the value at risk (VaR) forecasting ability of various univariate and multivariate models for a long equity portfolio. All of the considered models involve a generalized autoregressive conditional heteroskedasticity (GARCH)-type structure. The resulting forecasts are checked for desirable properties using violation-based backtests and compared in terms of predictive ability. We find that the VaR forecasts of almost all univariate models are inadequate, while the multivariate models have few problems passing these backtests. However, we do not find evidence that the multivariate models systematically outperform their univariate counterparts with regards to predictive accuracy, or vice versa.