Leena C Vankadara

Exciting News!

I am thrilled to announce that I will be joining the Gatsby Computational Neuroscience Unit at UCL as a Lecturer (Assistant Professor) in Feb 2025. This new role marks an exciting chapter in my academic career. Stay tuned for updates on my upcoming courses, research projects, and other activities at the Gatsby Unit!

I'm hiring Ph.D. students and Postdocs!

Prospective Ph.D. students: please apply here: Gatsby Unit PhD programme.

Prospective Postdocs: please get in touch via email!

Short Bio

I am an Applied Scientist at the AGI Foundations Lab at Amazon, where I am currently conducting fundamental research on the theory and science of scaling large language models. I previously earned my Ph.D., graduating summa cum laude, from the International Max Planck Research School for Intelligent Systems with Debarghya Ghoshdastidar at the Theoretical Foundations of Artificial Intelligence group, Technical University of Munich, and Ulrike von Luxburg at the Theory of Machine Learning group, University of Tuebingen. During my Ph.D., I spent 6 months at the Causality Lab of Amazon Research.

My research aims to develop efficient, reliable, and trustworthy machine learning models by understanding the theoretical foundations of deep learning and causal learning. Deep learning theory and causality present some of the most fascinating challenges in modern machine learning. Classical theories of statistical learning and optimization often struggle to explain the empirical success of deep learning algorithms or even provide reliable guidance on their behaviour. As models continue to scale to unprecedented levels, revealing emergent abilities, we need fundamentally new theories to explain their behavior. To tackle some of these questions, my research aims to i) develop a unified theory of scaling and ii) understand the learning dynamics and generalization in deep learning. Meanwhile, Causality remains in its nascency, with formal mechanisms still emerging. My research will address open problems in causality, including establishing robust formalisms for causal reasoning and developing frameworks to evaluate causal models.

Recent News

• [25/07/2024] Two papers have been accepted to NeurIPS 2024! Very excited to be presenting our papers: "On Feature Learning in Structured State Space Models" and "mup2: Effective Sharpness Aware Minimization Requires Layerwise Perturbation Scaling" at the conference in December!
• [24/09/2024] I am delighted to announce that I will be joining the Gatsby Computational Neuroscience Unit at UCL as a Lecturer (Assistant Professor) in February 2025!
• [24/09/2024] Our paper A Consistent Estimator for Confounding Strength has been accepted by the Mathematical Statistics and Learning Journal!
• [Upcoming 14/10/2024] Excited to be giving a talk on "Towards a theory of scaling" at the Theory and Practice of Deep Learning, IPAM, UCLA, Los Angeles, U.S.A!
• [Upcoming 14/12/2024] Will be giving a talk on "Towards a theory of scaling" at the International Joint Conference CFE-CMStatistics, London, UK.
• [25/07/2024] Excited to announce that our work "On Feature Learning in Structured State Space Models" has been accepted to the ICML NGSM Workshop 2024 as a Spotlight presentation, and it was also the Runner-up for Best Paper Award!
• [25/07/2024] We will be presenting our paper "Effective Sharpness Aware Minimization Requires Layerwise Perturbation Scaling" at the ICML HiLD Workshop 2024.
• [10/07/2024] Giving a tutorial on "Scaling and Reliability Foundations in Machine Learning" at the IEEE International Symposium on Information Theory, Athens, Greece!
• [20/06/2024] Giving a talk on "Scaling theory for sharpness-aware minimization" at INRIA-LILLE, Nord Europe, Villeneuve-d'Ascq, France.
• [06/05/2024] Our work "Explaining Kernel Clustering via Decision Trees" has been accepted to ICLR 2024!
• [02/05/2024] We will be presenting "Self-Compatibility: Evaluating Causal Discovery without Ground Truth" at AISTATS 2024!
• [05/02/2024] Giving a talk on "Interpolation and Regularization for Causal Learning" at the Gatsby Computational Neuroscience Unit, London, UK.
• [01/02/2024] Moved to the AGI Foundations team at Amazon as an Applied Scientist II. Will be working on theory and science of scaling large language models.
• [12/12/2023] Giving a talk on "Is Memorization Compatible with Causal Learning. The Case of High-Dimensional Linear Regression" at the International Joint Conference CFE-CMStatistics, Berlin, Germany.
• [01/01/2023] Started a new position in the Causality lab at Amazon Research as an Applied Scientist II.
• [21/12/2022] Submitted my PhD Thesis!
• [02/12/2022] Luca Rendsburg and I will be presenting our work on Interpolation and Regularization for Causal Learning. at NeurIPS 2022 in New Orleans, USA.
• [03/11/2022] New Preprint: A Consistent Estimator for Confounding Strength.
• [15/09/2022] Paper: Interpolation and Regularization for Causal Learning. accepted to NeurIPS 2022.
• [14/09/2022] Giving a talk on Causal properties of interpolating estimators in the Department of Mathematical Statistics, Technical University of Munich.
• [27/05/2022] I will be in London for two weeks attending the annual meeting of the Institute of Mathematical Statistics and COLT 2022!
• [19/05/2022] Giving a talk on "Interpolation and Regularization for Causal Learning" at the Women in Data Science conference in Chemnitz.
• [27/04/2022] Mahalakshmi Sabanayagam will be presenting our work on Graphon based Clustering and Testing of Networks: Algorithms and Theory at ICLR 2022!
• [24/03/2022] Giving a talk on "Kernels for High-Dimensional Large-Scale Clustering" at the SIAM Conference on Imaging Science.

Advisees/Interns

Incoming

• Leello Tadesse Dadi (Starting November 2024), École Polytechnique Fédérale de Lausanne (EPFL)
• Valentyn Boreiko (Starting November 2024), University of Tuebingen

Current

• Anish Dhir, Imperial College, London
• Vaclav Voracek, University of Tuebingen

Previous

• Moritz Haas, University of Tuebingen
• Jin Xu, University of Oxford
• Alexander Meulemans, ETH, Zurich
• Maximilian Fleissner, Technical University of Munich
• Mahalakshmi Sabanayagam, Technical University of Munich
• Margareta Schlüter, University of Tuebingen

Publications

Most recent publications on Google Scholar.
^‡ indicates equal contribution.

• Selected
• All

Vitæ

• Gatsby Unit, UCL Starting Feb 2025

  Lecturer (Assistant Professor)
  
• Amazon Research January 2024 - now

  Applied Scientist II
  AGI Foundations, Amazon
• Amazon Research January 2023 - now

  Applied Scientist II
  Causality Lab
• IMPRS-IS 2018 - 2022

  Ph.D. Candidate
  Theory of Machine Learning Group
• Amazon Research Fall 2020

  Applied Scientist, Intern
  Causality Lab
• Max Planck Institute (MPI-IS) June - December 2018

  Research Scientist, Intern
  Statistical Learning Theory Group
• University of Tuebingen April 2017 – June 2018

  Teaching Assistant
  Theory of Machine learning
  
• University of Tuebingen 2017

  Master's Thesis
  Theory of Machine learning group
• University of Hamburg 2014 - 2017

  Master's Student
  Masters in Machine learning