I am a post-doctoral researcher at INRIA Paris in the Sierra team with Francis Bach working on optimization for machine learning. I received my PhD at UBC in 2024, where I worked with Mark Schmidt. Prior to UBC, I studied at EPFL with Martin Jaggi, and had the chance to intern at the MPI with Philipp Hennig and at RIKEN with Emtiyaz Khan.
Selected works
Why Adam Outperforms Gradient Descent on Language Models: A Heavy-Tailed Class Imbalance Problem
F. Kunstner, R. Yadav, A. Milligan, M. Schmidt, A. Bietti.
2024 arXiv arXiv code .bib
Searching for Optimal Per-Coordinate Step-sizes with Multidimensional Backtracking
F. Kunstner, V. S. Portella, M. Schmidt, N. Harvey.
2023 NeurIPS arXiv code OpenReview proceedings poster .bib
Noise is not the main factor behind the gap between SGD and Adam on transformers, but sign descent might be
F. Kunstner, J. Chen, J. W. Lavington, M. Schmidt.
2023 ICLR arXiv code OpenReview poster .bib
Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent
F. Kunstner, R. Kumar, M. Schmidt.
2021 AISTATS arXiv proceedings poster .bib
BackPACK: Packing more into backprop
F. Dangel, F. Kunstner, P. Hennig.
2020 ICLR arXiv website OpenReview poster .bib
Software utilities
- Tex2UTF8: For places that do not support Latex but happily render UTF8 (finally!)
- DSDL: An automated dataset downloader for libsvm datasets