| CARVIEW |
Rahul Ramesh
I am a 5th year CIS Ph.D. student at the University of Pennsylvania advised by Pratik Chaudhari. I am interested in a principled data-centric view of deep learning. My research seeks to answer questions like: (1) What do “typical” learnable tasks look like? (2) How do we build optimal representations from unlabeled data? (3) How does data change over time and how should models adapt?
A theme the spans my work is the idea of task competition, i.e., all tasks need not share an optimal representation and we should instead train on data from a related subset of tasks.
I spent summer 2023 with NTT Research at the Harvard Center for Brain Science and worked on understanding why language models are able to solve many different tasks with Hidenori Tanaka. Previously, I interned at Amazon AI labs and worked on pre-trained models for image classification with Avinash Ravichandran and Aditya Deshpande.
I graduated from IIT Madras in 2019, at the top of my class, with a dual degree in computer science and a minor in physics. I was advised by Balaraman Ravindran and worked on building hierarchical representations for reinforcement learning using successor representations.
I am currently on the job market and am looking for research scientist and post-doc opportunities!
Select Publications
@InProceedings{pmlr-v235-ramesh24a,
title = {Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable Tasks},
author = {Ramesh, Rahul and Lubana, Ekdeep Singh and Khona, Mikail and Dick, Robert P. and Tanaka, Hidenori},
booktitle = {Proceedings of the 41st International Conference on Machine Learning},
pages = {42074--42103},
year = {2024},
editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix},
volume = {235},
series = {Proceedings of Machine Learning Research},
month = {21--27 Jul},
publisher = {PMLR},
pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/ramesh24a/ramesh24a.pdf},
url = {https://proceedings.mlr.press/v235/ramesh24a.html},
}
@inproceedings{pmlr-v202-ramesh23a,
title={A Picture of the Space of Typical Learnable Tasks},
author =Ramesh, Rahul and Mao, Jialin and Griniasty, Itay and Yang, Rubing and Teoh, Han Kheng and Transtrum, Mark and Sethna, James and Chaudhari, Pratik},
booktitle={Proceedings of the 40th International Conference on Machine Learning},
pages ={28680--28700},
editor={Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan},
year ={2023},
volume={202},
series={Proceedings of Machine Learning Research},
month={23--29 Jul},
publisher={PMLR},
pdf={https://proceedings.mlr.press/v202/ramesh23a/ramesh23a.pdf},
url={https://proceedings.mlr.press/v202/ramesh23a.html}
}
@inproceedings{
ramesh2022model,
title={Model Zoo: A Growing Brain That Learns Continually},
author={Rahul Ramesh and Pratik Chaudhari},
booktitle={International Conference on Learning Representations},
year={2022},
url={https://openreview.net/forum?id=WfvgGBcgbE7}
}
@article{gao2022deep,
title={Deep Reference Priors: What is the best way to pretrain a model?},
author={Gao, Yansong and Ramesh, Rahul and Chaudhari, Pratik},
journal={arXiv preprint arXiv:2202.00187},
year={2022}
}
@inproceedings{pmlr-v202-de-silva23a,
title={The Value of Out-of-Distribution Data},
author={De Silva, Ashwin and Ramesh, Rahul and Priebe, Carey and Chaudhari, Pratik and Vogelstein, Joshua T},
booktitle={Proceedings of the 40th International Conference on Machine Learning},
pages={7366--7389},
year={2023},
editor={Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan},
volume={202},
series={Proceedings of Machine Learning Research},
month={23--29 Jul},
publisher={PMLR},
pdf={https://proceedings.mlr.press/v202/de-silva23a/de-silva23a.pdf},
url={https://proceedings.mlr.press/v202/de-silva23a.html},
}