Select Language

HTTP/2 200 server: GitHub.com content-type: text/html; charset=utf-8 last-modified: Tue, 09 Dec 2025 05:36:55 GMT access-control-allow-origin: * etag: W/"6937b577-8235" expires: Mon, 29 Dec 2025 04:44:19 GMT cache-control: max-age=600 content-encoding: gzip x-proxy-cache: MISS x-github-request-id: 2ED8:444BC:84DBC4:9546A2:695204CA accept-ranges: bytes age: 0 date: Mon, 29 Dec 2025 04:34:19 GMT via: 1.1 varnish x-served-by: cache-bom-vanm7210045-BOM x-cache: MISS x-cache-hits: 0 x-timer: S1766982859.237069,VS0,VE216 vary: Accept-Encoding x-fastly-request-id: 2c4ac3365820eea5e2462174cc33b7c96a3579fb content-length: 9871

Xingyou (Richard) Song

I am currently a Research Scientist at Google DeepMind, currently working on Reinforcement Learning for Gemini Reasoning. Previously I worked on a variety of topics in Automated Machine Learning, Transformers, and Robotics.

I have also worked/interned at OpenAI, Citadel Securities, and Microsoft Research Asia.

Email  /  DBLP  /  Google Scholar  /  Linkedin  /  Github  /  Twitter  /  CV

Gemini Contributions

• Advanced Version of Gemini with Deep Think Officially Achieves Gold-Medal Standard at the International Mathematical Olympiad
  Key Contributor
  Google DeepMind Blog Post 2025
  [Blog] [Website] [Paper]
• Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
  Thinking/Reasoning Contributor
  Gemini Technical Report 2025
  [arXiv] [Blog]

Text-to-Text Regression

• Regression Language Models for Code
  Yash Akhauri*, Xingyou Song*, Arissa Wongpanich, Bryan Lewandowski, Mohamed S. Abdelfattah
  Technical Report 2025
  Paper: [arXiv] [Code] [Data]
  External Media/References: [MarkTechPost]
• Performance Prediction for Large Systems via Text-to-Text Regression
  Yash Akhauri, Bryan Lewandowski, Cheng-Hsi Lin, Adrian N. Reyes, Grant C. Forbes, Arissa Wongpanich, Bangding Yang, Mohamed S. Abdelfattah, Sagi Perel, Xingyou Song
  Google Technical Report 2025
  Paper: [arXiv] [Google Research Blog] [Code]
  External Media/References: [MarkTechPost] [MG (Video)]
• OmniPred: Language Models as Universal Regressors
  Xingyou Song, Oscar Li, Chansoo Lee, Bangding (Jeffrey) Yang, Daiyi Peng, Sagi Perel, Yutian Chen
  Transactions on Machine Learning Research (TMLR) 2024
  Paper: [arXiv] [TMLR] [Code] [US Patent] [Poster]
  External Media/References: [HuggingFace Daily Papers] [arXiv Papers (Video)] [MarkTechPost] [Prompt Engineering Institute] [BNN Breaking] [MultiPlatform AI] [Linnk] [Tunadorable Papers] [CSDN (Chinese)]
  Talks: [Slides] [AutoML Seminar] [ML Collective]

Decoding-based Regression

Xingyou Song*, Dara Bahri*

Transactions on Machine Learning Research (TMLR) 2025

Paper: [arXiv] [TMLR] [Code]
External Media/References: [MarkTechPost]

Understanding LLM Embeddings for Regression

Eric Tang, Bangding Yang, Xingyou Song

Transactions on Machine Learning Research (TMLR) 2025

Paper: [arXiv] [TMLR]
External Media/References: [MarkTechPost] [BAAI]

Open Source Vizier: Google's Optimizer

• Open Source Vizier: Distributed Infrastructure and API for Reliable and Flexible Blackbox Optimization
  Xingyou Song, Sagi Perel, Chansoo Lee, Greg Kochanski, Daniel Golovin
  Automated Machine Learning Conference (AutoML-Conf) Systems Track 2022
  Paper: [arXiv] [Google AI Blog] [AutoML-Conf] [PMLR] [Poster]     Code: [Github] [PyPI] [Docs]
  External Media/References: [Google Research 2022 Algorithms] [Deep Learning Tuning Playbook] [MarkTechPost] [Weights & Biases] [The Sequence] [Deep Learning Weekly] [Analytics India Magazine] [Electronic Smith] [gHacks] [ML News by Yannic Kilcher (Video)] [WebBigdata (Japanese)] [RandomAccess (Spanish)]
  Presentations: [Slides] [AutoML-Conf 2022 Video] [AutoML Seminar Video] [AutoML-Conf 2023 Tutorial Video]
• The Vizier Gaussian Process Bandit Algorithm
  Xingyou Song, Qiuyi Zhang, Chansoo Lee, Emily Fertig, Tzu-Kuo Huang, Lior Belenkil, Greg Kochanski, Setareh Ariafar, Srinivas Vasudevan, Sagi Perel, Daniel Golovin
  Google DeepMind Technical Report 2024
  Paper: [arXiv]
  External Media/References: [ChinaZ] [BAAI] [TuringPost] [DAIR.AI] [NLP Newsletter] [AIModels.FYI] [Open Data Science] [Appy Pie]
  Presentations: [Slides] [BAAI Talk]

OptFormer Project: Language Modeling for Optimization

• Towards Learning Universal Hyperparameter Optimizers with Transformers
  Yutian Chen, Xingyou Song, Chansoo Lee, Zi Wang, Qiuyi Zhang, David Dohan, Kazuya Kawakami, Greg Kochanski, Arnaud Doucet, Marc'aurelio Ranzato, Sagi Perel, Nando de Freitas
  Neural Information Processing Systems (NeurIPS) 2022
  Paper: [arXiv] [Code] [Google AI Blog] [US Patent] [Poster]
  External Media/References: [Tensorflow Transformer Tutorial] [Vevesta Labs] [Accenture Federal Viewpoints] [MarkTechPost] [The Evolving Enterprise] [LogicTech World] [Analytics Vidhya] [Same OG] [Tech News Crypt] [The Perfectech] [IANoticas (Spanish)] [MundoXDigital (Spanish)] [Digizom (Spanish)] [DataLearner (Chinese)] [HubAI (Chinese)] [AI Times (Korean)] [Web Big Data (Japanese)]
  Slides: [OptFormer Slides] [LLMs for Optimization Slides]
  Talk Videos: [ML Collective] [AutoML Seminar] [AutoML-Conf 2023 Tutorial]

Language Model Embeddings Can Be Sufficient for Bayesian Optimization

Tung Nguyen, Qiuyi Zhang, Bangding Yang, Chansoo Lee, Jorg Bornschein, Yingjie Miao, Sagi Perel, Yutian Chen, Xingyou Song

Google DeepMind Technical Report 2024

Paper: [arXiv] [Code]
External Media/References: [MarkTechPost]

Position: Leverage Foundational Models for Black-Box Optimization

Xingyou Song, Yingtao Tian, Robert Tjarko Lange, Chansoo Lee, Yujin Tang, Yutian Chen

International Conference on Machine Learning (ICML) 2024

Paper: [arXiv] [Poster]
External Media/References: [TuringPost] [Eye on AI] [TransferLab]

Reinforcement Learning and Robotics

• Discovering Adaptable Symbolic Algorithms from Scratch
  Stephen Kelly, Daniel S. Park, Xingyou Song, Mitchell McIntire, Pranav Nashikkar, Ritam Guha, Wolfgang Banzhaf, Kalyanmoy Deb, Vishnu Naresh Boddeti, Jie Tan, Esteban Real
  International Conference on Intelligent Robots and Systems (IROS) 2023 (Best Overall Paper Finalist, Top 12/2760 of Submissions)
  Paper: [arXiv] [Video] [Poster]
  External Media/References: [HuggingFace Daily Papers]

Efficient Transformers and Attention

• Rethinking Attention with Performers
  Krzysztof Choromanski*, Valerii Likhosherstov*, David Dohan*, Xingyou Song*, Andreea Gane*, Tamas Sarlos*, Peter Hawkins*, Jared Davis*, Afroz Mohiuddin, Lukasz Kaiser, David Belanger, Lucy Colwell, Adrian Weller
  International Conference on Learning Representations (ICLR) 2021 (Oral, Top 2% of Submissions)
  Paper: [arXiv] [ICLR] [Google AI Blog] [Slides]     Code: [Protein LM Code] [Performer Code]
  External Media/Blogposts: [Forbes] [Google Research 2022] [Hopfield Networks (ICLR Blog Track)] [Schmidhuber AI Blog] [HuggingFace] [Kerasmitas] [IARAI] [Hopfield Networks] [InfoQ] [TowardsDataScience] [AIM] [Eyerys] [TeddyKoker] [ExBulletin] [MarkTechPost] [Synced (v2)] [Synced (v1)] [VentureBeat] [LinkResearcher (Chinese)] [Zhuanlan (Chinese)] [KeXue (Chinese)] [MDLI (Hebrew)] [MIT Press (Sec 15.6)] [Google Research 2020]
  Forums: [Hacker News (YCombinator)] [Paper Explained (Reddit)] [FAVOR Explained (Reddit)]
  Talk Videos: [LightOnAI] [IARAI Fireside Chat]
  External Videos: [Paper Explained] [High Performance NLP] [BiliBili (Chinese)]
• Hardness of Low Rank Approximation of Entrywise Transformed Matrix Products
  (a) Tamas Sarlos, Xingyou Song, David Woodruff, Qiuyi Zhang
  Neural Information Processing Systems (NeurIPS) 2023
  [arXiv] [NeurIPS]
• Unlocking Pixels for Reinforcement Learning via Implicit Attention
  Krzysztof Choromanski*, Deepali Jain*, Wenhao Yu*, Xingyou Song, Jack Parker-Holder, Tingnan Zhang, Valerii Likhosherstov, Aldo Pacchiano, Anirban Santara, Yunhao Tang, Jie Tan, Adrian Weller.
  [arXiv] [Site]
• Sub-Linear Memory: How to Make Performers SLiM
  Valerii Likhosherstov, Krzysztof Choromanski, Jared Davis, Xingyou Song, Adrian Weller
  Neural Information Processing Systems (NeurIPS) 2021
  [arXiv] [Code]

Miscellaneous

• Gradientless Descent: High-Dimensional Zeroth-Order Optimization
  (a) Daniel Golovin, John Karro, Greg Kochanski, Chansoo Lee, Xingyou Song, Qiuyi Zhang
  International Conference on Learning Representations (ICLR) 2020 (Spotlight, Top 6% of Submissions)
  [arXiv] [ICLR] [Slides]
• Debiasing First-order Heuristic for Approximate Bi-level Optimization
  Valerii Likhosherstov*, Xingyou Song*, Krzysztof Choromanski, Jared Davis, Adrian Weller
  Interntional Conference on Machine Learning (ICML) 2021
  [arXiv] [ICML] [Code]
• An Ode to an ODE
  Krzysztof Choromanski*, Jared Quincy Davis*, Valerii Likhosherstov*, Xingyou Song, Jean-Jacques Slotine, Jacob Varley, Honglak Lee, Adrian Weller, Vikas Sindhwani
  Neural Information Processing Systems (NeurIPS) 2020
  [arXiv] [NeurIPS]
• Stochastic Flows and Geometric Optimization on the Orthogonal Group
  Krzysztof Choromanski*, David Cheikhi*, Jared Davis*, Valerii Likhosherstov*, Achille Nazaret*, Achraf Bahamou*, Xingyou Song*, et al.
  International Conference on Machine Learning (ICML) 2020
  [arXiv] [ICML]