| CARVIEW |
Select Language
HTTP/2 200
server: GitHub.com
content-type: text/html; charset=utf-8
last-modified: Fri, 29 Aug 2025 04:35:08 GMT
access-control-allow-origin: *
etag: W/"68b12dfc-6a7b"
expires: Mon, 29 Dec 2025 20:06:57 GMT
cache-control: max-age=600
content-encoding: gzip
x-proxy-cache: MISS
x-github-request-id: A93D:3FD64F:948EEF:A69BAA:6952DD09
accept-ranges: bytes
age: 0
date: Mon, 29 Dec 2025 19:56:57 GMT
via: 1.1 varnish
x-served-by: cache-bom-vanm7210084-BOM
x-cache: MISS
x-cache-hits: 0
x-timer: S1767038217.347075,VS0,VE208
vary: Accept-Encoding
x-fastly-request-id: 2333f98540d918ddf07ad5ff79324cebe5384930
content-length: 7007
Xingjian Shi
Xingjian Shi
施行健
Member of Technical Staff, OpenAI
- 4677 Old Ironsides Drive, Santa Clara
- CA 95054, United States
- xshiab AT connect DOT ust DOT hk
- My Github
- My Google Scholar
About Me
I'm currently building multimodal models at OpenAI.
Before that, I lead model development at Boson AI, founded by Dr. Mu Li and Dr. Alex Smola. We created the Higgs series of LLM and multimodal LLMs, like Higgs-Llama and Higgs-Audio, designed to advance the state of the art in language and audio understanding and generation.
Before Boson AI, I was a senior applied scientist at Amazon that leads two projects: AutoGluon Multimodal and DeepEarth. AutoGluon Multimodal goes beyond traditional AutoML tools by leveraging foundation models. DeepEarth builds foundation models for earth. I obtained Ph.D degree from the Hong Kong University of Science and Technology in 2018. My Ph.D supervisor was Prof. Dit-Yan Yeung. I received my B.E. degree from Shanghai Jiao Tong University in 2014. I love open source and participated in projects like Apache/MXNet.
Research Interest
- Multimodal generative model (Vision & Audio & Language)
- Multimodal AutoML
- Deep learning for earth science
Selected Publications by Topic
Deep Learning for Earth Science
-
Bridging Remote Sensors with Multisensor Geospatial Foundation ModelsBoran Han, Shuai Zhang, Xingjian Shi, Markus Reichstein.IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.[paper]
-
PreDiff: Precipitation Nowcasting with Latent Diffusion Models.Zhihan Gao, Xingjian Shi, Boran Han, Hao Wang, Xiaoyong Jin, Danielle Maddix Robinson, Yi Zhu, Mu Li, Yuyang Bernie Wang.Thirty-Sixth Annual Conference on Neural Information Processing Systems (NeurIPS), 2023.
-
Towards Geospatial Foundation Models via Continual Pretraining.Matias Mendieta, Boran Han, Xingjian Shi, Yi Zhu, Chen Chen.International Conference on Computer Vision (ICCV), 2023.[paper] [project page]
-
Earthformer: Exploring Space-Time Transformers for Earth System Forecasting.Zhihan Gao, Xingjian Shi*, Hao Wang, Yi Zhu, Yuyang Wang, Mu Li, Dit-Yan Yeung (* Contact person)Thirty-Fifth Annual Conference on Neural Information Processing Systems (NeurIPS), 2022.
-
Deep learning and the weather forecasting problem -- precipitation nowcasting.Zhihan Gao, Xingjian Shi*, Hao Wang, Dit-Yan Yeung, Wang-chun Woo, and Wai-Kin Wong (* indicates corresponding author)Deep learning for the Earth Sciences: A Comprehensive Approach to Remote Sensing, Climate Science and Geosciences, G. Camps-Valls, D. Tuia, X.X. Zhu, and M. Reichstein (eds.), Wiley & Sons, 2021.
-
Deep Learning for Precipitation Nowcasting: A Benchmark and A New Model.Xingjian Shi, Zhihan Gao, Leonard Lausen, Hao Wang, Dit-Yan Yeung, Wai-kin Wong, Wang-chun WooThirty-First Annual Conference on Neural Information Processing Systems (NeurIPS), 2017. (Accepted as Spotlight)
-
Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting.Xingjian Shi, Zhourong Chen, Hao Wang, Dit-Yan Yeung, Wai-kin Wong, Wang-chun WooTwenty-Ninth Annual Conference on Neural Information Processing Systems (NeurIPS), 2015.
Multimodal AutoML
-
Tailoring Instructions to Student's Learning Levels Boosts Knowledge Distillation.Yuxin Ren*, Zihan Zhong*, Xingjian Shi, Yi Zhu, Chun Yuan, Mu Li (* indicates equal contribution)The 61st Annual Meeting of the Association for Computational Linguistics (ACL), 2023.[paper] [project page]
-
XTab: Cross-table Pretraining for Tabular Transformers.Bingzhao Zhu, Xingjian Shi, Nick Erickson, Mu Li, George Karypis, Mahsa ShoaranFortieth International Conference on Machine Learning (ICML), 2023.[paper] [project page]
-
Parameter-Efficient Fine-Tuning Design Spaces.Jiaao Chen, Aston Zhang, Xingjian Shi, Mu Li, Alex Smola, Diyi YangThe Eleventh International Conference on Learning Representations (ICLR), 2023.[paper] [project page]
-
Learning Multimodal Data Augmentation in Feature Space.Zichang Liu, Zhiqiang Tang, Xingjian Shi, Aston Zhang, Mu Li, Anshumali Shrivastava, Andrew Gordon WilsonThe Eleventh International Conference on Learning Representations (ICLR), 2023.[paper] [project page]
-
Benchmarking Multimodal AutoML for Tabular Data with Text Fields.Xingjian Shi*, Jonas Mueller*, Nick Erickson, Mu Li, Alexander J. Smola (* indicates equal contribution)Proceedings of the Neural Information Processing Systems (NeurIPS) Track on Datasets and Benchmarks, 2021.[paper] [project page]
Generative Models
-
L3Ms--Lagrange Large Language Models.Guneet S Dhillon, Xingjian Shi, Yee Whye Teh, Alexander J. SmolaThe Thirteenth International Conference on Learning Representations (ICLR), 2025.[paper]LayoutDiffuse: Adapting Foundational Diffusion Models for Layout-to-Image Generation.Jiaxin Cheng, Xiao Liang, Xingjian Shi, Tong He, Tianjun Xiao, Mu LiThe AI for Content Creation (AI4CC) workshop at CVPR, 2023.[paper] [project page]Symbolic Music Generation with Transformer-GANs.Aashiq Muhamed*, Liang Li*, Xingjian Shi, Suri Yaddanapudi, Wayne Chi, Dylan Jackson, Rahul Suresh, Zachary C. Lipton, Alexander J. Smola (* indicates equal contribution)Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI), 2021.[paper] [project page]
Deep Learning System
-
GluonCV and GluonNLP: Deep Learning in Computer Vision and Natural Language Processing.Jian Guo, He He, Tong He, Leonard Lausen, Mu Li, Haibin Lin, Xingjian Shi, Chenguang Wang, Junyuan Xie, Sheng Zha, Aston Zhang, Hang Zhang, Zhi Zhang, Zhongyue Zhang, Shuai Zheng, Yi ZhuJournal of Machine Learning Research (JMLR), 2020.[paper]
Spatiotemporal Modeling
-
GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs.Xingjian Shi*, Jiani Zhang*, Junyuan Xie, Hao Ma, Irwin King, Dit-Yan Yeung (* indicates equal contribution)Thirty-Fourth Conference on Uncertainty in Artificial Intelligence (UAI), 2018.[paper] [project page]
-
Spatiotemporal Modeling for Crowd Counting in Videos.Feng Xiong, Xingjian Shi, Dit-Yan YeungSixteenth IEEE International Conference on Computer Vision (ICCV), 2017.[paper] [project page]
Academic Service
- Area Chair: ICLR 2026
- PC Member: IJCAI 2015, AAAI 2018-now
- Senior PC Member: IJCAI 2021
- Conference Reviewer: ICML (2019-now), NeurIPS (2019-now), ICLR (2021-now)
- Journal Reviewer:
- ACM Transactions on Intelligent Systems and Technology
- ACM Transactions on Knowledge Discovery from Data
- Computational Intelligence
- Data Mining and Knowledge Discovery
- Frontiers of Computer Science
- IEEE Journal of Biomedical and Health Informatics
- IEEE Transactions on Big Data
- IEEE Transactions on Image Processing
- IEEE Transactions on Information Forensics and Security
- IEEE Transactions on Knowledge and Data Engineering
- IEEE Transactions on Neural Networks and Learning Systems
- International Journal of Data Science and Analytics
- Neural Computation
- Sustainable Cities and Society
- Weather and Forecasting
Working Experiences
-
- Member of Technical Staff at OpenAI
- Aug, 2025-Now
-
- Member of Technical Staff at Boson AI
- May, 2023-Aug, 2025
-
- Senior Applied Scientist at AWS
- Apr, 2021-Mar, 2023
-
- Applied Scientist at AWS
- Jul, 2019-Apr, 2021
-
- Applied Scientist Intern at AWS
- Nov, 2017-Apr, 2018
Selected Awards
-
- Hong Kong PhD Fellowship
- 2014-2018
-
- NeurIPS Travel Award
- 2015, 2017
-
- National Scholarship, Shanghai Jiao Tong University (Top 1%)
- 2011-2013
Presentations
-
AutoGluon: Empowering (Multimodal) AutoML for the Next 10 Million Users, NeurIPS Workshop, 2022
-
Practical Automated Machine Learning with Tabular, Text, and Image Data, KDD Tutorial, 2020
-
Deep Learning for Precipitation Nowcasting: A Benchmark and A New Model, NeurIPS Spotlight Presentation, 2017
-
Dynamic Key-Value Memory Networks for Knowledge Tracing, VALSE, 2017
-
모두를 위한 MXNET, AWS Seoul Summit, 2017
-
A Practitioner's Guide to MXNet, HKUST CSE Seminar, 2017
-
Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting, VALSE Seminar, 2016