| CARVIEW |
Biography
Hello! This is Jianing “Jed” Yang (杨佳宁). I am an AI Engineer at Figure AI, where I’m working on building the future of general-purpose humanoid robots. I recently completed my Ph.D. in Computer Science and Engineering at University of Michigan, where I was a member of SLED lab, advised by Joyce Chai and frequently collaborating with David Fouhey. My research focused on understanding the 3D physical world through vision and language, enabling robots to act in such environments in a generalizable and controllable manner, and using natural language as feedback to teach and improve embodied agents. My dream is to build and deploy household robots to homes around the world to help humans with daily tasks and needs.
Before joining UMich, I obtained my Master’s in Machine Learning from Carnegie Mellon University where I worked with Prof. Louis-Philippe Morency and Prof. Matt Gormley on Multimodal Natural Language Understanding, Dataset Bias Analysis, and Machine Learning. I got my Bachelor’s in Computer Science from Georgia Tech.
Before I delved fully into research, I worked as a Software Development Engineer at Amazon Web Services, intern and full-time.
Outside of work, you will find me skiing/cooking/baking/photographing.
Click here for my full bio and here for my CV.
- 3D Computer Vision
- Embodied AI
- Robobtics
- Multimodal Machine Learning
- Natural Language Processing
Ph.D. in Computer Science and Engineering, 2025
University of Michigan
M.S. in Machine Learning, 2020
Carnegie Mellon University
B.S. in Computer Science, 2018
Georgia Institute of Technology
News
- [Oct. 2025] 🤖 I joined Figure AI as an AI Engineer! Let’s build the future of general-purpose humanoid robots! 🚀
- [Sep. 2025] 🎓 I successfully defended my PhD dissertation and graduated with a Doctor of Philosophy in Computer Science and Engineering from the University of Michigan! 🎉
- [June 2025] 🚀 I’m organizing the first 3D-LLM/VLA Workshop at CVPR 2025! Come join us and learn about how to bridge Language, Vision and Action in 3D Environments!
- [May 2025] ⭐️ I returned to Meta FAIR Perception team for summer 2025 internship!
- [May 2025] 🎉 LIFT-GS is accepted to ICML 2025!
- [Mar. 2025] 🔥 Fast3R is accepted to CVPR 2025! 3D reconstruction from 1000+ images in one forward pass at up to 251 FPS!⚡️ Try out the
demo with your own images/videos! Everything is open-sourced! - [Mar. 2025] 🎉 3D-GRAND is accepted to CVPR 2025!
- [Jan. 2025] 🚀 I moved to the Bay Area and joined Adobe Research as a Research Scientist Intern to work on 3D reconstruction (mentor: Hao Tan)!
- [Oct. 2024] 🎉 Multi-object hallucination is accepted to NeurIPS and Teachable Reinforcement Learning is accepted to EMNLP. Check them out!
- [June 2024] 🚀 We released 3D-GRAND, the first million-scale densly-grounded 3D-text dataset for 3D-LLMs! Trained with this data, our model obtained stronger 3D grounding capabilities and drastically reduces hallucinations. We also proposed and released 3D-POPE, a benchmark to evaluate 3D-text hallucinations for 3D-LLMs. Try out live demo on our website!
- [Feb. 2024] ⭐️ I will join the Meta Embodied AI team Summer 2024 as a Research Scientist Intern!
- [Jan. 2024] 🎉 LLM-Grounder is accepted to ICRA 2024!
- [Sep. 2023] Preprint of LLM-Grounder, is now available on arXiv and featured in Hugging Face 🤗 Daily Papers! Watch our YouTube video demo, or try it out yourself on our live demo. Chat with an LLM agent to ground 3D objects!
- [June 2023] 🏆 We won 🥇 First Place ($500,000) in the first-ever Amazon Alexa Prize SimBot Challenge! It was an absolute honor to co-lead the amazing Team SEAGULL with Yichi Zhang! Big thank you and congrats to all of our team members! 🎉 Read our technical report here. (Media coverage: U-M, Amazon Science, 机器之心)
- [May 2023] We open-sourced the 📸 Chat-with-NeRF (Twitter: 1, 2) project with a live demo. Try it out - chat with a 3D room and let us know what you think! 🙌
- [Oct. 2022] Our paper DANLI is accepted to EMNLP 2022 (oral)!
Click here for news archive
- [Aug. 2022] Team SEAGULL advanced to Phase Two of the 2023 Amazon Alexa Prize SimBot Challenge! I will continue to co-lead our team with Yichi to represent UMich! 🎉
- [May 2022] I passed my qualification exam and became a Ph.D. candidate 🙌
- [Nov. 2021] We are selected as one of the First Amazon Alexa Prize SimBot Challenge participants! I will be co-leading Team SEAGULL with Yichi to represent UMich!. 🎉
- [Aug. 2021] I started my Ph.D. at University of Michigan!
- [Mar. 2021] 🎉 Our paper MTAG is accepted to NAACL 2021! The code is available here.
- [Dec. 2020] 🎉 I finished my Master’s and graduated from CMU! 🎉 I will continue to stay as a Research Assistant at CMU’s MultiComp lab until my PhD starts in Fall 2021.
- [July 2020] I’m attending ACL 2020 virtually. Check out our video presentation on QA bias analysis!
- [Apr. 2023] Team SEAGULL advanced to one of the five finalists for the SimBot Challenge and was covered by U-M CSE News! 🎉
- [Feb. 2023] Team SEAGULL is selected as one of the 2023 Amazon Alexa Prize SimBot Challenge semi-finalists! 💪
- [July 9th-17th 2022] I will attend NAACL 2022 in-person in Seattle! Hit me up!
- [Nov. 2020] I’m attending EMNLP 2020 virtually. Happy to chat there!
Publications
SAB3R: Semantic-Augmented Backbone in 3D Reconstruction
SEAGULL: An Embodied Agent for Instruction Following through Situated Dialog
What Gives the Answer Away? Question Answering Bias Analysis on Video QA Datasets
SUOD: Toward Scalable Unsupervised Outlier Detection
Industry Experience
- FAIR Perception team
- Graphics and 3D Imaging team
- FAIR Embodied AI team
Led a load balancing project to decrease system latency from 20 seconds to milliseconds.
Received award for technical soundness and leadership at 2019 Q2 AWS Identity organization meeting.
Blogs

In this blog, I will document my journey to explore Omniverse Isaac Sim in 2024 and some useful tips and tricks I found along the way. I will keep updating this blog as I learn more about Omniverse Isaac Sim.
Academia Experience
- Designed a graph neural network (GNN) algorithm for fusion of multimodal temporal data
- Analyzed language artifacts in video QA datasets
- Built pipeline for multimodal question answering about social situations
- Coordinated annotation of dataset
- Advisor: Prof. Louis-Philippe Morency
- Designed new algorithms to improve scheduled sampling training for seq2seq models
- Validated effectiveness of the method on NER, Machine Translation and Text Summarization tasks
- Advisor: Prof. Matt Gormley
- Built cardiac arrest prediction pipeline using multimodal temporal data collected from ICU patients
- Advisor: Prof. Jimeng Sun














