| CARVIEW |
Shibo Zhao
Email: shiboz@andrew.cmu.edu Links: CV | Research Statement
I will be 5th year PhD candidate at CMU Robotics Institute, advised by Prof. Sebastian Scherer.I am also grateful to have wonderful mentors including Professor. Michael Kaess and Professor. Ji Zhang. My research interests include 3D computer vision and robotics. Currently, I am focusing on building spatial intelligence system for robotics, which incorporates a adaptive sensor fusion framework for robust state estimation, a 4D scene representation with flexible levels of detail, and scene reasoning to support VLA navigation. I developed Super Odometry , which is adopted as major methods in various CMU projects to achieve state estimation such as DARPA Subterranean Challenge. I am currently a Research Intern at Apple. I am on the job market and actively seeking full-time research scientist/engineer opportunities starting in 2026.
News
- I am the organizer of Open SuperX SLAM for real-time robotics perception and Tartan SLAM series to invite speakers to share insights. Feel free to reach out if you are interested in my work and want to collaborate :-).
- I was invited to give a talk on IMU Foundation Model at Apple Curpertino, USA 2025.
- I was invited as a author to write a chapter in SLAM Handbook "From Localization and Mapping to Spatial Intelligence", USA 2025
- 1 paper is accepted at ICLR 2025 and 2 papers are accepted at CVPR 2025 and 5 papers are accepted at ICRA2025.
- I'm the organizer of ICCV23 SLAM Challenge and Workshop on Robot Learning and SLAM, at ICCV 2023.
- I was the organizer of Tartan SLAM series at CMU, Robotics Institute, 2022.
- I developed robotic state estimation and mapping algorithms, which enabled a team of robots to explore kilometer-scale underground environments in the DARPA Subterranean Challenge. I helped the team won No.1 in DARPA Tunnel Challenge,No.2 in DARPA Urban Challenge and No.4 in DARPA Final Challenge, USA 2021.
- I'm co-organizing the Robotic Perception and Mapping: Frontier Vision & Learning Techniques at IROS 2023.
- I'm co-organizing the Joint Workshop on Long-Term Visual Localization, Visual Odometry and Geometric and Learning-based SLAM at CVPR 2020.
- Start PhD at CMU, Robotics Institute, advised by Prof. Sebastian Scherer,USA 2021
Research Highlights
How can we enable robots to perceive, adapt, and understand their surroundings like humans, in real-time and under uncertainty? Just as humans rely on vision to navigate complex environments, robots need robust and intelligent perception systems, "eyes" that can endure sensor degradation, adapt to changing conditions, and recover from failure. However, today's visual systems are fragile, easily disrupted by occlusion, lighting variation, motion blur, or dynamic scenes. A single sensor dropout or unexpected environmental change can trigger cascading errors, severely compromising autonomy. My research goal is to develop resilient and intelligent visual perception systems—building the "robust eyes" robots need to thrive in the real world.
How to achieve robust robot perception for anywhere and anytime?
How to capture high quality and divere data to train the model from real-world ( Requires Three Capabilities)?
-
Generalization: How to achieve a unified sensor fusion for different modalities and robots?
- Unified Sensor Fusion: [Super Odometry, IROS2021]
- Multi-robot Deployment:
[Multi-robot Exploration, JFR 2022]
-
Adaptation: How perception system adapts to any enviroments?
- Hierarchical Adaptation:
[Super Odometry2] - + Visual Uncertainty Estimation [Multi-spectral Odometry]
- + LiDAR Uncertainty Estimation: [SuperLoc,ICRA2025]
- Thermal Odometry:
[Deep Thermal Features, IROS2020]
- Hierarchical Adaptation:
-
Scene Understanding: How to understand surrounding environments in real-time?
- Spatial-temporal 4D Mapping: [Super Map]
- + Spatial Temporal Tracking:
[Wild-4D SLAM] - Semantic Prior: [LIOM, IROS2020]
How can we efficiently learn spatial and semantic knowlege from above real-world data ?
How can we leverage the strength of data-driven methods (eg: foundation models) and optimization methods?
-
Foundation Models for Robotics
- IMU Foundation Model: [Tartan IMU,CVPR2025]
- + IMU Feature Observibility:
[AirIO]
-
Differentiable Optimization
- Differentiable Factor Graph: [Pypose,CVPR2023]
- Distributed Factor Graph Optimization: [SuperLoop]
- Multi Robot on Gaussian Splatting: [MAC-Ego3D, CVPR2025]
-
Scalable Evaluation
- Robustness Evaluation on Odometry: [SubT-MRS,CVPR2024]
- Robustness Evaluation on Reconstruction: [Reconstruction from Noisy Video, ICLR 2025]
Publications ( show selected / show all by date / show all by topic )
Topics:
Generalization /
Differentiable Optimization /
Adaptation /
Scene Understanding
Scalable Evaluation /
Foundation Models
(*/†: indicates equal contribution.)
AirIO: Learning Inertial Odometry with Enhanced IMU Feature Observability
Yuheng Qiu, Can Xu, YuTian Chen, Shibo Zhao, Junyi Geng, Sebastian Scherer
Scalable Benchmarking and Robust Learning for Noise-Free Ego-Motion and 3D Reconstruction from Noisy Video
Xiaohao Xu, Tianyi Zhang, Shibo Zhao, Xiang Li, Sibo Wang, Yongqi Chen, Ye Li, Bhiksha Raj, Matthew Johnson-Roberson, Sebastian Scherer, Xiaonan Huang
Tartan IMU: A Light Foundation Model for Inertial Positioning in Robotics
Shibo Zhao, Sifan Zhou, Raphael Blanchard, Yuheng Qiu, Wenshan Wang, Sebastian Scherer
MAC-Ego3D: Multi-Agent Gaussian Consensus for Real-Time Collaborative Ego-Motion and Photorealistic 3D Reconstruction
Xiaohao Xu, Feng Xue, Shibo Zhao, Yike Pan, Sebastian Scherer, Xiaonan Huang
SuperOdom2.0: Hierarchical Adaptation Enables Robust Odometry Towards All-degraded Environments
Shibo Zhao, Sifan Zhou, Yuchen Zhang, Ji Zhang, Chen Wang, Wenshan Wang, Sebastian Scherer
SuperMap: Spatio Temporal Semantic SLAM Enabling Robots to Understand Evolving World in Real Time
Shibo Zhao, Guofei Chen, Honghao Zhu, Zhiheng Li, Changwei Yao, Nader Zantout, Seungchan Kim, Wenshan Wang,
MSO: Uncertainty-aware Multi-spectral Inertial Odometry
Shibo Zhao, Parv Maheshwari, Tianhao Wu, Yao He, Mukai Yu, Andrew Jong, Aayush Fadia, Ranai Srivastav, Wenshan Wang, Sebastian Scherer
SuperLoc: The Key to Robust LiDAR-Inertial Localization Lies in Predicting Alignment Risks
Shibo Zhao, Honghao Zhu, Yuanjun Gao, Beomsoo Kim, Yuheng Qiu, Aaron M. Johnson, Sebastian Scherer
SuperLoop: Distributed Multi-robot SLAM System with Online Place Recognition
Shibo Zhao, Jay Karhade, Damanpreet Singh, Mansi Sarawata, Sebastian Scherer,
SubT-MRS Dataset: Pushing SLAM Towards All-weather Environments
Shibo Zhao, Yuanjun Gao, Tianhao Wu, Damanpreet Singh, Rushan Jiang, Haoxiang Sun, Mansi Sarawata, Yuheng Qiu, Warren Whittaker, Ian Higgins, Yi Du, Shaoshu Su, Can Xu, John Keller, Jay Karhade, Lucas Nogueira, Sourojit Saha, Ji Zhang, Wenshan Wang, Chen Wang, Sebastian Scherer
Present and Future of SLAM in Extreme Underground Environments: The DARPA SubT Challenge
Kamak Ebadi, Lukas Bernreiter, Harel Biggie, Gavin Catt, Yun Chang, Shibo Zhao, Sebastian Scherer, Luca Carlone, et al.
PyPose: A Library for Robot Learning with Physics-based Optimization
Chen Wang, Dasong Gao, Kuan Xu, Junyi Geng, Yaoyu Hu, Yuheng Qiu, Bowen Li, Fan Yang, Brady Moon, Abhinav Pandey, Aryan, Jiahe Xu, Tianhao Wu, Haonan He, Daning Huang, Zhongqiang Ren, Shibo Zhao, Taimeng Fu, Pranay Reddy, Xiao Lin, Wenshan Wang, Jingnan Shi, Rajat Talak, Kun Cao, Yi Du, Han Wang, Huai Yu, Shanzhao Wang, Siyu Chen, Ananth Kashyap, Rohan Bandaru, Karthik Dantu, Jiajun Wu, Lihua Xie, Luca Carlone, Marco Hutter, Sebastian Scherer
Super Odometry: IMU-centric LiDAR-Visual-Inertial Estimator for Challenging Environments
Shibo Zhao, Hengrui Zhang, Peng Wang, Lucas Nogueira, Sebastian Scherer
TP-TIO: A Robust Thermal-Inertial Odometry with Deep ThermalPoint
Shibo Zhao, Peng Wang, Hengrui Zhang, Zheng Fang, Sebastian Scherer
A Robust Laser-Inertial Odometry and Mapping Method for Large-Scale Highway Environments
Shibo Zhao, Zheng Fang, HaoLai Li, Sebastian Scherer
Dedicated to my best friend Zhaoyi