Workshop on Real-to-Sim: Bridging the Gap between Neural Rendering and Robot Learning
CVPR 2025 Workshop, Nashville, USA
Introduction
Recent advancements in neural rendering techniques, such as Neural Radiance Fields
(NeRF) and 3D Gaussian Splatting (3DGS), have shown great promise in creating photorealistic 3D
scene reconstructions from real-world data like monocular images and videos. These techniques open
up exciting new possibilities for robot learning by allowing robots to be trained in highly realistic
simulated environments derived from real-world scenes. This emerging “real-to-sim” approach aims to
close the gap between simulation and reality by transitioning from real-world data to reconstructed
neural representations, training robots in these realistic environments, and then deploying them back
into the real world with a minimized sim-to-real gap.
News
Workshop website is launched.
April 15, 2025
Schedule
The workshop will take place on 12 June 2025 from 13:45 - 17:25 CDT.
In the talk session, we will engage different perspectives from experts from diverse communities. In the panel session, we will raise several acute questions that bring debates from different views.
13:45 - 14:00 Welcome & Introduction
14:00 - 14:30 Keynote Talk 1 - Marco Pavone: Building Physical AI with Foundation-Model-Driven Closed-Loop Simulation
14:30 - 15:00 Keynote Talk 2 - Dhruv Shah: Evaluating and Improving Steerability of Generalist Robot Policies
15:00 - 15:30 Thematic Talk - Wayne Wu: Scaling-up Urban Simulation for Autonomous Micro-mobility
16:00 - 16:30 Keynote Talk 3 - Gordon Wetzstein: Long-context and controllable video world models
16:30 - 17:00 Keynote Talk 4 - Lingjie Liu: Towards Next-Gen 3D Reconstruction and Generation: From Visual Fidelity to Multimodal and Physical Understanding
17:00 - 17:25 Panel Discussion
Open Questions
How can neural rendering techniques helps robot learning?
How to extract useful data from real-world environments and apply them to simulation environment?
How do robots leverage hybrid data to reduce the sim-to-real gap ?
