NeRF4ADR:
Neural Fields for Autonomous Driving and Robotics

Introduction

Neural fields, or coordinate-based neural networks, have emerged as novel representations for a myriad of signals, including but not limited to 3D geometry, images, and sound. This innovative approach has seen great progress in various computer vision and computer graphics tasks, particularly in novel view synthesis and scene reconstruction. However, the incorporation of neural fields into autonomous driving and robotics is still in its early stages. Now, an intriguing question has surfaced — can these novel representations inform continuous decision-making in autonomous driving and robotics? Recent efforts indeed provide promising evidence. For instance, 3D neural fields enhance multi-view consistency, which in turn significantly improves scene understanding for robotics and autonomous driving. To accelerate advancements in the application of neural fields in robotics, this workshop aspires to serve as a confluence of researchers from various disciplines such as machine learning, computer vision, computer graphics, autonomous driving, and robotics. The objectives of this gathering are to exchange ideas, discuss the current progress of neural fields in autonomous driving and robotics, explore potential areas in autonomous driving and robotics that are ripe for the adoption of neural fields, underscore the limitations of neural fields, and discuss potential failure cases in this sector. Moreover, it provides an opportunity for the ICCV community to deliberate over this exciting and growing domain of neural fields for autonomous driving and robotics.

TL;DR

Neural fields have made significant advancements in computer vision and computer graphics but are yet to be fully explored in autonomous driving and robotics. This workshop aims to bring together experts from various fields to discuss the potential of neural fields in these sectors, exchange ideas, identify limitations, and address failure cases.

Call for Papers

Style and Author Instructions

All submissions should anonymized. Papers with more than 4 pages (excluding references) will be reviewed as long papers, and papers with more than 8 pages (excluding references) will be rejected without review. Supplementary material is optional with supported formats: pdf, mp4 and zip. All papers that were not previously presented in a major conference, will be peer-reviewed by three experts in the field in a double-blind manner. In case you are submitting a previously accepted conference paper, please also attach a copy of the acceptance notification email in the supplementary material documents.

All submissions should adhere to the ICCV 2023 author guidelines.

Submission Portal: https://cmt3.research.microsoft.com/NeRF4ADR2023/Submission/Index

Paper Review Timeline:

Invited Speakers

Jiajun Wu

Stanford University

Jiajun Wu is an Assistant Professor of Computer Science at Stanford University, affiliated with the Stanford AI Lab (SAIL) and the Stanford Vision and Learning Lab (SVL). His research focuses on machine perception, reasoning, and interaction with the physical world, drawing inspiration from human cognition. His current research topics include Physical Scene Understanding, Dynamics Models, Neuro-Symbolic Visual Reasoning, Generative Visual Models, and Multi-Modal Perception. Before joining Stanford, he was a Visiting Faculty Researcher at Google Research, New York City, and completed his PhD at MIT.

Jon Barron

Google Research

Jon Barron is a senior staff research scientist at Google Research in San Francisco, where he works on computer vision and machine learning. He received a PhD in Computer Science from the University of California, Berkeley in 2013, where he was advised by Jitendra Malik, and he received an Honours BSc in Computer Science from the University of Toronto in 2007. He received a National Science Foundation Graduate Research Fellowship in 2009, the C.V. Ramamoorthy Distinguished Research Award in 2013, the PAMI Young Researcher Award in 2020. His works have received awards at ECCV 2016, TPAMI 2016, ECCV 2020, ICCV 2021, CVPR 2022, and the Communications of the ACM (2022).

Luca Carlone

Massachusetts Institute of Technology

Luca Carlone is an Associate Professor at MIT AeroAstro. His work includes seminal results on certifiably correct algorithms for localization and mapping, as well as approaches for visual-inertial navigation and distributed mapping. He is a recipient of the Best Student Paper Award at IROS 2021, the Best Paper Award in Robot Vision at ICRA 2020, a 2020 Honorable Mention From the IEEE Robotics and Automation Letters, a Track Best Paper award at the 2021 IEEE Aerospace Conference, the 2017 Transactions on Robotics King-Sun Fu Memorial Best Paper Award, the Best PaperAward at WAFR 2016, the Best Student Paper Award at the 2018 Symposium on VLSI Circuits, and he was best paper finalist at RSS2015 and RSS 2021.

Lingjie Liu

University of Pennsylvania

Lingjie Liu is the Aravind K. Joshi Assistant Professor at the University of Pennsylvania, leading the Penn Computer Graphics Lab and affiliated with the General Robotics, Automation, Sensing & Perception Lab. Her research is at the crossroads of Computer Graphics, Computer Vision, and AI, with a particular focus on Neural Scene Representations and 3D Reconstruction. Prior to this, she was a Lise Meitner Postdoctoral Research Fellow at the Max Planck Institute for Informatics, and earned her Ph.D. from the University of Hong Kong in 2019.

Andrew Davison

Imperial College London

Andrew Davison is a Professor of Robot Vision at Imperial College London, leading the Dyson Robotics Laboratory. He pioneers in SLAM (Simultaneous Localisation and Mapping) using vision, significantly contributing to real-time 3D vision and robotics. Andrew also co-founded SLAMcore, a start-up specializing in applied Spatial AI solutions. His current research extends into "Spatial AI," aiming to enhance real-time 3D vision's performance across various dimensions like dynamics, scale, and efficiency.

Vincent Sitzmann

Massachusetts Institute of Technology

Vincent Sitzmann is an Assistant Professor at MIT EECS, where he leads the Scene Representation Group. His research interest lies in neural scene representations, focusing on how neural networks learn to represent information about our world. Vincent's goal is to allow independent agents to reason about our world based on visual observations, such as inferring a complete model of a scene with information on geometry, material, lighting, etc., from only a few observations, a task that is currently challenging for AI.

Jamie Shotton

Wayve

Jamie Shotton serves as Chief Scientist at Wayve. He has a distinguished career, having worked as Partner Director of Science at Microsoft earlier. His work significantly contributed to Kinect and HoloLens technologies. Jamie has received numerous accolades including the Royal Academy of Engineering's MacRobert and Silver Medals. He holds a PhD in Computer Vision and Machine Learning from the University of Cambridge. His interests span artificial intelligence, computer vision, machine learning, and autonomous driving among others

Schedule