Generalizable Priors for Robot Manipulation

Overview

Generalist robot policies are those capable of performing complex manipulation tasks across a wide range of environments. Recent years have seen significant progress toward this goal, driven by advances in large-scale teleoperated robot datasets, structured representations for policy learning, hierarchical planning with vision-language models (VLMs), and online learning for adaptation to novel tasks and environments. Although multiple promising approaches are emerging, it is essential to understand the tradeoffs inherent in each to develop methods that not only generalize to new scenarios but also execute tasks with high precision and reliability.

An ideal framework for learning such generalizable policies should: (1) scale beyond simple pick-and-place tasks and continually improve as more task data becomes available, (2) learn effectively from diverse data sources, including robot teleoperation, simulated environments, and human demonstration videos, and (3) utilize representations of the world that are applicable across tasks requiring varying levels of precision and dexterity. This workshop will focus on a central question: What are the right priors for generalizable policy learning, and how can we best incorporate these priors into policy learning frameworks?

Our speakers and panelists are leading researchers in robotics and machine learning, working at the forefront of topics including end-to-end control, sim-to-real transfer, learning from human videos, and large-scale robotic data collection, among others. We invite the community to submit their latest work and ideas for discussion.

Areas of Interest

We aim to investigate the following topics and research questions:

• What main bottlenecks prevent current robot policies from generalizing to unseen environments?
  • Is the challenge primarily a data problem, or are there fundamental limitations in current approaches?
• Can pretrained models from related fields, such as vision and language, be leveraged to improve robot policies?
  • What is the most effective way to incorporate these models into robotics?
  • Should we fine-tune them on robot data, or use their representations and outputs for downstream policy learning?
• How can we best utilize simulated environments to promote generalization?
  • Do simulated environments need to closely resemble the real world for successful policy transfer, or can they be substantially different?
  • What role do generative models play in the sim-to-real pipeline? Are they consistent enough to make reliable long-term predictions over full trajectories?
• Is it possible to design universal priors that work across a wide range of tasks and domains, including simulation, the real world, and internet videos?
  • Can object-centric representations provide this capability?
  • Can 3D representations facilitate cross-domain transfer?
• How should we design standardized benchmarks for evaluating such generalist policies?

Schedule

Presentation Instructions

Each poster panel will be shared between two papers (arranged side-by-side, vertically). Please make sure your poster does not exceed 0.92 m (H) × 0.94 m (W) (close to A0 portrait, but slightly scaled down).

Each oral presentation will be a 5-minute spotlight (4 minutes for the talk and 1 minute for Q&A).

All accepted papers will be presented as posters.

Invited Speakers

Jeannette Bohg

Stanford University
USA

Ajay Mandlekar

NVIDIA
USA

Xiaolong Wang

University of California, San Diego
USA

Jiafei Duan

University of Washington / AI2
USA

Yang Gao

Tsinghua University
China

Georgia Chalvatzaki

TU Darmstadt
Germany

Harold Soh

National University of Singapore
Singapore

Edward Johns

Imperial College London
UK

Accepted Papers

Organizing Committee

Siddhant Haldar

New York University
USA

Mara Levy

University of Maryland
USA

Jiafei Duan

University of Washington
USA

Ivan Kapelyukh

Imperial College London
UK

Abhishek Gupta

University of Washington
USA