4D Hand Object Interaction:

Video

The whole workshop video is here.

Introduction

Hand-object interactions (HOI) feature regularly in our daily activities in which we use one hand or two to manipulate objects directly or to use various tools. Through HOI, humans excel in manipulation tasks, master new skills, and adapt to complex and continuously changing environments. Recently, there is a surge of interest in understanding and generating hand-object interaction, to support applications in robotics, augmented reality, and other important fields. Such applications usually require a detailed understanding of hands dynamically interacting with a wide range of objects in 4D (3D space + 1D time), and struggle to synthesize or execute dexterous manipulations in 4D at the same level of skill as humans.

The key goal of this workshop is to assemble experts in 4D vision, hand-object interaction, dexterous manipulation, and animation synthesis to synchronize and coordinate the efforts. First of all, the workshop would help researchers working on 4D HOI understanding to know about dexterous manipulation and tailor their research topics toward interaction-oriented 4D understanding. Secondly, embodied AI researchers could improve their understanding of the limitation of current 4D HOI perception and design their methods properly. Finally, in the workshop we will also host a competition designed for benchmarking the progress of 4D HOI geometric understanding, to invite more researchers to the field.

We have invited a few leading researchers with extensive related experience to make keynote presentations at our workshop. We will also organize a competition to benchmark the current progress of 4D HOI geometric understanding and to also encourage more researchers to join the field. We will organize a panel discussion at the conclusion of our full-day workshop. We envision that such discussions will inspire people and stimulate novel research for 4D hand-object interaction.

Topics

In this workshop, we will encourage participating researchers to focus on topics below:

• HOI 4D reconstruction
• HOI pose tracking in 4D
• 4D semantic scene understanding
• Action recognition and segmentation
• Multi-hand/multi-object HOI
• Multimodal HOI perception
• HOI synthesis
• Dexterous manipulation of rigid/articulated/deformable objects
• Generalizable robot learning
• Learning from demonstrations
• Applications in AR/VR/robotics

HOI4D Challenge Timeline (11:59 PM Pacific Standard Time)

We host an online leaderboard(https://hoi4d.top) through which participants could indicate the track they are working on and submit their results. We will evaluate the submissions on our server and update the leaderboard accordingly

Workshop Schedule (PDT)

Organizers

Li Yi

Tsinghua University

He Wang

Peking University

Xiaolong Wang

UC San Diego

Yu-Wei Chao

NVIDIA

David Fouhey

University of Michigan

Sifei Liu

NVIDIA

Yunze Liu

Tsinghua University

Hao Su

UC San Diego

Shubham Tulsiani

CMU

Contact