Abstract

We introduce DexYCB, a new dataset for capturing hand grasping of objects. We first compare DexYCB with a related one through cross-dataset evaluation. We then present a thorough benchmark of state-of-the-art approaches on three relevant tasks: 2D object and keypoint detection, 6D object pose estimation, and 3D hand pose estimation. Finally, we evaluate a new robotics-relevant task: generating safe robot grasps in human-to-robot object handover.

Paper & Document

PDF

Supplementary

arXiv

Studio CAD Model

Studio Hardware

RealSense Calibration & Recording Guide

@INPROCEEDINGS{chao:cvpr2021,
  author    = {Yu-Wei Chao and Wei Yang and Yu Xiang and Pavlo Molchanov and Ankur Handa and Jonathan Tremblay and Yashraj S. Narang and Karl {Van Wyk} and Umar Iqbal and Stan Birchfield and Jan Kautz and Dieter Fox},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  title     = {{DexYCB}: A Benchmark for Capturing Hand Grasping of Objects},
  year      = {2021},
}

Data

DexYCB is licensed under CC BY-NC 4.0.

We provide two options for downloading the dataset:

1. A single compressed file that contains everything: dex-ycb-20210415.tar.gz (119G). Download this file to a new folder and extract it with:

   tar zxvf dex-ycb-20210415.tar.gz

2. A breakdown to multiple compressed files, each containing a sub-folder. Download all the files below to a new folder and extract them individually:

   • 20200709-subject-01.tar.gz (12G)
   • 20200813-subject-02.tar.gz (12G)
   • 20200820-subject-03.tar.gz (12G)
   • 20200903-subject-04.tar.gz (12G)
   • 20200908-subject-05.tar.gz (12G)
   • 20200918-subject-06.tar.gz (12G)
   • 20200928-subject-07.tar.gz (12G)
   • 20201002-subject-08.tar.gz (12G)
   • 20201015-subject-09.tar.gz (12G)
   • 20201022-subject-10.tar.gz (12G)
   • bop.tar.gz (1.2G)
   • calibration.tar.gz (16K)
   • models.tar.gz (1.4G)

Once you successfully download and extract the dataset, you should have a folder with the following structure:

├── 20200709-subject-01/
├── 20200813-subject-02/
├── ...
├── calibration/
└── models/

For instructions about using the dataset please see dex-ycb-toolkit.

Code

dex-ycb-toolkit
A Python package that provides evaluation and visualization tools for the DexYCB dataset.

Video

Contact

Send any comments or questions to Yu-Wei Chao: ychao@nvidia.com.