This is a part of the official PyTorch implementation of paper:

DexHandDiff: Interaction-aware Diffusion Planning for Adaptive Dexterous Manipulation

Zhixuan Liang, Yao Mu, Yixiao Wang, Tianxing Chen, Wenqi Shao, Wei Zhan, Masayoshi Tomizuka, Ping Luo, Mingyu Ding

CVPR 2025

Project Page | Paper

Strongly suggest to enter the project page to see our demos.

You can directly use the pre-trained models here and perform sampling with specific guides.

• Adroit Door

To use DexHandDiff, you need to install the requirements. We recommend using a virtual environment to avoid conflicts with other packages. You can create a virtual environment and install the requirements as jannerm/diffuser.

To run the sampling, you can use the following command:

python ./scripts/plan_guided.py

You should check the config files in config/ directory to see the available options. The default config file is config/adroit_door.py, which is used for the Adroit Door task. You can modify this option in Line 34 of plan_guided.py to use other config files.

The published checkpoint can be used in Line 135 and 136 of config/adroit_door.py.

All guidance functions are in ./diffuser/sampling/guides.py. And the guidance functions generated by LLM are in ./diffuser/sampling/llmguide.py.

DexHandDiff employs joint state-action diffusion with interaction-aware guidance. Before interaction (top middle), guidance aligns the hand to the object contact point. Upon contact (bottom middle), additional guidance steers both hand and object states towards the goal (“&” means state concatenation at input level), enforcing physical constraints and avoiding ghost states. A learned dynamics model further ensures consistency between states and actions. Our DexHandDiff utilizes extended behavior model to aggregate multiple condition terms to guide the diffusion process.

@InProceedings{Liang_2025_CVPR,
    author    = {Liang, Zhixuan and Mu, Yao and Wang, Yixiao and Chen, Tianxing and Shao, Wenqi and Zhan, Wei and Tomizuka, Masayoshi and Luo, Ping and Ding, Mingyu},
    title     = {DexHandDiff: Interaction-aware Diffusion Planning for Adaptive Dexterous Manipulation},
    booktitle = {Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR)},
    month     = {June},
    year      = {2025},
    pages     = {1745-1755}
}

The diffusion model implementation is based on Michael Janner's diffuser repo.