ICRA 2025
Haoxiao Wang¹, Kaichen Zhou¹*, Binrui Gu¹, Zhiyuan Feng², Weijie Wang³, Peilin Sun³, Yicheng Xiao⁴, Jianhua Zhang¹, Hao Dong¹*
¹Peking University, ²Tsinghua University, ³Zhejiang University, ⁴Southeast University
*equal contributions, *corresponding author
Installation • Dataset • Training • Testing • Results • Citation
Download the ClearGrasp dataset from ClearGrasp. This dataset contains RGB-D images of transparent objects for depth completion and manipulation tasks.
data/
├── cleargrasp/
│ ├── train/
│ │ ├── rgb/
│ │ ├── depth/
│ │ ├── mask/
│ │ └── init_depth/
│ └── test/
│ ├── rgb/
│ ├── depth/
│ ├── mask/
│ └── init_depth/
└── data_json/
└── cleargrasp_train_0_1.json
Our released implementation is tested on:
- Ubuntu 20.04 / Ubuntu 22.04
- Python 3.10.x
- NVIDIA CUDA 12.4
- 8x NVIDIA GTX 4090 / 8x NVIDIA A100 RTX GPUs
conda create -n transdiff python=3.10
conda activate transdiff
pip install -r requirements.txtUse the provided script for ClearGrasp dataset:
cd src
chmod +x run_cleargrasp.sh
./run_cleargrasp.shScript contents:
#!/bin/bash
python main.py \
--dir_data DATA_PATH \
--data_name CLEARGRASP \
--split_json DATA_JSON_PATH \
--patch_height 144 --patch_width 256 \
--gpus 0,1,2,3,4,5,6,7 \
--loss "1.0*L1+1.0*L2+1.0*DDIM" \
--epochs 30 \
--batch_size 64 \
--max_depth 1.5 \
--save CLEARGRASP_results \
--model_name Transdiff_Diffusion \
--backbone_module swin \
--backbone_name swin_large_naive_l4w722422k \
--head_specify DDIMDepthEstimate_Swin_ADDHAHI \python main.py \
--dir_data DATA_PATH \
--data_name CLEARGRASP \
--split_json DATA_JSON_PATH \
--patch_height 144 --patch_width 256 \
--gpus 0 \
--max_depth 1.5 \
--batch_size 1 \
--test_only \
--pretrain path/to/trained/model.pt \
--save test_results \
--save_image \
--model_name Transdiff_Diffusion \
--backbone_module swin \
--backbone_name swin_large_naive_l4w722422k \
--head_specify DDIMDepthEstimate_Swin_ADDHAHI1. MMDetection Installation:
# If mmdet installation fails, try:
pip install -U openmim
mim install mmengine
mim install "mmcv>=2.0.0"
mim install mmdet2. OpenEXR Installation:
# On Ubuntu/Debian:
sudo apt-get install libopenexr-dev
# On CentOS/RHEL:
sudo yum install OpenEXR-devel
# Using conda:
conda install -c conda-forge openexr-pythonIf you find this work useful in your research, please cite:
@article{wang2025transdiff,
title={Transdiff: Diffusion-based method for manipulating transparent objects using a single rgb-d image},
author={Wang, Haoxiao and Zhou, Kaichen and Gu, Binrui and Feng, Zhiyuan and Wang, Weijie and Sun, Peilin and Xiao, Yicheng and Zhang, Jianhua and Dong, Hao},
journal={arXiv preprint arXiv:2503.12779},
year={2025}
}