Hongyu Shen 1,2,*
Junfeng Ni2,3,*,
Yixin Chen✉2,
Weishuo Li2,
Mingtao Pei1,
Siyuan Huang✉2
✉ indicates corresponding author
* these authors contributed equally to this work
1Beijing Institute of Technology
2State Key Laboratory of General Artificial Intelligence, BIGAI
3Tsinghua University
Trace3D leverages the proposed Gaussian Instance Tracing to enhance multi-view consistency and reduce ambiguous Gaussians, resulting in high-quality 3D instance segmentation.
- Tested System: Ubuntu 22.04, CUDA 11.8
- Tested GPUs: RTX4090
- Basic environment
conda create -n trace3d python=3.10
conda activate trace3d
pip install torch==2.0.1 torchvision==0.15.2 --index-url https://download.pytorch.org/whl/cu118
pip install -r requirements.txt- SAM for segmentation
git clone https://github.com/facebookresearch/segment-anything.git
cd segment-anything
pip install -e .
mkdir sam_ckpt; cd sam_ckpt
wget https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth data
├── nerf_llff_data # Link: https://drive.google.com/drive/folders/14boI-o5hGO9srnWaaogTU5_ji7wkX2S7
│ └── [fern|flower|fortress|horns|leaves|orchids|room|trex]
│ ├── [sparse/0] (colmap results)
│ └── [images|images_2|images_4|images_8]
│
└── replica # Link: https://www.dropbox.com/sh/9yu1elddll00sdl/AAC-rSJdLX0C6HhKXGKMOIija?dl=0
└── [office_0|room_0|...]
├── traj_w_c.txt
├── [sparse/0] (colmap results)
└── [rgb|depth|sam|]Get SAM masks
python get_sam_masks.py --sam_checkpoint {SAM_CKPT_PATH} --file_path {IMAGE_FOLDER}Before running: please specify the information in the scripts (e.g. replica.sh). More options can be found in conf/ and arguments/ and you can them adjusted in config file.
#--- Edit the config file replica.sh
dataset=replica_900
path=./data/${dataset}
scene='room_0' Scene reconstruction
bash replica.sh train_rgbMerge patch masks
bash replica.sh merge_patchesDelete Ambiguous Gaussians
bash replica.sh remove_ab_gausContrastive lifting
bash replica.sh train_contra3D Object Extraction
bash replica.sh eval_3dNovel View 2D Instance Segmentation
bash replica.sh eval Some codes are borrowed from Egolifter, SA3D, Omniseg3D, FlashSplat and Gaussian-Editor. We thank all the authors for their great work.
@inproceedings{shen2025trace3d,
title={Trace3D: Consistent Segmentation Lifting via Gaussian Instance Tracing},
author={Shen, Hongyu and Ni, Junfeng, and Chen, Yixin and Li, Weishuo and Pei, Mingtao and Huang, Siyuan},
booktitle=ICCV,
year={2025}
}