CARVIEW

MOTORHOMES

Select Language

HTTP/2 200 server: GitHub.com content-type: text/html; charset=utf-8 last-modified: Wed, 06 Aug 2025 06:17:58 GMT access-control-allow-origin: * strict-transport-security: max-age=31556952 etag: W/"6892f396-51c9" expires: Mon, 29 Dec 2025 21:33:27 GMT cache-control: max-age=600 content-encoding: gzip x-proxy-cache: MISS x-github-request-id: EE70:2C10E1:9594D5:A7C909:6952F14F accept-ranges: bytes age: 0 date: Mon, 29 Dec 2025 21:23:27 GMT via: 1.1 varnish x-served-by: cache-bom-vanm7210096-BOM x-cache: MISS x-cache-hits: 0 x-timer: S1767043408.505655,VS0,VE268 vary: Accept-Encoding x-fastly-request-id: 29a5e4c891f3976aef7edaae54b2205ee66b5fee content-length: 4857 Trace3D

Trace3D: Consistent Segmentation Lifting via Gaussian Instance Tracing

ICCV 2025

Hongyu Shen^1,2*, Junfeng Ni^2,3*, Yixin Chen^2✉, Weishuo Li², Mingtao Pei¹, Siyuan Huang^2✉

¹Beijing Institute of Technology ²State Key Laboratory of General Artificial Intelligence, BIGAI ³Tsinghua University

* Equal contribution ✉ Corresponding author

Paper Code

Trace3D leverages the proposed Gaussian Instance Tracing to enhance multi-view consistency and reduce ambiguous Gaussians, resulting in high-quality 3D instance segmentation.

We visualize the feature map from PCA and hierarchical segmentations in each scene, facilitating object extraction and scene editing applications, e.g., Captain America wielding Thor's hammer.

Abstract

We address the challenge of lifting 2D visual segmentation to 3D in Gaussian Splatting. Existing methods often suffer from inconsistent 2D masks across viewpoints and produce noisy segmentation boundaries as they neglect these semantic cues to refine the learned Gaussians. To overcome this, we introduce Gaussian Instance Tracing (GIT), which augments the standard Gaussian representation with an instance weight matrix across input views. Leveraging the inherent consistency of Gaussians in 3D, we use this matrix to identify and correct 2D segmentation inconsistencies. Furthermore, since each Gaussian ideally corresponds to a single object, we propose a GIT-guided adaptive density control mechanism to split and prune ambiguous Gaussians during training, resulting in sharper and more coherent 2D and 3D segmentation boundaries. Experimental results show that our method extracts clean 3D assets and consistently improves 3D segmentation in both online (e.g., self-prompting) and offline (e.g., contrastive lifting) settings, enabling applications such as hierarchical segmentation, object extraction, and scene editing.

Mask inconsistency

The multi-view inconsistent 2D segmentation masks introduce significant ambiguity in both feature lifting and mask prediction.

Ambiguous gaussian

Gaussians nearby the edge often belong to both foreground and background at the same time, making precise 3D segmentation difficult.

Method

We augments each 3D Gaussian with an instance weight matrix across views to capture its association with object instances. Leveraging the 3D consistency of Gaussians, GIT identifies and corrects 2D segmentation inconsistencies. Additionally, GIT-guided adaptive density control mechanism can split or prune ambiguous Gaussians, resuling clean and consistent instance boundaries in both 2D and 3D.

Results

3D Object Extraction

Results on Replica demonstrate that our method not only accurately extracts the target object but also significantly reduces artifacts compared to the baselines.

Novel View 2D Instance Segmentation

Our method also achieves higher-quality 2D instance segmentation results on both the Replica and NVOS datasets compared to other baselines. Our improvements in consistent instance maps and the reduction of ambiguous Gaussians yield more precise novel-view instance segmentation and feature maps, even for some tiny structures.

Replica

Novel view synthesis of 2D segmentation on Replica dataset. The results of each baseline provide the segmentation on the left and the feature map from PCA on the right. The TP, FP, and FN predictions are color-coded in the segmentation.

NVOS

Novel view synthesis of 2D segmentation on NVOS dataset. The TP, FP, and FN predictions are color-coded in the segmentation. Our results with GIT produce fewer false positive predictions that are hard to observe from the input view.

Part-level Segmentation

The videos below show how our method enables clean 3D segmentation and flexible object extraction, supporting various downstream applications.

BibTeX

@inproceedings{shen2025trace3d,
  title={Trace3D: Consistent Segmentation Lifting via Gaussian Instance Tracing},
  author={Shen, Hongyu and Ni, Junfeng and Chen, Yixin and Li, Weishuo and Pei, Mingtao and Huang, Siyuan},
  booktitle=ICCV,
  year={2025}
}

Original Source | Taken Source