GitHub - trace-3d/Trace3D: Official implementation of ICCV25 paper "Trace3D: Consistent Segmentation Lifting via Gaussian Instance Tracing" (original) (raw)

Hongyu Shen 1,2,* Junfeng Ni2,3,*,Yixin Chen✉2,Weishuo Li2,Mingtao Pei1,Siyuan Huang✉2
✉ indicates corresponding author * these authors contributed equally to this work 1Beijing Institute of Technology
2State Key Laboratory of General Artificial Intelligence, BIGAI 3Tsinghua University

Trace3D leverages the proposed Gaussian Instance Tracing to enhance multi-view consistency and reduce ambiguous Gaussians, resulting in high-quality 3D instance segmentation.

Installation

Tested System: Ubuntu 22.04, CUDA 11.8
Tested GPUs: RTX4090

Basic environment

conda create -n trace3d python=3.10 conda activate trace3d pip install torch==2.0.1 torchvision==0.15.2 --index-url https://download.pytorch.org/whl/cu118 pip install -r requirements.txt

SAM for segmentation

git clone https://github.com/facebookresearch/segment-anything.git cd segment-anything pip install -e . mkdir sam_ckpt; cd sam_ckpt wget https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth

Data

data
├── nerf_llff_data  # Link: https://drive.google.com/drive/folders/14boI-o5hGO9srnWaaogTU5_ji7wkX2S7
│   └── [fern|flower|fortress|horns|leaves|orchids|room|trex]
│       ├── [sparse/0] (colmap results)
│       └── [images|images_2|images_4|images_8]
│
└── replica		# Link: https://www.dropbox.com/sh/9yu1elddll00sdl/AAC-rSJdLX0C6HhKXGKMOIija?dl=0
    └── [office_0|room_0|...]
        ├── traj_w_c.txt
        ├── [sparse/0] (colmap results)
        └── [rgb|depth|sam|]

Training

Get SAM masks

python get_sam_masks.py --sam_checkpoint {SAM_CKPT_PATH} --file_path {IMAGE_FOLDER}

Before running: please specify the information in the scripts (e.g. replica.sh). More options can be found in conf/ and arguments/ and you can them adjusted in config file.

#--- Edit the config file replica.sh dataset=replica_900 path=./data/${dataset} scene='room_0'

Scene reconstruction

bash replica.sh train_rgb

Merge patch masks

bash replica.sh merge_patches

Delete Ambiguous Gaussians

bash replica.sh remove_ab_gaus

Contrastive lifting

bash replica.sh train_contra

Evaluation

3D Object Extraction

Novel View 2D Instance Segmentation

Acknowledgements

Some codes are borrowed from Egolifter, SA3D, Omniseg3D, FlashSplat and Gaussian-Editor. We thank all the authors for their great work.

Citation

@inproceedings{shen2025trace3d, title={Trace3D: Consistent Segmentation Lifting via Gaussian Instance Tracing}, author={Shen, Hongyu and Ni, Junfeng, and Chen, Yixin and Li, Weishuo and Pei, Mingtao and Huang, Siyuan}, booktitle=ICCV, year={2025} }