GitHub - meta-scenes/MetaScenes (original) (raw)

METASCENES: Towards Automated Replica Creation for Real-world 3D Scans

CVPR 2025

Huangyue Yu,Baoxiong Jia,Yixin Chen,Yandan Yang,Puhao Li,Rongpeng Su,
Jiaxin Li,Qing Li,Wei Liang,Song-Chun Zhu,Tengyu Liu,Siyuan Huang

We propose METASCENES, a large-scale simulatable 3D scene dataset constructed by replacing objects in real-world 3D scans with realistic and high-quality object assets retrieved or reconstructed from diverse sources.

News

• [2025-03] Training & Inference code as well as preprocessing code is released!
• [2025-03] We release the MetaScenes dataset. Fill out the form for the download link!
• [2025-02] 🎉MetaScenes is accepted by CVPR 2025! Code and dataset will come shortly, stay tuned!

⚙️Installation

conda create -n metascenes python=3.9 conda activate metascenes pip install torch==2.0.1 torchvision==0.15.2 --index-url https://download.pytorch.org/whl/cu118 pip install -r requirements.txt

🏘️Data

MetaScenes Dataset

See DATA.md for detailed instructions on data download, processing, visualization.

Fill out the form for the download!

MetaScenes Preprocessing

MetaScenes provides a comprehensive pipeline to construct replica scenes of real-world environments. We provide the code of the following three key preprocessing components:

1. Heuristic-based Room Layout Estimation
2. Object Pose Alignment
3. Physics-based Scene Optimization

See PREPROCESS.md for detailed instructions for setting up the environment, running each component, and understanding the output generated by each process.

🔥Scan2Sim

Scan2Sim is a multi-modal alignment model designed to retrieve the most optimal asset candidate from a set of candidates, leveraging ground truth optimal asset selection annotations from METASCENES.

See MODEL.md for the inventory of available checkpoints and detailed instructions on training and testing

😃Acknowledgements

Some codes are borrowed from ULIP2. We thank all the authors for their great work.

Citation

@inproceedings{yu2025metascenes, title={METASCENES: Towards Automated Replica Creation for Real-world 3D Scans}, author={Yu, Huangyue and Jia, Baoxiong and Chen, Yixin and Yang, yandan and Li, Puhao and Su, Rongpeng and Li, Jiaxin and Li, Qing and Liang, Wei and Zhu, Song-Chun and Liu, Tengyu and Huang, Siyuan}, booktitle=Conference on Computer Vision and Pattern Recognition(CVPR), year={2025} }