Huangyue Yu, Baoxiong Jia, Yixin Chen, Yandan Yang, Puhao Li, Rongpeng Su,
Jiaxin Li, Qing Li, Wei Liang, Song-Chun Zhu, Tengyu Liu, Siyuan Huang

We propose METASCENES, a large-scale simulatable 3D scene dataset constructed by replacing objects in real-world 3D scans with realistic and high-quality object assets retrieved or reconstructed from diverse sources.

• [2025-03] Training & Inference code as well as preprocessing code is released!
• [2025-03] We release the MetaScenes dataset. Fill out the form for the download link!
• [2025-02] 🎉MetaScenes is accepted by CVPR 2025! Code and dataset will come shortly, stay tuned!

conda create -n metascenes python=3.9
conda activate metascenes
pip install torch==2.0.1 torchvision==0.15.2 --index-url https://download.pytorch.org/whl/cu118
pip install -r requirements.txt

See DATA.md for detailed instructions on data download, processing, visualization.

Fill out the form for the download!

MetaScenes provides a comprehensive pipeline to construct replica scenes of real-world environments. We provide the code of the following three key preprocessing components:

1. Heuristic-based Room Layout Estimation
2. Object Pose Alignment
3. Physics-based Scene Optimization

See PREPROCESS.md for detailed instructions for setting up the environment, running each component, and understanding the output generated by each process.

Scan2Sim is a multi-modal alignment model designed to retrieve the most optimal asset candidate from a set of candidates, leveraging ground truth optimal asset selection annotations from METASCENES.

See MODEL.md for the inventory of available checkpoints and detailed instructions on training and testing

Some codes are borrowed from ULIP2. We thank all the authors for their great work.

@inproceedings{yu2025metascenes,
  title={METASCENES: Towards Automated Replica Creation for Real-world 3D Scans},
  author={Yu, Huangyue and Jia, Baoxiong and Chen, Yixin and Yang, yandan and Li, Puhao and Su, Rongpeng and Li, Jiaxin and Li, Qing and Liang, Wei and Zhu, Song-Chun and Liu, Tengyu and Huang, Siyuan},
  booktitle=Conference on Computer Vision and Pattern Recognition(CVPR),
  year={2025}
}