You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Please follow the LaMa's guidance to config the environment and download the pre-trained checkpoint. Then please refer to the following scripts to in-paint the background.
# step 1: prepare the images and corresponding masks.
python preprocess/scripts/prepare_lamain_xxx.py # Please modify the data path first.
# step 2: run LaMa to in-paint the background.
bash preprocess/scripts/run_lama.sh
# step 3:
python preprocess/scripts/rename_lamaout.py
Editable Novel View Synthesis
Please download our pre-trained checkpoints, and put the folder in udc-nerf/pretrained_ckpts. The following scripts can be used to generate demo videos in debug/rendered_view/render_xxx_edit/.
python scripts/edit_toydesk2.sh # for toydesk2, takes more than 1.5h.
python scripts/edit_scannet0113_multi.sh # for scannet0113_multi, takes about 30min.
Training
In our experiments, we used two scenes in ToyDesk Dataset and four scenes in ScanNet Dataset, i.e, 0024, 0038, 0113, 0192. The following scripts are two examples, and please refer to the training scripts in scripts/ for more details.
python scripts/train_toydesk2.sh # for toydesk2
python scripts/train_scannet0113_multi.sh # for scannet0113_multi
Evaluation
The following scripts are two examples, and please refer to the evaluation scripts in scripts/ for more details.
python scripts/test_toydesk2.sh # for toydesk2
python scripts/test_scannet0113_multi.sh # for scannet0113_multi
Acknowledgements
This project is built upon Object-NeRF. The in-painted images are obtained by LaMa. Kudos to these researchers.
Citation
@inproceedings{wang2023udcnerf,
title={Learning Unified Decompositional and Compositional NeRF for Editable Novel View Synthesis},
author={Wang, Yuxin and Wu, Wayne and Xu, Dan},
booktitle={ICCV},
year={2023}
}
About
Official code for ICCV2023 paper: Learning Unified Decompositional and Compositional NeRF for Editable Novel View Synthesis
