| CARVIEW |
vMAP
Vectorised Object Mapping for Neural Field SLAM
CVPR 2023
vMAP can map each object into a separate MLP and then recompose them.
Video
Abstract
TL;DR: We present vMAP, an object-level real-time mapping system, with each object represented by a separate MLP neural field model, and object models are optimised in parallel via vectorised training.
We present vMAP, an object-level dense SLAM system using neural field representations. Each object is represented by a small MLP, enabling efficient, watertight object modelling without the need for 3D priors. As an RGB-D camera browses a scene with no prior information, vMAP detects object instances on-the-fly, and dynamically adds them to its map. Specifically, thanks to the power of vectorised training, vMAP can optimise as 50 individual objects in a single scene, with an extremely efficient training speed of 5Hz map update. We experimentally demonstrate significantly improved scene-level and object-level reconstruction quality compared to prior neural field SLAM systems.
Method
vMAP takes an RGB-D image stream as input, detects objects on-the-fly, and dynamically adds them to its map. Each object is represented by a tiny MLP and the 3D bound is continually updated via the data association across frames. vMAP achieves highly efficient vectorised training by stacking the batch of MLPs together and optimised in a single training step. Thanks to the disentangled object-level mapping design, each object model can be stopped and resumed without any inter-object interference, and can be further recomposed into new 3D scene.
Results
Replica Dataset
3D Reconstruction
Visualisation of Scene Reconstructed by Method at Object-level.
Scene Recomposition
Background
2D Novel View Synthesis
Visualisation of Scene from a Novel View.
ScanNet Dataset
Visualisation at Scene-level.vMAP
NICE-SLAM
Visualisation of Object .
vMAP
objSDF
TSDF
Live Table Sequence
Visualisation of Method .
BibTeX
@inproceedings{kong2023vmap,
title={vmap: Vectorised object mapping for neural field slam},
author={Kong, Xin and Liu, Shikun and Taher, Marwan and Davison, Andrew J},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={952--961},
year={2023}
}