Official task check: https://github.com/KTH-RPL/DeFlow, Here is the inference for DynamicMap Benchmark.
Environment: Clone the repo and build the environment, check detail installation for more information. Conda/Mamba is recommended.
git clone --recursive https://github.com/KTH-RPL/DeFlow.git
cd DeFlow
mamba env create -f environment.yamlCUDA package (need install nvcc compiler), the compile time is around 1-5 minutes:
mamba activate deflow
# CUDA already install in python environment. I also tested others version like 11.3, 11.4, 11.7, 11.8 all works
cd assets/cuda/mmcv && python ./setup.py install && cd ../../..Download data: KTH-RPL/DynamicMap_Benchmark
Or another environment setup choice is Docker which isolated environment, you can pull it by.
If you have different arch, please build it by yourself cd DeFlow && docker build -t zhangkin/seflow by going through build-docker-image section.
# option 1: pull from docker hub
docker pull zhangkin/seflow
# run container
docker run -it --gpus all -v /dev/shm:/dev/shm -v /home/kin/data:/home/kin/data --name deflow zhangkin/seflow /bin/zsh
# then `mamba activate seflow` python environment is ready to useDownload pre-trained weights for models are available in huggingface. The weight is trained on Argoverse 2 Sensor dataset only, while DynamicMap Benchmark av2 is from Argoverse 2 Map dataset. So no overfitting issue.
wget https://huggingface.co/kin-zhang/OpenSceneFlow/resolve/main/deflow_best.ckpt
python main.py checkpoint=/home/kin/deflow_best.ckpt dataset_path=/home/kin/data/00
python main.py checkpoint=/home/kin/deflow_best.ckpt dataset_path=/home/kin/data/av2Running time & Result screenshot:
@inproceedings{zhang2024deflow,
author={Zhang, Qingwen and Yang, Yi and Fang, Heng and Geng, Ruoyu and Jensfelt, Patric},
booktitle={2024 IEEE International Conference on Robotics and Automation (ICRA)},
title={{DeFlow}: Decoder of Scene Flow Network in Autonomous Driving},
year={2024},
pages={2105-2111},
doi={10.1109/ICRA57147.2024.10610278}
}
This implementation is based on codes from several repositories. Thanks to these authors who kindly open-sourcing their work to the community. Please see our paper reference part to get more information. Thanks to Kyle Vedder (ZeroFlow) who kindly discussed their results with us and HKUST Ramlab's member: Jin Wu who gave constructive comments on this work. The computations were enabled by the supercomputing resource Berzelius provided by National Supercomputer Centre at Linköping University and the Knut and Alice Wallenberg Foundation, Sweden.
❤️: ZeroFlow, NSFP, FastNSF. Others good code style and tools: forecast-mae, kiss-icp
