This repository is an official implementation of StreamPETR.

• [2023/07/14] StreamPETR is accepted by ICCV 2023.
• [2023/05/03] StreamPETR-Large is the first online multi-view method that achieves comparable performance (62.0 mAP, 67.6 NDS and 65.3 AMOTA) with the baseline of lidar-based method.

Please follow our documentation step by step. If you like our work, please recommend it to your colleagues and friends.

1. Environment Setup.
2. Data Preparation.
3. Training and Inference.

The detailed results can be found in the training log. For other results on nuScenes val set, please see Here. Notes:

• FPS is measured on NVIDIA RTX 3090 GPU with batch size of 1 (containing 6 view images, without using flash attention) and FP32.
• The training time is measured with 8x 2080ti GPUs.
• RepDETR3D uses deformable attention, which is inspired by DETR3D and Sparse4D.

• StreamPETR code (also including PETR and Focal-PETR)
• Flash attention
• Deformable attention (RepDETR3D)
• Checkpoints
• Sliding window training
• Efficient training in streaming video
• TensorRT inference
• 3D object tracking

We thank these great works and open-source codebases:

• 3D Detection. MMDetection3d, DETR3D, PETR, BEVFormer, SOLOFusion, Sparse4D.
• Multi-object tracking. MOTR, PF-Track.

If you find StreamPETR is useful in your research or applications, please consider giving us a star 🌟 and citing it by the following BibTeX entry.

@article{wang2023exploring,
  title={Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection},
  author={Wang, Shihao and Liu, Yingfei and Wang, Tiancai and Li, Ying and Zhang, Xiangyu},
  journal={arXiv preprint arXiv:2303.11926},
  year={2023}
}