Sida Peng (彭思达) Tenure-track Assistant Professor 3D Vision Group School of Software Technology Zhejiang University Email: pengsida at zju dot edu dot cn [Google Scholar][GitHub][Notion]

Short Bio

News

• [2025-11] One paper accepted to TPAMI 2025.
• [2025-09] One paper accepted to TOG 2025.
• [2025-07] One paper accepted to ICCV 2025 Highlight.
• [2025-04] Received the 2025 China3DV Outstanding Young Researcher Award.
• [2025-04] One paper selected as CVPR 2025 Oral.
• [2024-12] Received the 2024 CCF Outstanding Doctoral Dissertation Award.
[2024-11] Received the Outstanding Doctoral Dissertation Award of Zhejiang Province.
[2024-09] Recognised as one of the Top 2% Scientists Worldwide 2024 by Stanford University.
[2024-08] EasyVolcap is selected as the 2024 CCF Excellent Open-Source Graphics Software Award.
[2024-07] Two papers accepted to SIGGRAPH Asia 2024 (1 Journal Track and 1 Conference Track).
[2024-07] Eight papers accepted to ECCV 2024.
[2024-05] Invited talk at VALSE 2024 on NeRF Tutorial (slides).
[2024-04] Invited talk at The Chinese University of Hongkong, Shenzhen, on Physical World Simulation.
[2024-04] Haotong is awarded the Ph.D. Student Fund 2024 of the National Natural Science Foundation of China.
[2024-04] One paper accepted to SIGGRAPH 2024.
[2024-04] Four papers selected as CVPR 2024 Highlight.
[2024-03] One paper accepted to TPAMI 2024.
[2024-03] Invited talk at The University of Hong Kong on Real-Time Dynamic View Synthesis.
[2024-02] Seven papers accepted to CVPR 2024.
[2024-01] Invited talk at GAMES Webinar on Modeling of Dynamic Urban Scenes.
[2024-01] Invited talk at VALSE Webinar on learning research (slides).
[2024-01] One paper accepted to ICLR 2024.
[2024-01] One paper accepted to TPAMI 2024.
[2024-01] My article on learning research has been accepted to CCCF 2024.
[2023-12] Our proposal for 1st Workshop on Neural Volumetric Video has been accepted to CVPR 2024.
[2023-12] Invited talk at NVIDIA on Real-Time Dynamic View Synthesis.
[2023-09] One paper accepted to SIGGRAPH Asia 2023 Technical Communications.
[2023-09] Two papers accepted to NeurIPS 2023.
[2023-08] Invited talk at the SSP organized by China Computer Federation.
[2023-08] Invited talk at Shanghai Jiao Tong University on learning research (slides).
[2023-08] One paper accepted to SIGGRAPH Asia 2023.
[2023-07] Joined the School of Software Technology at Zhejiang University as an assistant professor.
[2023-07] Three papers accepted to ICCV 2023.
[2023-06] Honored to be selected as the 2023 WAIC Rising Star.
[2023-06] Our team won the first place in Image Matching Challenge 2023 (1st in 494 teams).
[2023-06] Invited talk at Shanghai AI Laboratory on learning research (slides).
[2023-06] Defended my Ph.D. thesis on Implicit Neural Representations for Dynamic Human Bodies.
[2023-05] Wrote a handbook on learning research and released it on GitHub.
[2023-03] Nine papers accepted to CVPR 2023 (2 highlights).
[2022-10] Received the 2022 National Scholarship.
[2022-03] Honored to be selected as the 2022 Apple Scholar in AI/ML (15 in the world were awarded).
[2022-03] Three papers accepted to CVPR 2022 (1 oral).
[2021-10] Received the 2021 National Scholarship.
[2021-06] Neural Body was selected as the CVPR 2021 best paper candidate (top 0.5%).
[2021-03] One paper accepted to CVPR 2021.
[2020-10] Received the 2020 National Scholarship.
[2020-10] Honored to be selected as the 2020 CCF-CV Excellent Young Researcher (3 in China were awarded).
[2020-03] One paper accepted to CVPR 2020 oral presentation (top 5%).
[2019-10] Received the 2019 National Scholarship.
[2019-03] One paper accepted to CVPR 2019 oral presentation (top 5%).

Research Summary

I like advancing spatial agents and solving fundamental research problems that arise from important challenges. I also like exploring the capability boundaries of new techniques in addressing these challenges.

After the CVPR 2025 submission, I decided to pursue research projects that address crucial problems in the industry with significant impact. Such problems typically lack well-established solutions, necessitating us to explore new techniques and provide new insights. I would avoid participating in projects that lack both practical value and unresolved problems, as I believe they only produces useless papers, consuming my passion for research.

The following are my target systems:

• In the past few years, I have focused on developing two systems: Driving Simulator and Volumetric Video.
• Moving forward, I will focus on training spatial agents and developing their fundamental abilities.

[Show the research progress]

Closed-Loop Simulator

Spatial Agent

Model Training

[Show the research progress]

Camera Pose Estimation

Depth Estimation

Object Motion Estimation

[Show the research progress]

Lower-cost Capture

Faster Reconstruction

Lower Storage

Faster Rendering

Selected Preprint

MatchAnything: Universal Cross-Modality Image Matching with Large-Scale Pre-Training
Xingyi He, Hao Yu, Sida Peng, Dongli Tan, Zehong Shen, Hujun Bao, Xiaowei Zhou
[Paper][Project Page][Code]

Selected Publications (* denotes equal contribution, and † denotes the corresponding author)

Street Gaussians: Modeling Dynamic Urban Scenes with Gaussian Primitives
Sida Peng, Yushi Long, Yunzhi Yan, Haotong Lin, Chenxu Zhou, Haiyang Sun, Kun Zhan, Xianpeng Lang, Hujun Bao, Xiaowei Zhou
TPAMI 2025.
[Paper][Project Page][Code]

Towards Depth Foundation Model: Recent Trends in Vision-Based Depth Estimation
Zhen Xu*, Hongyu Zhou*, Sida Peng, Haotong Lin, Haoyu Guo, Jiahao Shao, Peishan Yang, Qinglin Yang, Sheng Miao, Xingyi He, Yifan Wang, Yue Wang, Ruizhen Hu, Yiyi Liao, Xiaowei Zhou, Hujun Bao
CVMJ 2025.
[Paper]

Split4D: Decomposed 4D Scene Reconstruction Without Video Segmentation
Yongzhen Hu, Yihui Yang, Haotong Lin, Yifan Wang, Junting Dong, Yifu Deng, Xinyu Zhu, Fan Jia, Hujun Bao, Xiaowei Zhou, Sida Peng^†
SIGGRAPH Asia 2025 Journal Track. TOG 2025.

Hierarchy UGP: Hierarchy Unified Gaussian Primitive for Large-Scale Dynamic Scene Reconstruction
Hongyang Sun*, Qinglin Yang*, Jiawei Wang*, Zhen Xu, Chen Liu, Yida Wang, Kun Zhan, Hujun Bao, Xiaowei Zhou, Sida Peng^†
ICCV 2025.
[Paper]

ReTracker: Exploring Image Matching for Robust Online Any Point Tracking
Dongli Tan, Xingyi He, Sida Peng, Yiqing Gong, Xing Zhu, Jiaming Sun, Ruizhen Hu, Yujun Shen, Hujun Bao, Xiaowei Zhou
ICCV 2025 Highlight.
[Paper]

StreetCrafter: Street View Synthesis with Controllable Video Diffusion Models
Yunzhi Yan*, Zhen Xu*, Haotong Lin, Haian Jin, Haoyu Guo, Yida Wang, Kun Zhan, Xianpeng Lang, Hujun Bao, Xiaowei Zhou, Sida Peng^†
CVPR 2025.
[Paper][Project Page][Code]

Prompting Depth Anything for 4K Resolution Accurate Metric Depth Estimation
Haotong Lin, Sida Peng^†, Jingxiao Chen, Songyou Peng, Jiaming Sun, Minghuan Liu, Hujun Bao, Jiashi Feng, Xiaowei Zhou, Bingyi Kang
CVPR 2025.
[Paper][Project Page][Code]

LiDAR-RT: Gaussian-based Ray Tracing for Dynamic LiDAR Re-simulation
Chenxu Zhou*, Lvchang Fu*, Sida Peng^†, Yunzhi Yan, Zhanhua Zhang, Yong Chen, Jiazhi Xia, Xiaowei Zhou
CVPR 2025.
[Paper][Project Page][Code]

Multi-view Reconstruction via SfM-guided Monocular Depth Estimation
Haoyu Guo*, He Zhu*, Sida Peng, Haotong Lin, Yunzhi Yan, Tao Xie, Wenguan Wang, Xiaowei Zhou, Hujun Bao
CVPR 2025 Oral presentation.
[Paper][Project Page][Code]

FreeTimeGS: Free Gaussian Primitives at Anytime Anywhere for Dynamic Scene Reconstruction
Yifan Wang*, Peishan Yang*, Zhen Xu*, Jiaming Sun, Zhanhua Zhang, Yong Chen, Hujun Bao, Sida Peng, Xiaowei Zhou
CVPR 2025.
[Paper][Project Page][Interactive Demo][Media Coverage]

Street Gaussians for Modeling Dynamic Urban Scenes
Yunzhi Yan, Haotong Lin, Chenxu Zhou, Weijie Wang, Haiyang Sun, Kun Zhan, Xianpeng Lang, Xiaowei Zhou, Sida Peng^†
ECCV 2024.
[Paper][Project Page][Code]

GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation
Yinghao Xu, Zifan Shi, Yifan Wang, Hansheng Chen, Ceyuan Yang, Sida Peng, Yujun Shen, Gordon Wetzstein
ECCV 2024.
[Paper][Project Page]

Neural 3D Scene Reconstruction with Indoor Planar Priors
Xiaowei Zhou, Haoyu Guo, Sida Peng, Yuxi Xiao, Haotong Lin, Qianqian Wang, Guofeng Zhang, Hujun Bao
TPAMI 2024.
[Paper][Supplementary Material][Project Page][Code]

4K4D: Real-Time 4D View Synthesis at 4K Resolution
Zhen Xu, Sida Peng, Haotong Lin, Guangzhao He, Jiaming Sun, Yujun Shen, Hujun Bao, Xiaowei Zhou
CVPR 2024.
[Paper][Project Page][Code]

Detector-Free Structure from Motion
Xingyi He, Jiaming Sun, Yifan Wang, Sida Peng, Qixing Huang, Hujun Bao, Xiaowei Zhou
CVPR 2024. First Place in Image Matching Challenge 2023.
[Paper][Project Page][Code]

SpatialTracker: Tracking Any 2D Pixels in 3D Space
Yuxi Xiao, Qianqian Wang, Shangzhan Zhang, Nan Xue, Sida Peng, Yujun Shen, Xiaowei Zhou
CVPR 2024 Highlight.
[Paper][Project Page][Code]

Efficient LoFTR: Semi-Dense Local Feature Matching with Sparse-Like Speed
Yifan Wang*, Xingyi He*, Sida Peng^†, Dongli Tan, Xiaowei Zhou
CVPR 2024 Highlight.
[Paper][Project Page][Code]

Animatable Implicit Neural Representations for Creating Realistic Avatars from Videos
Xiaowei Zhou, Sida Peng^†, Zhen Xu, Junting Dong, Qianqian Wang, Shangzhan Zhang, Qing Shuai, Hujun Bao
TPAMI 2024.
[Paper][Supplementary Material][Project Page][Code]

A Guide to Conducting a Research Project
Sida Peng
CCCF 2024.
[Article][Project Page][Talk]

EasyVolcap: Accelerating Neural Volumetric Video Research
Zhen Xu, Tao Xie, Sida Peng, Haotong Lin, Qing Shuai, Zhiyuan Yu, Guangzhao He, Jiaming Sun, Hujun Bao, Xiaowei Zhou
SIGGRAPH Asia 2023 Technical Communications.
[Paper][Code]

Im4D: High-Fidelity and Real-Time Novel View Synthesis for Dynamic Scenes
Haotong Lin, Sida Peng^†, Zhen Xu, Tao Xie, Xingyi He, Hujun Bao, Xiaowei Zhou
SIGGRAPH Asia 2023.
[Paper][Project Page][Code]

Representing Volumetric Videos as Dynamic MLP Maps
Sida Peng*, Yunzhi Yan*, Qing Shuai, Hujun Bao, Xiaowei Zhou
CVPR 2023.
[Paper][Project Page][Code]

Learning Neural Volumetric Representations of Dynamic Humans in Minutes
Chen Geng*, Sida Peng*, Zhen Xu*, Hujun Bao, Xiaowei Zhou
CVPR 2023.
[Paper][Project Page][Code]

Grid-guided Neural Radiance Fields for Large Urban Scenes
Linning Xu*, Yuanbo Xiangli*, Sida Peng, Xingang Pan, Nanxuan Zhao, Christian Theobalt, Bo Dai, Dahua Lin
CVPR 2023.
[Paper][Project Page]

Implicit Neural Representations with Structured Latent Codes for Human Body Modeling
Sida Peng, Chen Geng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Xiaowei Zhou, Hujun Bao
TPAMI 2023.
[Paper][Code]

TotalSelfScan: Learning Full-body Avatars from Self-Portrait Videos of Faces, Hands, and Bodies
Junting Dong, Qi Fang, Yudong Guo, Sida Peng, Qing Shuai, Hujun Bao, Xiaowei Zhou
NeurIPS 2022.
[Paper][Project Page][Code]

Efficient Neural Radiance Fields for Interactive Free-viewpoint Video
Haotong Lin*, Sida Peng*, Zhen Xu, Yunzhi Yan, Qing Shuai, Hujun Bao, Xiaowei Zhou
SIGGRAPH Asia 2022.
[Paper][Project Page][Code]

MPS-NeRF: Generalizable 3D Human Rendering from Multiview Images
Xiangjun Gao, Jiaolong Yang, Jongyoo Kim, Sida Peng, Zicheng Liu, Xin Tong
TPAMI 2022.
[Paper][Project Page][Code]

NerfCap: Human Performance Capture with Dynamic Neural Radiance Fields
Kangkan Wang, Sida Peng, Xiaowei Zhou, Jian Yang, and Guofeng Zhang
TVCG 2022.
[Paper]

Learning to Estimate Object Poses without Real Image Annotations
Haotong Lin*, Sida Peng*, Zhize Zhou, Xiaowei Zhou
IJCAI 2022.
[Paper][Code]

Neural 3D Scene Reconstruction with the Manhattan-world Assumption
Haoyu Guo*, Sida Peng*, Haotong Lin, Qianqian Wang, Guofeng Zhang, Hujun Bao, Xiaowei Zhou
CVPR 2022 Oral presentation.
[Paper][Supplementary Material][Project Page][Code]

3D-aware Image Synthesis via Learning Structural and Textural Representations
Yinghao Xu, Sida Peng, Ceyuan Yang, Yujun Shen, Bolei Zhou
CVPR 2022.
[Paper][Project Page][Code]

Neural Rays for Occlusion-aware Image-based Rendering
Yuan Liu, Sida Peng, Lingjie Liu, Qianqian Wang, Peng Wang, Christian Theobalt, Xiaowei Zhou, Wenping Wang
CVPR 2022.
[Paper][Project Page][Code]

Towards Efficient and Photorealistic 3D Human Reconstruction: A Brief Survey
Lu Chen, Sida Peng, Xiaowei Zhou
Visual Informatics 2022.
[Paper]

Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies
Sida Peng*, Junting Dong*, Qianqian Wang, Shangzhan Zhang, Qing Shuai, Xiaowei Zhou, Hujun Bao
ICCV 2021.
[Paper][Supplementary Material][Project Page][Code][Extension]

Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans
Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Hujun Bao, Xiaowei Zhou
CVPR 2021 Best paper candidate.
[Paper][Supplementary Material][Project Page][Code]

PVNet: Pixel-wise Voting Network for 6DoF Object Pose Estimation
Sida Peng*, Xiaowei Zhou*, Yuan Liu, Haotong Lin, Qixing Huang, Hujun Bao
TPAMI 2020.
[Paper][Code]

Deep Snake for Real-Time Instance Segmentation
Sida Peng, Wen Jiang, Huaijin Pi, Xiuli Li, Hujun Bao, Xiaowei Zhou
CVPR 2020 Oral presentation.
[Paper][Code]

GIFT: Learning Transformation-Invariant Dense Visual Descriptors via Group CNNs
Yuan Liu, Zehong Shen, Zhixuan Lin, Sida Peng, Hujun Bao, Xiaowei Zhou
NeurIPS 2019.
[Paper][Supplementary Material][Project Page][Code]

PVNet: Pixel-wise Voting Network for 6DoF Estimation
Sida Peng*, Yuan Liu*, Qixing Huang, Xiaowei Zhou, Hujun Bao
CVPR 2019 Oral presentation.
[Paper][Supplementary Material][Project Page][Code]

Honors

• China3DV Outstanding Young Researcher Award
  2025
• CCF Outstanding Doctoral Dissertation Award
  2024
• CCF Excellent Open-Source Graphics Software Award
  2024
• Zhejiang Province Outstanding Doctoral Dissertation Award
  2023
• Zhejiang University Outstanding Doctoral Dissertation Award
  2023
• WAIC Rising Star Award
  2023
• First place in Image Matching Challenge 2023
  2023
• Outstanding Graduates of Zhejiang Province
  2023
• National Scholarship
  2022
• Apple Scholars in AI/ML PhD Fellowship
  2022
• National Scholarship
  2021
• Style3D Graduate Fellowship
  2021
• First place in China Graduate AI Innovation Competition
  2021
• National Scholarship
  2020
• CCF-CV Excellent Young Researcher Award
  2020
• First place in China Graduate AI Innovation Competition
  2019
• National Scholarship
  2019
• Chen TianZhou Educational Foundation Scholarship
  2019
• Chiang Chen Industrial Charity Foundation Grant
  2019
• Graduate Student Scholarship
  2019

Services

• Area Chair: 3DV 2025
• Technical Papers Committee: SIGGRAPH Asia 2024
• GAMES Committee
• Conference Committee: Eurographics (2024)
• Conference Reviewer: CVPR, ICCV, ECCV, NeurIPS, SIGGRAPH, SIGGRAPH Asia, Eurographics
• Journal Reviewer: TPAMI, TVCG, IJCV, TCSVT

Teaching

• Introduction to Game Development
  Winter, 2025
• Computer Graphics
  Autumn, 2025
• Introduction to Game Development
  Spring, 2025
• Computer Graphics
  Autumn, 2024

Teaching Assistant

• Computational Photography
  Spring, 2023
• Computational Photography
  Spring, 2022
• Computational Photography
  Spring, 2021
• Computational Photography
  Spring, 2020
• Computational Photography
  Spring, 2019