HTTP/2 301 server: GitHub.com content-type: text/html location: https://talegqz.github.io/BAGS/ access-control-allow-origin: * expires: Mon, 29 Dec 2025 07:40:41 GMT cache-control: max-age=600 x-proxy-cache: MISS x-github-request-id: 9B3E:292AC1:874487:97FF56:69522E21 accept-ranges: bytes age: 0 date: Mon, 29 Dec 2025 07:30:41 GMT via: 1.1 varnish x-served-by: cache-bom-vanm7210090-BOM x-cache: MISS x-cache-hits: 0 x-timer: S1766993441.378374,VS0,VE201 vary: Accept-Encoding x-fastly-request-id: 45637fff6e629aef7eedd1ed16645f094a9b7748 content-length: 162 HTTP/2 200 server: GitHub.com content-type: text/html; charset=utf-8 last-modified: Fri, 13 Dec 2024 05:27:23 GMT access-control-allow-origin: * etag: W/"675bc5bb-3c3c" expires: Mon, 29 Dec 2025 07:40:41 GMT cache-control: max-age=600 content-encoding: gzip x-proxy-cache: MISS x-github-request-id: C098:2B0FD4:87B3E2:986D49:69522E21 accept-ranges: bytes age: 0 date: Mon, 29 Dec 2025 07:30:41 GMT via: 1.1 varnish x-served-by: cache-bom-vanm7210090-BOM x-cache: MISS x-cache-hits: 0 x-timer: S1766993442.608656,VS0,VE209 vary: Accept-Encoding x-fastly-request-id: 3db12a2cd6cd08eaefee97f8062ac9c5a6b5040b content-length: 3770

BAGS: Building Animatable Gaussian Splatting from a Monocular Video with Diffusion Priors

Arxiv 2024

Tingyang Zhang*¹, Qingzhe Gao*^2,1, Weiyu Li³, Libin Liu¹, Baoquan Chen¹,

*: Joint first authors with equal contributions to this work

¹Peking Unviersity, ²Shandong Unviersity, ³The Hong Kong University of Science and Technology

arXiv Code

Abstract

Animatable 3D reconstruction has significant applications across various fields, primarily relying on artists' handcraft creation. Recently, some studies have successfully constructed animatable 3D models from monocular videos. However, these approaches require sufficient view coverage of the object within the input video and typically necessitate significant time and computational costs for training and rendering. This limitation restricts the practical applications. In this work, we propose a method to build animatable 3D Gaussian Splatting from monocular video with diffusion priors. The 3D Gaussian representations significantly accelerate the training and rendering process, and the diffusion priors allow the method to learn 3D models with limited viewpoints.We also present the rigid regularization to enhance the utilization of the priors. We perform an extensive evaluation across various real-world videos, demonstrating its superior performance compared to the current state-of-the-art methods.

Given a single casual video, our method constructs an animatable 3D Gaussian Splatting model with diffusion priors. This not only compensates for unseen view information but also enables fast training and real-time rendering.

Pipeline: we construct a canonical space using Gaussian Splatting. In the absence of a templated parametric model, we develop a neural bones representation to animate the canonical space to match the input video. Additionally, we utilize a diffusion model to address unseen view information and apply a rigid constraint to facilitate training. After training, the model can be manually manipulated to achieve novel pose rendering.

Video

Comparisons

Input

Banmo

Ours

BibTeX

@misc{zhang2024bags,
          title={BAGS: Building Animatable Gaussian Splatting from a Monocular Video with Diffusion Priors}, 
          author={Tingyang Zhang and Qingzhe Gao and Weiyu Li and Libin Liu and Baoquan Chen},
          year={2024},
          eprint={2403.11427},
          archivePrefix={arXiv},
          primaryClass={cs.CV}
      }