HTTP/2 301 server: GitHub.com content-type: text/html location: https://ttaoretw.github.io/DreaMo/ access-control-allow-origin: * strict-transport-security: max-age=31556952 expires: Mon, 29 Dec 2025 05:06:32 GMT cache-control: max-age=600 x-proxy-cache: MISS x-github-request-id: 4E09:234FE9:858413:95FA2B:69520A00 accept-ranges: bytes age: 0 date: Mon, 29 Dec 2025 04:56:32 GMT via: 1.1 varnish x-served-by: cache-bom-vanm7210034-BOM x-cache: MISS x-cache-hits: 0 x-timer: S1766984193.691673,VS0,VE211 vary: Accept-Encoding x-fastly-request-id: c328ba9f14d30de22b74986a9a1d5518e701295c content-length: 162 HTTP/2 200 server: GitHub.com content-type: text/html; charset=utf-8 last-modified: Wed, 18 Dec 2024 04:11:23 GMT access-control-allow-origin: * strict-transport-security: max-age=31556952 etag: W/"67624b6b-ce1d" expires: Mon, 29 Dec 2025 05:06:33 GMT cache-control: max-age=600 content-encoding: gzip x-proxy-cache: MISS x-github-request-id: CD63:1387E:83C589:943B74:69520A00 accept-ranges: bytes age: 0 date: Mon, 29 Dec 2025 04:56:33 GMT via: 1.1 varnish x-served-by: cache-bom-vanm7210034-BOM x-cache: MISS x-cache-hits: 0 x-timer: S1766984193.921455,VS0,VE217 vary: Accept-Encoding x-fastly-request-id: e0a9a7224a1da05ac6ecdab56429871e56fc4bc9 content-length: 5032

DreaMo: Articulated 3D Reconstruction From A Single Casual Video

Tao Tu*¹, Ming-Feng Li*², Chieh Hubert Lin³, Yen-Chi Cheng⁴, Min Sun^1,5, Ming-Hsuan Yang³

¹National Tsing Hua University, ²CMU, ³UC Merced, ⁴UIUC, ⁵Amazon

Paper

DreaMo reconstructs an articulated 3D model from just one Internet video, even when the target subject lacks sufficient view coverage.

Abstract

Articulated 3D reconstruction has valuable applications in various domains, yet it remains costly and demands intensive work from domain experts. Recent advancements in template-free learning methods show promising results with monocular videos. Nevertheless, these approaches necessitate a comprehensive coverage of all viewpoints of the subject in the input video, thus limiting their applicability to casually captured videos from online sources.

In this work, we study articulated 3D shape reconstruction from a single and casually captured internet video, where the subject's view coverage is incomplete. We propose DreaMo that jointly performs shape reconstruction while solving the challenging low-coverage regions with view-conditioned diffusion prior and several tailored regularizations. In addition, we introduce a skeleton generation strategy to create human-interpretable skeletons from the learned neural bones and skinning weights.

We conduct our study on a self-collected internet video collection characterized by incomplete view coverage. DreaMo shows promising quality in novel-view rendering, detailed articulated shape reconstruction, and skeleton generation. Extensive qualitative and quantitative studies validate the efficacy of each proposed component, and show existing methods are unable to solve correct geometry due to the incomplete view coverage.

Video

Novel View Rendering

Ours

BANMo

Ours

BANMo

Ours

BANMo

Ours

BANMo

Ours

BANMo

Ours

BANMo

Ours

BANMo

Ours

BANMo

Ours

BANMo

Ours

BANMo

Ours

BANMo

Ours

BANMo

Ours

BANMo

Ours

BANMo

Ours

BANMo

Ours

BANMo

Ours

BANMo

Ours

BANMo

Ours

BANMo

Ours

BANMo

3D Shape Reconstruction

Ours

BANMo

Ours

BANMo

Ours

BANMo

Ours

BANMo

Ours

BANMo

Ours

BANMo

BibTeX

@article{tu2023dreamo,
  title={DreaMo: Articulated 3D Reconstruction From A Single Casual Video},
  author={Tu, Tao and Li, Ming-Feng and Lin, Chieh Hubert and Cheng, Yen-Chi and Sun, Min and Yang, Ming-Hsuan},
  journal={arXiv preprint arXiv:2312.02617},
  year={2023}
}

References

[BANMo] Gengshan Yang, et al. “Building animatable 3d neural models from many casual videos.” CVPR. 2022.