CARVIEW

MOTORHOMES

Select Language

HTTP/2 301 server: GitHub.com content-type: text/html location: https://lemonatsu.github.io/anerf/ x-github-request-id: 7150:328FD3:878BEE:9842A0:69522D22 accept-ranges: bytes age: 0 date: Mon, 29 Dec 2025 07:26:26 GMT via: 1.1 varnish x-served-by: cache-bom-vanm7210054-BOM x-cache: MISS x-cache-hits: 0 x-timer: S1766993186.454490,VS0,VE196 vary: Accept-Encoding x-fastly-request-id: f70c4e54b0c9ebe8ce103b6caa1da53798db9eb2 content-length: 162 HTTP/2 200 server: GitHub.com content-type: text/html; charset=utf-8 last-modified: Tue, 07 Dec 2021 13:06:24 GMT access-control-allow-origin: * etag: W/"61af5c50-3949" expires: Mon, 29 Dec 2025 07:36:26 GMT cache-control: max-age=600 content-encoding: gzip x-proxy-cache: MISS x-github-request-id: EF77:2916CC:882CDE:98E648:69522D22 accept-ranges: bytes age: 0 date: Mon, 29 Dec 2025 07:26:26 GMT via: 1.1 varnish x-served-by: cache-bom-vanm7210054-BOM x-cache: MISS x-cache-hits: 0 x-timer: S1766993187.679681,VS0,VE221 vary: Accept-Encoding x-fastly-request-id: 8a6417839cecf3e47a84084f43b13f758d121543 content-length: 4507 A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose

Home Abstract Overview BibTex ☰

A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose

NeurIPS 2021

Shih-Yang Su¹ Frank Yu¹ Michael Zollhöfer² Helge Rhodin¹

¹The University of British Columbia ²Reality Labs Research

Paper

Code

Our A-NeRF jointly learns a neural body model of the user and works with diverse body poses while also refining the initial 3D articulated skeleton pose estimate from a single or, if available, multiple views without tedious camera calibration.
(Left) Articulated human representation learned from synthetic dataset and rendered in unseen poses/novel views.
(Center) Pose refinment on Human3.6M dataset. Faces are blurred out for anonimity.
(Right) 3D reconstruction of A-NeRF body model.

Abstract

While deep learning reshaped the classical motion capture pipeline with feed-forward networks, generative models are required to recover fine alignment via iterative refinement. Unfortunately, the existing models are usually hand-crafted or learned in controlled conditions, only applicable to limited domains. We propose a method to learn a generative neural body model from unlabelled monocular videos by extending Neural Radiance Fields (NeRFs). We equip them with a skeleton to apply to time-varying and articulated motion. A key insight is that implicit models require the inverse of the forward kinematics used in explicit surface models. Our reparameterization defines spatial latent variables relative to the pose of body parts and thereby overcomes ill-posed inverse operations with an overparameterization. This enables learning volumetric body shape and appearance from scratch while jointly refining the articulated pose; all without ground truth labels for appearance, pose, or 3D shape on the input videos. When used for novel-view-synthesis and motion capture, our neural model improves accuracy on diverse datasets.

Overview

A-NeRF is a generative model that can be rendered and optimized on a photometric loss. First, the skeleton pose is initialized with an off-the-shelf estimator (orange). Second, this pose is refined via a skeleton-relative embedding (blue) that, when fed to NeRF (green), drives the implicit body model that is rendered by ray-marching (red). A key property of the skeleton-relative embedding is that a single 3D query location maps to an overcomplete reparametrization, with the same point represented relative to each skeleton bone (right).

Video

Citation

Shih-Yang Su, Frank Yu, Michael Zollhöfer, and Helge Rhodin. "A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose", NeurIPS, 2021

BibTex

    @inproceedings{su2021anerf,
        title={A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose},
        author={Su, Shih-Yang and Yu, Frank and Zollh{\"o}fer, Michael and Rhodin, Helge},
        booktitle = {Advances in Neural Information Processing Systems},
        year={2021}
    }

Acknowledgments

We thank Sida Peng for helpful discussions related to NeuralBody. We thank Bastian Wandt, Daniel Ajisafe, Yuliang Zou, and Meng-Li Shih for many insightful discussions and feedbacks. We also thank Advanced Research Computing at the University of British Columbia and Compute Canada for providing computational resources.

Original Source | Taken Source