| CARVIEW |
Select Language
HTTP/2 301
server: GitHub.com
content-type: text/html
location: https://ajzhai.github.io/NeRF2Physics/
x-github-request-id: B37B:328FD3:97D5E3:AA809E:6953306C
accept-ranges: bytes
age: 0
date: Tue, 30 Dec 2025 01:52:46 GMT
via: 1.1 varnish
x-served-by: cache-bom-vanm7210048-BOM
x-cache: MISS
x-cache-hits: 0
x-timer: S1767059566.030440,VS0,VE202
vary: Accept-Encoding
x-fastly-request-id: dcb55ad474bc0225eb9c5ce246d618db01e1291e
content-length: 162
HTTP/2 200
server: GitHub.com
content-type: text/html; charset=utf-8
last-modified: Mon, 08 Apr 2024 01:46:19 GMT
access-control-allow-origin: *
strict-transport-security: max-age=31556952
etag: W/"66134c6b-2ebd"
expires: Tue, 30 Dec 2025 02:02:46 GMT
cache-control: max-age=600
content-encoding: gzip
x-proxy-cache: MISS
x-github-request-id: 3F42:3655F2:98436A:AAEF49:6953306C
accept-ranges: bytes
age: 0
date: Tue, 30 Dec 2025 01:52:46 GMT
via: 1.1 varnish
x-served-by: cache-bom-vanm7210048-BOM
x-cache: MISS
x-cache-hits: 0
x-timer: S1767059566.246265,VS0,VE218
vary: Accept-Encoding
x-fastly-request-id: 15997f6c6ce928330a5818d0a1d178265c7fffe3
content-length: 4235
NeRF2Physics: Physical Property Understanding from Language-Embedded Feature Fields
NeRF2Physics: Physical Property Understanding from
NeRF2Physics: Physical Property Understanding from
Language-Embedded Feature Fields
CVPR 2024
- Albert J. Zhai
- Yuan Shen
- Emily Y. Chen
- Gloria X. Wang
- Xinlei Wang
- Sheng Wang
- Kaiyu Guan
- Shenlong Wang
- University of Illinois at Urbana-Champaign
Abstract
Can computers perceive the physical properties of objects solely through vision? Research in cognitive science and vision science has shown that humans excel at identifying materials and estimating their physical properties based purely on visual appearance. In this paper, we present a novel approach for dense prediction of the physical properties of objects using a collection of images. Inspired by how humans reason about physics through vision, we leverage large language models to propose candidate materials for each object. We then construct a language-embedded point cloud and estimate the physical properties of each 3D point using a zero-shot kernel regression approach. Our method is accurate, annotation-free, and applicable to any object in the open world. Experiments demonstrate the effectiveness of the proposed approach in various physical property reasoning tasks, such as estimating the mass of common objects, as well as other properties like friction and hardness.
Predicting Physical Properties
We visualize input images of objects from the ABO dataset along with our model's CLIP feature PCA components, zero-shot material segmentation, and predicted mass density.
Our model makes reasonable predictions of materials across different parts of objects in 3D, allowing for grounded predictions of physical properties.
Our method can be used to predict physical properties in an open-vocabulary manner.
We show that it can be used to predict mass density, Young's modulus, thermal conductivity, hardness, and friction coefficients, all without supervision.
Physically Realistic Digital Twins
We show that realistic physical interactions can be simulated using mass-aware digital twins created by NeRF2Physics, enabling applications in immersive computing and simulation.
Here, the ball hits each object with the same initial momentum.