| CARVIEW |
GIR: 3D Gaussian Inverse Rendering for Relightable Scene Factorization
Haocheng Feng3, Jian Zhang2† Bin Zhou1† Errui Ding3 Jingdong Wang3
IEEE Transactions on Transactions on Pattern Analysis and Machine Intelligence (T-PAMI) 2025
Abstract
Method
Pipeline of GIR
In this paper, we answer this question by introducing GIR, a novel inverse rendering framework based on 3DGS that estimates material properties, geometry, and illumination from multi-view images in high fidelity. We address the challenge of accurately modeling surface normals in 3D Gaussian representations. The complexity arises from the discrete and often in-homogeneous distribution of these Gaussians. This makes the learning of accurate normals nontrivial without the use of regularization. Moreover, the normal regularization methods designed for learning implicit Signed Distance Functions (SDF), such as the one used in NeuS cannot be applied in this context. We thus propose an efficient self-regularization method that ensures the shortest axis of each visible 3D Gaussian forms an obtuse angle with the camera's principal axis. Furthermore, we propose an approach for indirect illumination reconstruction that is suited for 3D Gaussian representation. Extensive experimental analyses have demonstrated that our proposed approach significantly outperforms existing methods on various datasets across multiple tasks.
Normal Estimation
Gaussians present on the surface and observed from the viewpoint should be visible. Any Gaussian that is not visible, as determined by $\mathbf{n} \cdot \mathbf{v} \leq 0$, does not contribute to color calculations. Therefore, the normal direction of these invisible Gaussians needs to be modified in order to reconstruct the desired image.
Indirect Illumination
The illustration of our indirect illumination reconstruction: Instead of employing recursive ray tracing, we utilize spherical harmonic coefficients to encode the view direction, enabling the simulation of indirect illumination.
BibTeX
@article{shi2025gir,
author = {Shi, Yahao and Wu, Yanmin and Wu, Chenming and Liu, Xing and Zhao, Chen and Feng, Haocheng and Zhang, Jian and Zhou, Bin and Ding, Errui and Wang, Jingdong},
title = {GIR: 3D Gaussian Inverse Rendering for Relightable Scene Factorization},
journal = {IEEE Transactions on Transactions on Pattern Analysis and Machine Intelligence},
year = {2025},
}