3D Gaussian Splatting (3DGS) has recently emerged as a state-of-the-art 3D reconstruction and rendering technique due to its high-quality results and fast training and rendering time. However, pixels covered by the same Gaussian are always shaded in the same color up to a Gaussian falloff scaling factor. Furthermore, the finest geometric detail any individual Gaussian can represent is simply an ellipsoid. These properties of 3D Gaussian Splatting greatly limit the expressivity of individual Gaussian primitves.

To address these issues, we draw inspiration from texture and alpha mapping in traditional graphics and integrate it with 3DGS. Specifically, we propose a new generalized Gaussian appearance representation that augments each Gaussian with alpha (A), RGB, or RGBA texture maps to model spatial color and opacity variation across the extent of each Gaussian. As such, each Gaussian is capable of representing a richer set of texture patterns and geometric structures, instead of just a single color and ellipsoid as in naive Gaussian Splatting.

Surprisingly, we found that the expressivity of Gaussians can already be greatly improved by using alpha-only texture maps, and further augmenting Gaussians with RGB texture maps achieves the highest expressivity. We validate our method on a wide variety of standard benchmark datasets and our own custom captures at both the object level and scene level, and demonstrate image quality improvements over existing methods while using a similar or lower number of Gaussians.

There are multiple concurrent works that explore concepts similar to our paper. Here are some of them:

• GStex: Per-Primitive Texturing of 2D Gaussian Splatting for Decoupled Appearance and Geometry Modeling
  Gstex focused on texture editing of optimized Gaussian splating models by augmenting each Gaussian with RGB texture maps. However, GStex did not model spatially varying opacity with alpha texture maps.
  
  
• SuperGaussians: Enhancing Gaussian Splatting Using Primitives with Spatially Varying Colors
  Instead of texture maps, SuperGaussians used movable kernels and tiny MLPs to model spatially varying color. However, SuperGaussians did not model spatially varying opacity.
  
  
• HDGS: Textured 2D Gaussian Splatting for Enhanced Scene Rendering
  HDGS focused on anti-aliasing when rendering optimized Gaussian splatting models at arbitrary viewpoints and resolutions. However, HDGS did not model spatially varying opacity with alpha texture maps.
  
  
• BillBoard Splatting (BBSplat): Learnable Textured Primitives for Novel View Synthesis
  Similar to our method, BBsplat augmented each Gaussian with RGBA texture maps to model spatially varying color and opacity.