Textured Gaussians for Enhanced 3D Scene Appearance Modeling

3D Gaussian Splatting (3DGS) has recently emerged as a state-of-the-art 3D reconstruction and rendering technique due to its high-quality results and fast training and rendering time. However, pixels covered by the same Gaussian are always shaded in the same color up to a Gaussian falloff scaling factor. Furthermore, the finest geometric detail any individual Gaussian can represent is simply an ellipsoid. These properties of 3D Gaussian Splatting greatly limit the expressivity of individual Gaussian primitves.

To address these issues, we draw inspiration from texture and alpha mapping in traditional graphics and integrate it with 3DGS. Specifically, we propose a new generalized Gaussian appearance representation that augments each Gaussian with alpha (A), RGB, or RGBA texture maps to model spatial color and opacity variation across the extent of each Gaussian. As such, each Gaussian is capable of representing a richer set of texture patterns and geometric structures, instead of just a single color and ellipsoid as in naive Gaussian Splatting.

Surprisingly, we found that the expressivity of Gaussians can already be greatly improved by using alpha-only texture maps, and further augmenting Gaussians with RGB texture maps achieves the highest expressivity. We validate our method on a wide variety of standard benchmark datasets and our own custom captures at both the object level and scene level, and demonstrate image quality improvements over existing methods while using a similar or lower number of Gaussians.

Leave a Comment