Gaussian Splats (Particle Fields)#

_images/gaussian-wintergarden.jpg — *Winter Garden 3D Gaussian Splat rendered together with polygonal robots*.#

3D scanning data created and provided by Andrii Shramko, Teleportour (teleportour.com)

Particle Fields are a new geometry type in OpenUSD for rendering 3D Gaussian Splats (3DGS) and other radiance fields such as 3D Gaussian Unscented Transforms (3DGUT).

Omniverse RTX renders these fields natively, so they participate in path tracing, camera effects, shadows, reflections, and refractions.

Note

Rendering particle fields with DLSS Frame Generation enabled may produce visual artifacts. If you encounter artifacts, consider disabling Frame Generation in the Rendering Settings.

Importing 3D Gaussian Splats into Omniverse#

To import a 3D Gaussian Splat into Omniverse, you must first convert it from a .ply to OpenUSD format. The OpenUSD schema for splats and other radiance fields is documented here, but the simplest way to convert a .ply to OpenUSD is to use the example Python script provided in the OpenUSD repository: py3dgsPlyToUsd.py.

python py3dgsPlyToUsd.py --input path/to/your/model.ply --output path/to/output.usd

Once the USD file is created, you can add it as a reference or payload to your stage by dragging and dropping in the same way as any other USD file.

For example, to convert the bouquet of flowers scene available here: http://developer.download.nvidia.com/ProGraphics/nvpro-samples/flowers_1.zip (CC-BY-SA License), run:

python py3dgsPlyToUsd.py --input flowers_1.ply --output flowers_1.usd

Then drag and drop from the content browser into the stage:

_images/gaussian-flowers.jpg — *Bouquet of flowers 3D Gaussian Splat imported into Kit. Notice that the flowers are refracted correctly through the glass bowl and cast shadows on the floor.*#

Particle Fields and Shadows#

Particle fields will receive shadows from other scene geometry. Note that since particle fields do not have a well-defined surface, depending on the training of the underlying particle field, the received shadows may look blurry or “blobby”.

_images/gaussian-shadows.avif — *Polygonal robot casting shadows from a manually placed light (upper-left) on the floor of a 3DGS scene. The blobby look to the shadows shows the shape of the underlying Gaussians.*#

By default, particle fields will not cast shadows on other scene objects (or themselves). This is so that particle field environments do not add redundant self-shadows that incur a performance hit. If you want particle fields to cast shadows, simply enable the Casts Shadows checkbox.

Particle Fields and Color#

In the current release, all ParticleField prims are assumed to have their radiance data in display-referred sRGB space. This means that the tonemapping pipeline is skipped for ParticleField prims and they are rendered to the camera “as-is”. This is done to ensure the result matches what is seen in other tools where color pipeline is completely ignored.

In a future version, RTX will respect any UsdColorSpaceAPI applied to the prims, allowing you to train Gaussians and other fields in other color spaces, such as linear Rec.709 or wide-gamut spaces.

Adjusting Particle Field Brightness#

Gaussian splats store captured lighting in their radiance. Scene lights can cast shadows onto the field, but changing a light’s intensity does not directly brighten the splats. To scale their brightness, use the emission_intensity parameter of ParticleFieldEmissive, the built-in NVIDIA Material Definition Language (MDL) material for particle fields.

A ParticleField3DGaussianSplat prim without a bound material uses this material by default. To edit its brightness, create an explicit instance and bind it to the particle field. The default intensity is 1.0. Values between 0.0 and 1.0 dim the splats; values greater than 1.0 brighten them.

The following example runs in Kit’s Python environment with a stage already open. Replace /World/MySplats with the path to your ParticleField3DGaussianSplat prim. The example binds a new material with an intensity of 0.5 to that prim.

import omni.usd
from pxr import Sdf, UsdShade

stage = omni.usd.get_context().get_stage()
splat_prim = stage.GetPrimAtPath("/World/MySplats")
if not splat_prim.IsValid():
    raise ValueError("Set the path to an existing particle field prim.")

material = UsdShade.Material.Define(stage, "/World/Looks/SplatBrightness")
shader = UsdShade.Shader.Define(stage, material.GetPath().AppendChild("Shader"))
shader.SetSourceAsset(Sdf.AssetPath("ParticleFieldEmissive.mdl"), "mdl")
shader.SetSourceAssetSubIdentifier("ParticleFieldEmissive", "mdl")
shader.CreateInput("emission_intensity", Sdf.ValueTypeNames.Float).Set(0.5)

shader_output = shader.CreateOutput("out", Sdf.ValueTypeNames.Token)
material.CreateSurfaceOutput("mdl").ConnectToSource(shader_output)
UsdShade.MaterialBindingAPI.Apply(splat_prim).Bind(material)

To change the value later, edit inputs:emission_intensity on /World/Looks/SplatBrightness/Shader. Every particle field bound to this material shares the same intensity. Bind separate material instances to adjust fields independently.

Improving Denoising in RTX - Real-Time 2.0#

Gaussian splats use stochastic transparency, which can produce noisy depth and color inputs for denoising. In RTX - Real-Time 2.0 mode, you can accumulate depth and albedo across Gaussian hits along a ray to stabilize the denoised image. These controls affect the denoising inputs within a frame; they do not set the number of frames to accumulate.

Refer to the Gaussian splat settings for the controls, defaults, and performance tradeoffs. Start with the default budget of 48 Gaussian hits (Max Gaussians To Accumulate) and compare image stability and frame time when adjusting it.

Other Particle Field Formats#

To train and convert other particle field formats, such as 3DGUT, please refer to the 3DGRUT project: https://github.com/nv-tlabs/3dgrut