Materials#

Overview#

This extension provides non-visual materials for the Lidar, Radar, and Acoustic sensor models. A non-visual material describes how a sensor’s wave (optical or mechanical) interacts with a surface, and is used by the sensor’s BSDF to compute reflection and transmission at ray-traced hits.

Non-visual materials are authored in USD using semantic labels (e.g. aluminum, concrete) and are mapped automatically to the underlying sensor models. No code changes are required to label content; advanced users can remap the material name-to-ID mapping or the BSDF behavior bound to a material via carb settings.

Concepts#

Design principles#

The material system is designed around four properties that the rest of this document expands on:

  • Completeness — boundary reflections, volumetric transmission and attenuation, inter-reflection, optional phase/polarization.

  • Layering — additional sheets (coatings, paints) compose on top of the base material via the USD coating attribute, no code changes required.

  • Semantic content workflow — content authors work in terms of named materials (aluminum, concrete) rather than spectral data, with the runtime handling the mapping to BSDFs and property sets.

  • Remappability — both the name-to-ID mapping and the BSDF-to-ID mapping can be overridden per modality at startup or at runtime (see Remappings).

Behaviors and properties#

A non-visual material has two parts:

  • A behavior — a BSDF (Bidirectional Scattering Distribution Function) kernel that defines how reflection and transmission are computed at a surface. Behaviors ship as the Constant, DefaultMaterial, CoreMaterial, CompositeMaterial, RetroReflectiveMaterial, AcousticMaterial, and ValidationMaterial plugins. Which behavior is bound to a material ID by default depends on the sensor’s wave type (see below).

  • A property set — the spectral and physical data that a particular BSDF reads (NvMatConfig plus the per-material JSON profile, holding spectral data, refractive indices, bulk properties, etc.).

The same behavior plugin (for example CompositeMaterial) is reused across many base materials; only the property set bound to that material ID changes.

Sensor-agnostic output#

BSDFs return reflection and transmission coefficients as unitless values; the sensor model is responsible for interpreting them in its own radiometric unit-space (normalized intensity, absolute power, radiance, etc.). At the authoring level this means the same USD material labels can be reused across lidar, radar, and acoustic content; at the runtime level the wave type still selects different BSDF bindings (see below).

Wave types and modalities#

Materials are spectrally and wave-type aware:

  • Electromagnetic wave type — used by lidar and radar. BSDFs read optical/electromagnetic properties (refractive index, extinction coefficient, etc.).

  • Mechanical wave type — used by acoustic. BSDFs read physical properties (density, compressibility) and use impedance math rather than optical reflectance.

The default property-to-BSDF binding is wave-type dependent: on the EM path, IDs 0 and 47 bind to DefaultMaterial and IDs 1-46 to CompositeMaterial; on the acoustic path, IDs 0-46 bind to AcousticMaterial (ID 47, the calibration target slot, has no acoustic interpretation and falls through to DefaultMaterial).

Built-in BSDF behaviors#

The names below are what you pass to matBehaviorToIdOverrides to remap a BSDF (see Remappings):

Behavior

Description

Constant

Returns a fixed constant factor for every hit regardless of geometry or material data; no Fresnel or spectral physics.

DefaultMaterial

Lambertian (cosine-weighted) reflection with a scalar factor.

CoreMaterial

Full spectral diffuse + specular response with reflection and transmission, driven by the material’s property data.

CompositeMaterial

Behavior-level combination of lambertian, spectral diffuse/specular, and retro-reflective responses, selected by the material’s coating and attribute bits. The default behavior for EM base materials 1-46.

RetroReflectiveMaterial

Specialized retro-reflective response (e.g. road signs).

AcousticMaterial

Mechanical-wave BSDF that uses bulk physical properties instead of optical ones.

ValidationMaterial

Test/debug BSDF gated by /app/sensors/nv/materials/validationBSDFMode; not intended for production content.

Optional BSDF features#

Most of these are controlled by the per-modality Settings:

  • Coatings and layers — additional material sheets (paint, clearcoat) layered on top of the base material. The presence of layers is driven purely by the coating USD attribute (the runtime encodes coatings into the material ID and resolveNumLayers reads them directly); enableMaterialInterLayerContribution is a separate flag that turns on full inter-layer reflection/transmission physics across those layers.

  • Phase and polarization — coherent tracking of phase and polarization state through material interactions. Enabled by enablePolarization (currently only wired for lidar).

  • Geometry-aware divergence — augments source-ray divergence using per-vertex normals/positions from rtxSensor (BSDF.h derives a geomDivergenceFactor from neighboring vertex normals). Enabled by enableRtxSensorGeometry (currently only the radar key is consumed, with cross-modality side effects on BSDF profile loading).

  • Continuation-ray output — BSDFs populate transmitted and reflected ray directions so the sensor model can fire secondary rays through transparent or partially transmissive surfaces.

  • Visual-band augmentation — diffuse reflectance and roughness from the visible-band material can be folded into the BSDF response. Enabled by enableRtxReflectanceInformation and enableAdditionalRtxReturnInformation.

Specifying Non-Visual Materials in USD#

A non-visual material is specified by adding three custom string attributes to a USD Material prim (typically under /Looks): a base material, an optional coating, and optional attributes. The geometry binds to that Material prim using the standard material:binding relationship.

def Cube "Cube3"
{
    rel material:binding = </Looks/new_enc>
    ...
}

def Scope "Looks"
{
    def Material "new_enc"
    {
        custom string omni:simready:nonvisual:base       = "aluminum"
        custom string omni:simready:nonvisual:coating    = "clearcoat"
        custom string omni:simready:nonvisual:attributes = "retroreflective"

        def Shader "Shader"
        {
            uniform asset info:mdl:sourceAsset = @OmniPBR.mdl@
            uniform token info:mdl:sourceAsset:subIdentifier = "OmniPBR"
            color3f inputs:diffuse_color_constant = (0.67, 0.67, 0.97)
            token outputs:out
        }
    }
}

Note

Only base is required. coating and attributes are optional and may be omitted or set to none. Multiple attributes may be combined in the attributes string (e.g. "retroreflective single_sided").

Note

The attribute prefix is configurable via the /rtx/materialDb/nonVisualMaterialSemantics/prefix setting (see Settings); the runtime concatenates this prefix with the :base/:coating/:attributes suffixes, so any prefix string is technically valid. The defaults shipped with content are omni:simready:nonvisual (current) and inputs:nonvisual (legacy).

Note

When the base material is calibration_lambertian, the shader’s inputs:diffuse_color_constant is consumed: the red channel defines the lambertian reflectance factor and the blue channel defines a surface-roughness perturbation (the green channel is currently unused). This consumption only happens through the DefaultMaterial BSDF and only on the electromagnetic spectrum path (lidar, radar): the default bindings put DefaultMaterial on IDs 0 and 47, with all other IDs on CompositeMaterial. On the acoustic spectrum path, IDs 0-46 bind to AcousticMaterial (which does not consume inputs:diffuse_color_constant) and ID 47 has no acoustic interpretation. Consumption also requires enableRtxReflectanceInformation to be enabled for the modality.

Supported Materials, Coatings, and Attributes#

Each USD label maps to an integer index. The runtime encodes the chosen base, coating, and attributes into a single 16-bit material ID that is consumed by the sensor’s BSDF (see Material ID Encoding).

Base Materials#

Material Name

Index

Description

none

0

Default, unlabeled, or unspecified

Metals

aluminum

1

Signs, poles, etc.

steel

2

Heavy construction metals

oxidized_steel

3

Rusted steel

iron

4

Manhole covers, drainage grills, etc.

oxidized_iron

5

Rusted iron

silver

6

Shiny metals

brass

7

Architecture

bronze

8

Statues, etc.

oxidized_bronze_patina

9

Old statues

tin

10

Food cans, etc.

Polymers

plastic

11

Generic plastics

fiberglass

12

Car bumpers, etc.

carbon_fiber

13

Car parts, bicycle parts, etc.

vinyl

14

Car interior, etc.

plexiglass

15

Light covers, etc.

pvc

16

Water tubing, etc.

nylon

17

Plastic bags, etc.

polyester

18

Some clothing, etc.

Glass

clear_glass

19

Glass that is clear with no contaminants

frosted_glass

20

Glass with volumetric particulates/imperfections

one_way_mirror

21

Building glass panels

mirror

22

Side mirrors, etc.

ceramic_glass

23

Heavy duty glass in construction

Other

asphalt

24

Roads, etc.

concrete

25

Construction

leaf_grass

26

Live vegetation

dead_leaf_grass

27

Dead vegetation

rubber

28

Tires, etc.

wood

29

Construction

bark

30

Trees, vegetation

cardboard

31

Boxes, etc.

paper

32

Newspapers, paper bags, writing paper, etc.

fabric

33

Clothing

skin

34

Human, pig, etc.

fur_hair

35

Human head, animal, etc.

leather

36

Clothing, car interior, etc.

marble

37

Construction

brick

38

Construction

stone

39

Nature, stones with structure

gravel

40

Nature, finer stones such as pebbles

dirt

41

Nature, very fine grains such as sand/dust

mud

42

Nature, wet dirt

water

43

Nature, water fountains, lakes, rivers, etc.

salt_water

44

Nature, oceans and seas, free from biologics

snow

45

Nature, frozen water droplets (crystalline)

ice

46

Nature, frozen water, larger bodies

calibration_lambertian

47

Special material with a defined diffuse reflectance, such as target panels with known reflectance

Coatings#

Coating Name

Index

Description

none

0

Default, unlabeled, or unspecified

paint

1

Painted

clearcoat

2

Clear-coated

paint_clearcoat

3

Painted and clear-coated

Indices 4-7 are reserved for future use.

Attributes#

Attributes are encoded as a bit field, so multiple attributes may apply to a single material. The “Bit value” column below is the value contributed to the attribute field of the material ID when that attribute is set.

Attribute Name

Bit value

Description

none

0

Unspecified attribute

emissive

1

Energy-emitting surface

retroreflective

2

Retro-reflective surface

single_sided

4

Single-sided surface (non-thin geometry)

visually_transparent

8

Material is visually transparent

Bit value 16 is reserved for future use.

Note

The visually_transparent attribute is currently inert in the shipped BSDFs: it is encoded into the material ID and decodable via isVisuallyTransparent in MaterialUtils.h, but no built-in BSDF currently changes its behavior based on it. It is reserved for content authoring and for downstream consumers that may opt in.

Material ID Encoding#

The runtime packs the chosen base, coating, and attributes into a single 16-bit material ID. The lower byte holds the base material; the upper byte holds the coating (3 bits) and attribute bit field (5 bits):

bit:  15  14  13  12  11   10   9   8    7   6   5   4   3   2   1   0
     +-----------------------+-------------+-------------------------------+
     |    attributes (5b)    | coating (3b)|       base material (8b)      |
     +-----------------------+-------------+-------------------------------+

Equivalently, given the indices from the tables above:

materialId = base | ((coating & 0x7) << 8) | ((attributeBits & 0x1F) << 11)

where attributeBits is the bitwise OR of the “Bit value” entries for every attribute listed in the USD attributes string.

Worked example. A material with base = "steel" (2), coating = "paint" (1), and attributes = "retroreflective" (bit value 2) encodes as:

base     = 0x02
coating  = 0x01 << 8           = 0x0100
attribs  = 0x02 << 11          = 0x1000
-----------------------------------------
materialId = 0x02 | 0x0100 | 0x1000 = 0x1102 = 4354

The preserveMaterialFlags setting (see Settings) is applied as a mask on the upper byte, allowing the coating and attribute bits to be selectively ignored at runtime.

Settings#

All settings below are carb settings and can be supplied as --/... command-line arguments at startup or set at runtime via the carb Python interface. The values shown are the defaults.

Per-modality settings#

The same settings are exposed independently for each of the three modalities under /app/sensors/nv/{lidar,radar,acoustic}/:

--/app/sensors/nv/lidar/enableRtxReflectanceInformation=false
--/app/sensors/nv/lidar/enableAdditionalRtxReturnInformation=false
--/app/sensors/nv/lidar/enableRtxSensorGeometry=false
--/app/sensors/nv/lidar/enablePolarization=false
--/app/sensors/nv/lidar/matBehaviorToIdOverrides=""
--/app/sensors/nv/lidar/matNameToIdMapOverrides=""

# Same settings exist under /app/sensors/nv/radar/...
# Same settings exist under /app/sensors/nv/acoustic/...

  • enableRtxReflectanceInformation - When true, rtxSensor allocates per-firing buffers for visible-band diffuse reflectance, specular reflectance, and shading normal. The lidar hit shader currently forwards only the diffuse reflectance (as NvMatInput::diffuseRefl) into the BSDF; specular and shading-normal buffers are reserved for future use. Required to consume inputs:diffuse_color_constant from the USD shader (notably for calibration_lambertian) and to layer paint reflectance over the base material.

  • enableAdditionalRtxReturnInformation - When true, rtxSensor additionally allocates per-firing buffers for visible-band index of refraction and roughness. The lidar hit shader forwards roughness (as NvMatInput::roughness) into the BSDF.

  • enableRtxSensorGeometry - Intended to enable per-vertex normals and positions in rtxSensor returns and to swap BSDFs over to a geometry-aware material profile (_dev.json).

  • enablePolarization - Intended to track polarization state through material interactions for coherent reflectance/transmittance.

  • matNameToIdMapOverrides / matBehaviorToIdOverrides - Remap the material name (data) and BSDF behavior bound to a given material ID. See Remappings.

Warning

The enableRtxSensorGeometry and enablePolarization keys exist under all three modality namespaces, but only a subset are actually consumed by the runtime today:

  • enableRtxSensorGeometry: only the radar key (/app/sensors/nv/radar/enableRtxSensorGeometry) is read. It both drives radar’s RTXSENSOR_RETURN_GEOMETRY and is checked by every BSDF profile loader (Composite/Core/RetroReflective/Acoustic/ Validation) when deciding whether to load the geometry-aware _dev.json material profile. The lidar and acoustic variants of this key are currently inert. Note that this means setting the radar key affects which material profiles are loaded for all sensor modalities, not just radar.

  • enablePolarization: only the lidar key (/app/sensors/nv/lidar/enablePolarization) is read. The radar and acoustic variants are currently inert.

Note

For the acoustic modality, although WpmAcoustic does set the RTXSENSOR_RETURN_MATERIAL_REFLECTANCE and RTXSENSOR_RETURN_MATERIAL_ADDITIONAL flags from enableRtxReflectanceInformation / enableAdditionalRtxReturnInformation, the AcousticMaterial BSDF does not currently read diffuse reflectance, roughness, or index of refraction (it operates on bulk physical properties). Enabling those flags for acoustic mainly adds GPU buffer cost without changing the BSDF result.

How per-modality settings affect output#

Note

These settings change what data flows into the BSDF, not the layout of GenericModelOutput. No new top-level fields or auxiliary data members are added when these are toggled. The effect appears as different values in the existing intensity/scalar field and in which hits pass the sensor’s detection threshold.

Concretely:

  • Lidar. BasicElements::scalar carries the per-point intensity. Enabling enableRtxReflectanceInformation lets the BSDF blend the visible-band diffuse term (and any USD-specified diffuse_color_constant for calibration_lambertian) into the reflectance, which directly scales the intensity. enableAdditionalRtxReturnInformation augments roughness, redistributing intensity between specular and diffuse lobes. enablePolarization is the only polarization key currently consumed and flows into the lidar BSDF as the CALC_PHASE_AND_POLARIZED material flag. Note: the lidar variant of enableRtxSensorGeometry is currently inert (see warning above).

  • Radar. Same BSDF inputs (diffuse reflectance, roughness) feed the WPM dmat-approx model; the result is a different RCS contribution per hit, which propagates to the radar detection list (range/Doppler/angle bins and detection scalar). Enabling these settings does not add new detection fields; it changes their values and which detections survive thresholding. enableRtxSensorGeometry for radar is consumed and enables geometry-aware RCS computation and the _dev.json material profiles. The radar variant of enablePolarization is currently inert.

  • Acoustic. BasicElements::scalar carries the received pressure level. The shipped AcousticMaterial BSDF operates on bulk physical properties (compressibility, density) and does not currently consume the visible-band diffuse/specular/roughness/IoR data, so enabling enableRtxReflectanceInformation and enableAdditionalRtxReturnInformation mainly adds GPU buffer cost without changing intensity. enableRtxSensorGeometry and enablePolarization are not consumed by the acoustic plugin at present.

Note

The lidar materialId auxiliary field (LidarAuxiliaryData::matId) is unrelated to these settings; its presence is governed by the /app/sensors/nv/lidar/auxOutputType setting (EXTRA or FULL).

Warning

Each enabled flag allocates additional GPU buffers per firing:

  • enableRtxReflectanceInformation: three separate per-firing GPU buffers (diffuse, specular, shading-normal), each float3 per firing, totalling 9 floats per firing.

  • enableAdditionalRtxReturnInformation: two scalar channels (IoR, roughness) = 2 floats/firing.

  • enableRtxSensorGeometry: per-firing vertex normals and positions.

Enable only the data the chosen BSDFs actually consume.

Global material settings#

--/app/sensors/nv/materials/resetMaterials=false
--/app/sensors/nv/materials/enableMaterialInterLayerContribution=false
--/app/sensors/nv/materials/preserveMaterialFlags=0xff

--/rtx/materialDb/nonVisualMaterialSemantics/prefix=omni:simready:nonvisual

  • resetMaterials - Set to true to re-process the material database after applying remappings at runtime.

  • enableMaterialInterLayerContribution - Enables inter-layer reflectance and transmittance calculations across stacked material layers.

  • preserveMaterialFlags - Bit mask filtering the upper byte of the encoded material ID (the byte that holds coating and attribute bits). Default 0xff honors all coating/attribute flags; setting it to 0 strips them and uses only the base material.

  • /rtx/materialDb/nonVisualMaterialSemantics/prefix - Selects the USD attribute prefix used to discover non-visual material labels. Set to inputs:nonvisual for legacy content; otherwise leave at the default omni:simready:nonvisual.

Remappings (advanced)#

The default mapping from base material names to material IDs (and the BSDF behavior bound to each ID) can be overridden per modality. Only IDs being changed need to be listed.

For example, to remap material IDs 5 and 6 for the lidar sensor:

--/app/sensors/nv/lidar/matBehaviorToIdOverrides="CompositeMaterial:5;CoreMaterial:6"
--/app/sensors/nv/lidar/matNameToIdMapOverrides="asphalt:5;aluminum:6"

This binds material ID 5 to the asphalt properties evaluated by the CompositeMaterial BSDF, and ID 6 to aluminum evaluated by the CoreMaterial BSDF (overriding the defaults of oxidized_iron and silver).

Remappings can be supplied at startup or applied at runtime. To apply at runtime, set both override settings first and then set:

--/app/sensors/nv/materials/resetMaterials=true

Warning

resetMaterials only takes effect after the override settings have been written. If toggled first, the reset uses the previous (or default) mapping.

Note

resetMaterials is self-clearing: once read by the runtime, it is automatically set back to false. Inspecting the setting after a reset will therefore show false, not the value you wrote. Always write true again to trigger another reset.

Troubleshooting#

Note

All content has the same material. The USD content is most likely using the legacy prefix inputs:nonvisual while the runtime is configured for omni:simready:nonvisual. Set:

--/rtx/materialDb/nonVisualMaterialSemantics/prefix=inputs:nonvisual

(or re-author the USD with the default prefix).

Note

Coating or attribute flags are being ignored. Check --/app/sensors/nv/materials/preserveMaterialFlags. If it is 0, only the base material is honored. If it is set as expected, verify the USD string tokens against the tables above (typos in the label silently fall back to none).

Note

Inspect material IDs in a viewport. Open RenderSettings -> Common in kit, then View -> DebugView -> Non-Visual Material ID. This renders the non-visual materials as false color and supports picking, which is useful for verifying content-to-ID mappings.