RTX Lidar Sensor
RTX Lidar sensors are simulated at render time on the GPU with RTX hardware.
Their results are then copied to the
RtxSensorCpu rendering buffer for use.
How They work
Before a RTX Lidar can render, there must be a camera asset with attributes set appropriately.
The render product from that camera will contain the
The following code snippet, when run in the Script Editor, will:
create a RTX Lidar sensor in the scene with the
render its results into a Hydra texture
create a point cloud from the results using the
RtxSensorCpuIsaacCreateRTXLidarScanBufferannotator that can read the data from the lidar.
import omni.kit.commands from pxr import Gf from omni.isaac.core.utils.render_product import create_hydra_texture import omni.replicator.core as rep lidar_config = "Example_Rotary" # 1. Create The Camera _, sensor = omni.kit.commands.execute( "IsaacSensorCreateRtxLidar", path="/sensor", parent=None, config=lidar_config, translation=(0, 0, 1.0), orientation=Gf.Quatd(1,0,0,0), ) # 2. Create and Attach a render product to the camera _, render_product_path = create_hydra_texture([1, 1], sensor.GetPath().pathString) # 3. Create a Replicator Writer that "writes" points into the scene for debug viewing writer = rep.writers.get("RtxLidarDebugDrawPointCloudBuffer") writer.attach([render_product_path]) # 4. Create Annotator to read the data from with annotator.get_data() annotator = rep.AnnotatorRegistry.get_annotator("RtxSensorCpuIsaacCreateRTXLidarScanBuffer") annotator.attach([render_product_path])
The action graph this creates hooks all the nodes up and makes this work when you hit play. You may want to explore an overview of the Action Graph it created, as well as details on the nodes specifically created for RTX Lidar.
The most important parameter to the
IsaacSensorCreateRtxLidar command is
LiDAR Config Files
config parameter is important, because it points to a JSON file, described below, that defines the behavior of the RTX Lidar. All the intrinsic parameters for the lidar are
stored in the config file. It is loaded when the sensor is created. It’s because of this process that the properties of an RTX Lidar, unlike most other properties in Isaac Sim,
can’t be changed once the RTX Lidar is created.
app.sensors.nv.lidar.profileBaseFolder setting in the
./exts/omni.isaac.sensor/config/extension.toml file holds the multiple search paths for the configuration file, and the first
JSON file found named
config is used. So use unique names for your lidar config files! There are several config files in
are a good staring place for making your own. The
app.sensors.nv.lidar.profileBaseFolder setting defaults to that location.
There are two scanning principles for RTX Lidar: Rotary, and Solid State. The
scanType parameter in the config file sets the scanning principle.
RTX Lidar Config Library
A library of several pre-made lidar config files are shipped with Omniverse Isaac Sim. These are designed with a best guess approximation of the sensors from the publicly available manuals and web pages, unless otherwise noted.
You will find the pre-made config files in the
./exts/omni.isaac.sensor/data/lidar_configs/ folder, in a sub-folder named for the company that makes each lidar model.
OS0 128 10hz @ 1024 resolution
OS0 128 10hz @ 2048 resolution
OS0 128 10hz @ 512 resolution
OS0 128 20hz @ 1024 resolution
OS0 128 20hz @ 512 resolution
OS1 32 10hz @ 1024 resolution
OS1 32 10hz @ 2048 resolution
OS1 32 10hz @ 512 resolution
OS1 32 20hz @ 1024 resolution
OS1 32 20hz @ 512 resolution
Simple Solid State
The Lidar config JSON files are worth looking into, because many of them have comments in them detailing where the parameters came from, so if you need to make your own config file, or change existing ones, you will have an idea of why the values are set as they are.
The material system for RTX LiDAR allows content creators to assign sensor material types, to partial material prim names on a USD stage. The materials will make the beams react different when they hit surfaces.
For examples of creating RTX Lidar see the examples:
./python.sh standalone_examples/api/omni.isaac.debug_draw/rtx_lidar.py Example_Rotary
./python.sh standalone_examples/api/omni.isaac.debug_draw/rtx_lidar.py Example_Solid_State
See the Isaac Sim Conventions documentation for a complete list of Omniverse Isaac Sim conventions.