16. Publish RTX Lidar Point Cloud

RTX or Raytraced lidar is an alternative to the PhysX lidar in Isaac Sim. It supports both Solid State and Rotating Lidar configuration via a JSON config file. Each RTX Sensor must be attached to its own viewport to simulate properly.

16.1. Learning Objectives

In this example, we will

  • Have a brief introduction on how to use RTX Lidar sensors

  • Create a RTX Lidar sensor

  • Publish sensor data to ROS as a PointCloud2 message

Prerequisite

16.2. Running The Example

  • In one terminal source your ROS workspace and run roscore

  • In a new terminal with your ROS environment sourced, run rviz -d ros_workspace/src/isaac_tutorials/rviz/rtx_lidar.rviz to start rviz and show the lidar point cloud

  • Finally, run the sample script

    ./python.sh standalone_examples/api/omni.isaac.ros_bridge/rtx_lidar.py
    

Once the scene finishes loading you should see the point cloud for the rotating lidar sensor being simulated

16.3. RTX Lidar Basics

While most of the sample code is faily generic, there are a few specific pieces needed to create and simulate the sensor

In order to activate a RTX Lidar sensor we first have to create a second viewport that the sensor will be attached to

vpi = omni.kit.viewport_legacy.get_viewport_interface()
instance = vpi.create_instance()
viewport_handle = vpi.get_instance("Viewport 2")
viewport = vpi.get_viewport_window(viewport_handle)

In order for the sensor to generate data properly we let the viewport know that it should create a buffer for the associated render variable. The RtxSensorCpu render variable, or AOV, is where all RTX Sensor simulated sensors, like RTX Lidar write their output data to.

viewport.add_aov("RtxSensorCpu", False)

We can then create the post process pipeline that takes the rendered RTX Lidar data and publishes it to ROS:

sensors.get_synthetic_data().activate_node_template(
    "RtxSensorCpu" + "ROS1PublishPointCloud", 0, [viewport.get_render_product_path()]
)

Note

You can specify an optional attributes={…} dictionary when calling activate_node_template to set node specific parameters. See the API Documentation for complete usage information.

And then finally create the sensor prim itself, and specify its configuration file. Note that the sensor is rotated so that it is created in a Z up configuration.

_, (_, sensor) = omni.kit.commands.execute(
    "IsaacSensorCreateRtxLidar",
    path="/sensor",
    parent=None,
    config="Example_Rotary",
    translation=(0, 0, 1.0),
    orientation=Gf.Quatd(0.707, 0.707, 0, 0),
)

Here Example_Rotary defines the configuration for the lidar sensor. The two generic configuration files provided are located in exts/omni.drivesim.sensors.nv.lidar/data. Example_Rotary.json and Example_Solid_State.json. To switch the lidar to the example solid state configuration you can replace config=”Example_Rotary”, with config=”Example_Solid_State”.

The process of using a RTX Lidar sensor is a bit manual currently, we are working on improvements and APIs to access the raw sensor data as well as documentation about the RTX Lidar configuration files themselves for future releases.

16.4. Summary

This tutorial covered

  • Creating a new viewport and enabling the correct render variable for RTX Lidar

  • Enabling the ROS1PublishPointCloud post process pipeline that will publish the PCL data

  • Creating an RTX Lidar sensor and specifying its configuration