ROS Bridge

ROS (Robot Operating System) is a popular platform for building robotic systems. The ROS Bridge Extension enables publishing and subscribing of several rostopics and rosservices that are commonly needed for robotic simulation.

Run the Sample

ROS Package

Before running the samples, download the issac_bridge rospackage here, put it into one of your ROS workspaces and catkin_build it. This package contains the definition of a custom IsaacPose service type, as well as several sample scripts. Go to ROS website for tutorials on building ROS workspaces and packages.

ROS Topics

Enable extension omni.isaac.ros_bridge from the extension manager menu Windows->Extension Manager. Roscore will immediately start running when the extension is loaded. If the extension window is not already docked, you can dock it into the bottom tabs, or anywhere else you prefer.

Note

Improper shutdown of kit can cause lingering roscore, use your task manager to kill the corresponding processes if necessary.

When the ros_bridge extension is loaded, a ROS sample will automatically show up under Isaac Robotics. This is a quick demo to show a working ROS bridge communicating camera, joint state, and tf information through rostopics. We will walk through how to build the bridge in the later sections. Open the Rostopics sample. Follow the menu, first load the robot.

pic1 pic2

Wait for the robot to load. Then Connect Joint State Topics and Connect Camera Topics.

ROS_JointState and ROS_Camera prims will be added to bottom of the stage tree, like so:

../_images/isaac_ros_stage.png

Camera

To look at the images that are published, in separate terminals:

rosrun image_view image_view topic:=/rgb

rosrun image_view image_view topic:=/depth

The published images should follow if you move the viewport camera in kit. If the depth image is not immediately obvious, move the camera to make sure the camera only sees the ground plane and the robot, and avoid anything on the horizon that is infinitely far.

Joint States

The robot’s joint state are being published on the topic /joint_states.

It is also subscribing to /joint_commmand topic, on which you can send commands to move the robot. The script src/ros_publisher.py in the isaac_bridge package you downloaded earlier gives an example of how to populate a JointState message and publish it. Run it to move the robot via position control.

TF

Continue to Connect TF Topics. A ROS_PoseTree prim will be added to the bottom of the stage tree. Transforms associated with the robot will be published onto the /tf topic.

Add Cube will add a cube onto the stage, and the cube’s pose will be appended to the existing ROS_PoseTree. The /tf topic now will be publishing both the transforms of the robot as well as the cube.

ROS Services

This is a quick demo to show objects inside kit being teleported to different poses via a ROS service.

Go to Isaac Robotics->ROS, open Rosservices.

Add Cube will add a cube onto the stage. A teleport_pos rosservice will be established automatically. In the isaac_bridge package, src/ros_pose_client.py sends periodic service requests to move the cube. Run it to see the cube move.

Add Cone will add a cone onto the stage. A separate rosservice teleport_pos_cone will be established for the cone automatically. src/ros_pose_client_2.py will move the cone.

If you run both clients simultaneously, both objects will move.

Add a Camera Topic

ROS topics can be added via python scripts, as well as manually through the GUI. Here we explain how to add a camera topic through the GUI.

To add a camera, go to Create->Isaac->ROS and add Camera.

../_images/isaac_ros_add_ros.png

Select ROS_Camera on your Stage Tree, then the Details tab will show the specs of the ROS camera. Relevant items are published to the topic /camera_info. You can chose to enable RGB or depth or both. They will be published on the topic /rgb and /depth respectively.

../_images/isaac_ros_camera_details.png

Press play on the editor. The camera associated with the current viewport is now being published. You can change the viewport’s camera by going to Cameras at the upper left corner of the viewport, and select desired camera view.

To look at the images that are published, in separate terminal:

rosrun image_view image_view topic:=/rgb

rosrun image_view image_view topic:=/depth

The published images should follow if you move the viewport camera in kit. Move the camera around if the depth image is not immediately visible. Make sure the camera only sees the ground and the foreground object.

Note

Currently, only one camera at a time is supported. Multi-camera support will be featured in future releases.

Add a Lidar Topic

To add a lidar topic, we first need to add a simulated lidar sensor.

Add a Simulated Lidar Sensor

To add a simulated Lidar, go to Create->Isaac->Sensors->Lidar, and add Lidar. The Lidar sensor will only work if physics is enabled in the stage. So make sure there is a physicsScene in your stage tree. If not, to go to Physics -> Add -> physicsScene to enable physics.

To visualize the Lidar rays, go to the Details tab, enable drawLidarPoints. Then press play in the editor. If the spinning rays are giving you a headache, you can set the rotationRate to 0, then the rays will become stationary. Please refer to LIDAR Simulation for more details on the simulated LIDAR.


We will also add an simple object into the scene for easier visualization. Go to Create->Mesh->Cube. If for some reason you don’t see the cube immediately, zoom out couple times.

The newly added cube will not show up in lidar without collision properties enabled. So select the cube on the stage tree, and go into the PhysX Properties tab. In the dropdown list of Add Prim Component, select CollisionAPI. Now the cube becomes detectable by lidar.

Add a Lidar Rostopic

To add a lidar rostopic, go to Create->Isaac->ROS and add Lidar. The rostopic for lidar /laser_scan will automatically start. However, it will not have any messages initially. We need to connect the newly added ROS lidar topic to the simulated lidar. To do this, we first need to open up the Relationship Editor by going to Window->Isaac->Relationship Editor. Select ROS_Lidar on the stage tree, and open up the relationship editor tab. (also see Relationship Editor UI) Under lidarPrim, add the path to the simulated Lidar. The rostopic /laser_scan should now be publishing the data from the simulated lidar. Make sure the editor is playing.


If you wish to visualize this in Rviz, then make sure to set the LaserScan topic to /laser_scan, and Fixed Frame to /sim_lidar.

Add Joint State Topics

Assuming you have a robot loaded on stage. Before adding the rostopic for joint states, make sure the robot is rigged with articulation. The ROS bridge can recognize an articulation tree and publish the states of all the joints associated with a given articulation tree.

../_images/isaac_ros_js_articulation.png

Add Joint State Publisher

To add a joint state rostopic, go to Create->Isaac->ROS and add Joint State. The rostopics /joint_state and /joint_command will automatically start. /joint_state is the topic on which kit will broadcast the simulated robot’s joint state. /joint_command is the topic that the simulated robot is listening to for joint state commands.

Nothing will be published on the /joint_state until it is connected to the robot of your choice. To do so, we first need to open up the Relationship Editor by going to Window->Isaac->Relationship Editor. Select ROS_JointState on the stage tree, and open up the relationship editor tab. Under articulationPrim, add the path to the root of the articulation tree. Press play on the editor, and joint information will start publishing on the /joint_state rostopic.


Add Joint Command Subscriber

The simulated robot is simultaneously listening to /joint_command topic for joint commands. The /joint_command topic takes input in JointState ROS message format. In the isaac_bridge rospackage, src/ros_publisher.py gives an example of how such message can be generated. Run the script to see the simulated Franka Panda robot move.

Add TF Topics

To add a tranform tree rostopic:

  • Go to Create->Isaac->ROS and add Pose Tree. The rostopic /tf will automatically start.

  • Open Relationship Editor by going to Window->Isaac->Relationship Editor. Select ROS_PoseTree on the stage tree, and open the relationship editor tab. Under targetPrims, add the path to the robot’s articulation tree root.

  • Press play on the editor, and the tranforms of each of the robot’s frame will start publishing on the /tf rostopic.

  • Multiple objects can be attached to the same /tf rostopic by adding the corresponding paths under the targetPrims.

../_images/isaac_ros_pose_tree_multi.png

Add Pose Teleport Services

To add a teleport service for any object:

  • Go to Create->Isaac->ROS and add Teleport. A /ROS_Telport prim will appear in the stage tree. The rosservice /teleport_pos will automatically start.

  • To connect up the service to the object in kit, open Relationship Editor by going to Window->Isaac->Relationship Editor. Then select ROS_Teleport on the stage tree. In the Relationship Editor tab. Add the path to the object you wish to control.

  • Press play on the editor, and the pose service is ready to receive pose requests.


The Pose Teleport Service uses a custom IsaacPose rosservice message. Download the isaac_bridge rospackage, put it into one of your ROS workspaces and catkin_build it. src/ros_pose_client.py and src/ros_pose_client_2.py in the isaac_bridge rospackage are examples of how to populate and send this message.

Currently, a single teleport service can only be connected to a single object. To add more objects, you must add more services. Make sure to have different names of the poseSrvTopic when creating new ones so they don’t interfere each other.

../_images/isaac_ros_service_name_change.png

Python API

ROS topics can also be added using the Python API.

Add a Camera Rostopic

Versions of the code below can be found in the joint state sample’s script. This can also be ran inside kit’s Script Editor after the Franka Panda robot is already loaded onto the stage.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
    from pxr import PhysicsSchema, Sdf
    import omni.usd
    import omni.kit
    import omni.isaac.RosBridgeSchema as ROSSchema

    stage = omni.usd.get_context().get_stage()
    camera_prim = ROSSchema.RosCamera.Define(stage, Sdf.Path("/ROS_Camera"))

    # adding prefix to the publisher topic in case multiple cameras are added
    camera_prim.CreateRosNodePrefixAttr("")
    camera_prim.CreateEnabledAttr(True)

    # publisher topic for camera_info
    camera_prim.CreateCameraInfoPubTopicAttr("/camera_info")
    # publisher topic for rgb
    camera_prim.CreateRgbPubTopicAttr("/rgb")
    # publisher topic for depth
    camera_prim.CreateDepthPubTopicAttr("/depth")
    camera_prim.CreateFrameIdAttr("/sim_camera")

    # enable RGB
    camera_prim.CreateRgbEnabledAttr(True)
    # enable depth
    camera_prim.CreateDepthEnabledAttr(True)
    camera_prim.CreateQueueSizeAttr(10)

Remember the editor must be playing for the rostopic to be active. This can also be done via Python.

1
2
3
    editor = omni.kit.editor.get_editor_interface()
    if not editor.is_playing():
        editor.play()

Add a Joint State Rostopic

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    stage = omni.usd.get_context().get_stage()
    robot_prim = stage.GetPrimAtPath("/panda")

    # check the robot has articulation
    assert robot_prim.HasAPI(PhysicsSchema.ArticulationAPI)

    # setup Rostopics to publish and receive joint state info
    js_prim = ROSSchema.RosJointState.Define(stage, Sdf.Path("/ROS_JointState"))

    # adding prefix to the published /joint_state topic if needed
    js_prim.CreateRosNodePrefixAttr("")
    js_prim.CreateEnabledAttr(True)

    # publisher topic
    js_prim.CreateJointStatePubTopicAttr("/joint_state")
    # subscripber topic
    js_prim.CreateJointStateSubTopicAttr("/joint_command")

    js_prim.CreateArticulationPrimRel()
    js_prim.CreateQueueSizeAttr(0)

    # The joint_state rostopic must be connected to the root of the robot's articulation in order to publish its states
    ROS_prim = stage.GetPrimAtPath("/ROS_JointState")
    ROS_prim.GetRelationship("articulationPrim").SetTargets(["/panda"])


    # editor must be playing for messages to be published and received
    editor = omni.kit.editor.get_editor_interface()
    if not editor.is_playing():
        editor.play()

IsaacPose Service Type

The Pose Teleport Service uses a custom IsaacPose rosservice type:

1
2
3
4
5
6
    std_msgs/Header header
    string[] names
    geometry_msgs/Pose[] poses
    geometry_msgs/Twist[] velocities
    geometry_msgs/Vector3[] scales
    ---

It must be built inside a rospackage named isaac_bridge. Go to ROS website for more details on building and using ROS Services.