ROS (Robot Operating System) is a popular platform for building robotic systems. The ROS Bridge Extension enables publishing and subscribing of several rostopics and rosservices that are commonly needed for robotic simulation.
Run the Sample¶
Before running the samples, download the
here, put it into one of your ROS workspaces and catkin_build it. This package contains the definition of a custom
IsaacPose service type, as well as several sample scripts. Go to ROS website for tutorials on building ROS workspaces and packages.
Enable extension omni.isaac.ros_bridge from the extension manager menu Windows->Extension Manager. Roscore will immediately start running when the extension is loaded. If the extension window is not already docked, you can dock it into the bottom tabs, or anywhere else you prefer.
Improper shutdown of kit can cause lingering roscore, use your task manager to kill the corresponding processes if necessary.
When the ros_bridge extension is loaded, a ROS sample will automatically show up under Isaac Robotics. This is a quick demo to show a working ROS bridge communicating camera, joint state, and tf information through rostopics. We will walk through how to build the bridge in the later sections. Open the Rostopics sample. Follow the menu, first load the robot.
Wait for the robot to load. Then
Connect Joint State Topics and
Connect Camera Topics.
ROS_JointState and ROS_Camera prims will be added to bottom of the stage tree, like so:
To look at the images that are published, in separate terminals:
rosrun image_view image_view topic:=/rgb
rosrun image_view image_view topic:=/depth
The published images should follow if you move the viewport camera in kit. If the depth image is not immediately obvious, move the camera to make sure the camera only sees the ground plane and the robot, and avoid anything on the horizon that is infinitely far.
The robot’s joint state are being published on the topic
It is also subscribing to
/joint_commmand topic, on which you can send commands to move the robot. The script
src/ros_publisher.py in the isaac_bridge package you downloaded earlier gives an example of how to populate a JointState message and publish it. Run it to move the robot via position control.
Connect TF Topics. A ROS_PoseTree prim will be added to the bottom of the stage tree. Transforms associated with the robot will be published onto the
Add Cube will add a cube onto the stage, and the cube’s pose will be appended to the existing ROS_PoseTree. The /tf topic now will be publishing both the transforms of the robot as well as the cube.
This is a quick demo to show objects inside kit being teleported to different poses via a ROS service.
Go to Isaac Robotics->ROS, open Rosservices.
Add Cube will add a cube onto the stage. A teleport_pos rosservice will be established automatically. In the isaac_bridge package,
src/ros_pose_client.py sends periodic service requests to move the cube. Run it to see the cube move.
Add Cone will add a cone onto the stage. A separate rosservice teleport_pos_cone will be established for the cone automatically.
src/ros_pose_client_2.py will move the cone.
If you run both clients simultaneously, both objects will move.
Add a Camera Topic¶
ROS topics can be added via python scripts, as well as manually through the GUI. Here we explain how to add a camera topic through the GUI.
To add a camera, go to Create->Isaac->ROS and add Camera.
Select ROS_Camera on your Stage Tree, then the Details tab will show the specs of the ROS camera. Relevant items are published to the topic /camera_info. You can chose to enable RGB or depth or both. They will be published on the topic
Press play on the editor. The camera associated with the current viewport is now being published. You can change the viewport’s camera by going to Cameras at the upper left corner of the viewport, and select desired camera view.
To look at the images that are published, in separate terminal:
rosrun image_view image_view topic:=/rgb
rosrun image_view image_view topic:=/depth
The published images should follow if you move the viewport camera in kit. Move the camera around if the depth image is not immediately visible. Make sure the camera only sees the ground and the foreground object.
Currently, only one camera at a time is supported. Multi-camera support will be featured in future releases.
Add a Lidar Topic¶
To add a lidar topic, we first need to add a simulated lidar sensor.
Add a Simulated Lidar Sensor¶
To add a simulated Lidar, go to Create->Isaac->Sensors->Lidar, and add Lidar. The Lidar sensor will only work if physics is enabled in the stage. So make sure there is a physicsScene in your stage tree. If not, to go to Physics -> Add -> physicsScene to enable physics.
To visualize the Lidar rays, go to the Details tab, enable drawLidarPoints. Then press play in the editor. If the spinning rays are giving you a headache, you can set the rotationRate to 0, then the rays will become stationary. Please refer to LIDAR Simulation for more details on the simulated LIDAR.
We will also add an simple object into the scene for easier visualization. Go to Create->Mesh->Cube. If for some reason you don’t see the cube immediately, zoom out couple times.
The newly added cube will not show up in lidar without collision properties enabled. So select the cube on the stage tree, and go into the PhysX Properties tab. In the dropdown list of Add Prim Component, select CollisionAPI. Now the cube becomes detectable by lidar.
Add a Lidar Rostopic¶
To add a lidar rostopic, go to Create->Isaac->ROS and add Lidar. The rostopic for lidar /laser_scan will automatically start. However, it will not have any messages initially. We need to connect the newly added ROS lidar topic to the simulated lidar. To do this, we first need to open up the Relationship Editor by going to Window->Isaac->Relationship Editor. Select ROS_Lidar on the stage tree, and open up the relationship editor tab. (also see Relationship Editor UI) Under lidarPrim, add the path to the simulated Lidar. The rostopic /laser_scan should now be publishing the data from the simulated lidar. Make sure the editor is playing.
If you wish to visualize this in Rviz, then make sure to set the LaserScan topic to /laser_scan, and Fixed Frame to /sim_lidar.
Add Joint State Topics¶
Assuming you have a robot loaded on stage. Before adding the rostopic for joint states, make sure the robot is rigged with articulation. The ROS bridge can recognize an articulation tree and publish the states of all the joints associated with a given articulation tree.
Add Joint State Publisher¶
To add a joint state rostopic, go to Create->Isaac->ROS and add Joint State. The rostopics /joint_state and /joint_command will automatically start. /joint_state is the topic on which kit will broadcast the simulated robot’s joint state. /joint_command is the topic that the simulated robot is listening to for joint state commands.
Nothing will be published on the /joint_state until it is connected to the robot of your choice. To do so, we first need to open up the Relationship Editor by going to Window->Isaac->Relationship Editor. Select ROS_JointState on the stage tree, and open up the relationship editor tab. Under articulationPrim, add the path to the root of the articulation tree. Press play on the editor, and joint information will start publishing on the /joint_state rostopic.
Add Joint Command Subscriber¶
The simulated robot is simultaneously listening to /joint_command topic for joint commands. The /joint_command topic takes input in
JointState ROS message format. In the isaac_bridge rospackage,
src/ros_publisher.py gives an example of how such message can be generated. Run the script to see the simulated Franka Panda robot move.
Add TF Topics¶
To add a tranform tree rostopic:
Go to Create->Isaac->ROS and add Pose Tree. The rostopic /tf will automatically start.
Open Relationship Editor by going to Window->Isaac->Relationship Editor. Select ROS_PoseTree on the stage tree, and open the relationship editor tab. Under targetPrims, add the path to the robot’s articulation tree root.
Press play on the editor, and the tranforms of each of the robot’s frame will start publishing on the /tf rostopic.
Multiple objects can be attached to the same /tf rostopic by adding the corresponding paths under the targetPrims.
Add Pose Teleport Services¶
To add a teleport service for any object:
Go to Create->Isaac->ROS and add Teleport. A /ROS_Telport prim will appear in the stage tree. The rosservice /teleport_pos will automatically start.
To connect up the service to the object in kit, open Relationship Editor by going to Window->Isaac->Relationship Editor. Then select ROS_Teleport on the stage tree. In the Relationship Editor tab. Add the path to the object you wish to control.
Press play on the editor, and the pose service is ready to receive pose requests.
The Pose Teleport Service uses a custom
IsaacPose rosservice message. Download the isaac_bridge
rospackage, put it into one of your ROS workspaces and catkin_build it.
src/ros_pose_client_2.py in the isaac_bridge rospackage are examples of how to populate and send this message.
Currently, a single teleport service can only be connected to a single object. To add more objects, you must add more services. Make sure to have different names of the
poseSrvTopic when creating new ones so they don’t interfere each other.
ROS topics can also be added using the Python API.
Add a Camera Rostopic¶
Versions of the code below can be found in the joint state sample’s script. This can also be ran inside kit’s Script Editor after the Franka Panda robot is already loaded onto the stage.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
from pxr import PhysicsSchema, Sdf import omni.usd import omni.kit import omni.isaac.RosBridgeSchema as ROSSchema stage = omni.usd.get_context().get_stage() camera_prim = ROSSchema.RosCamera.Define(stage, Sdf.Path("/ROS_Camera")) # adding prefix to the publisher topic in case multiple cameras are added camera_prim.CreateRosNodePrefixAttr("") camera_prim.CreateEnabledAttr(True) # publisher topic for camera_info camera_prim.CreateCameraInfoPubTopicAttr("/camera_info") # publisher topic for rgb camera_prim.CreateRgbPubTopicAttr("/rgb") # publisher topic for depth camera_prim.CreateDepthPubTopicAttr("/depth") camera_prim.CreateFrameIdAttr("/sim_camera") # enable RGB camera_prim.CreateRgbEnabledAttr(True) # enable depth camera_prim.CreateDepthEnabledAttr(True) camera_prim.CreateQueueSizeAttr(10)
Remember the editor must be playing for the rostopic to be active. This can also be done via Python.
1 2 3
editor = omni.kit.editor.get_editor_interface() if not editor.is_playing(): editor.play()
Add a Joint State Rostopic¶
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
stage = omni.usd.get_context().get_stage() robot_prim = stage.GetPrimAtPath("/panda") # check the robot has articulation assert robot_prim.HasAPI(PhysicsSchema.ArticulationAPI) # setup Rostopics to publish and receive joint state info js_prim = ROSSchema.RosJointState.Define(stage, Sdf.Path("/ROS_JointState")) # adding prefix to the published /joint_state topic if needed js_prim.CreateRosNodePrefixAttr("") js_prim.CreateEnabledAttr(True) # publisher topic js_prim.CreateJointStatePubTopicAttr("/joint_state") # subscripber topic js_prim.CreateJointStateSubTopicAttr("/joint_command") js_prim.CreateArticulationPrimRel() js_prim.CreateQueueSizeAttr(0) # The joint_state rostopic must be connected to the root of the robot's articulation in order to publish its states ROS_prim = stage.GetPrimAtPath("/ROS_JointState") ROS_prim.GetRelationship("articulationPrim").SetTargets(["/panda"]) # editor must be playing for messages to be published and received editor = omni.kit.editor.get_editor_interface() if not editor.is_playing(): editor.play()
IsaacPose Service Type¶
The Pose Teleport Service uses a custom
IsaacPose rosservice type:
1 2 3 4 5 6
std_msgs/Header header string names geometry_msgs/Pose poses geometry_msgs/Twist velocities geometry_msgs/Vector3 scales ---
It must be built inside a rospackage named isaac_bridge. Go to ROS website for more details on building and using ROS Services.