5. Transferring Policies from Isaac Gym Preview Releases

Current users of Isaac Gym Preview Releases may be interested in transferring a policy trained using an Isaac Gym Preview Release to Isaac Sim with the new Omniverse Isaac Gym. This section delineates some of the differences between the Isaac Gym Preview Releases and the Isaac Sim RL extensions in hopes of facilitating the process of transferring policies trained in the Isaac Gym Preview Releases to Isaac Sim.

5.1. Learning Objectives

In this tutorial, we will walk through some key differences between Isaac Gym Preview Releases and Isaac Sim. We will cover

  1. Differences in quaternion convention

  2. Differences in joint ordering

  3. Tuning physics parameters on assets

5-10 Minute Tutorial

5.2. Getting Started

  • Please refer to previous tutorials in the Reinforcement Learning Tutorials series to learn about RL extensions in Isaac Sim.

  • Please refer to Isaac Gym for more details on Standalone Isaac Gym.

5.3. Quaternion Convention

The Isaac Sim RL extensions use various classes and methods in omni.isaac.core, which adopts wxyz as the quaternion convention. However, the quaternion convention used in Isaac Gym Preview Releases is xyzw. Therefore, if a policy trained in one of the Isaac Gym Preview Releases takes in quaternions as part of its observations, remember to switch all quaternions to use the xyzw convention in the observation buffer self.obs_buf. Similarly, please ensure all quaternions are in wxyz before passing them in any of the utility functions in omni.isaac.core.

5.4. Joint Order

Isaac Sim’s ArticulationView in omni.isaac.core assumes a breadth-first ordering for the joints in a given kinematic tree. Specifically, for the following kinematic tree, the method ArticulationView.get_joint_positions returns a tensor of shape (number of articulations in the view, number of joints in the articulation). Along the second dimension of this tensor, the values represent the articulation’s joint positions in the following order: [Joint 1, Joint 2, Joint 4, Joint 3, Joint 5]. On the other hand, the Isaac Gym Preview Releases assume a depth-first ordering for the joints in the kinematic tree; In the example below, the joint orders would be the following: [Joint 1, Joint 2, Joint 3, Joint 4, Joint 5].

../_images/isaac_sim_rl_transfer_kinematic_tree.png

With this in mind, it is important to change the joint order to depth-first in the observation buffer before feeding it into an existing policy trained in one of the Isaac Gym Preview Releases. Similarly, you would also need to change the joint order in the output (the action buffer) of the Isaac Gym Preview Release trained policy to breadth-first before applying joint actions to articulations via methods in ArticulationView.

5.5. Physics Parameters

One factor that could dictate the success of policy transfer from Isaac Gym Preview Releases to Isaac Sim is to ensure the physics parameters used in both simulations are identical or very similar. In general, the sim parameters specified in the task configuration yaml file overwrite the corresponding parameters in the USD asset. However, there are additional parameters in the USD asset that are not included in the task configuration yaml file. These additional parameters may sometimes impact the performance of Isaac Gym Preview Release trained policies and hence need modifications in the USD asset itself to match the values set in Isaac Gym Preview Releases.

For instance, the following parameters in the RigidBodyAPI could be modified in the USD asset to yield better policy transfer performance:

RigidBodyAPI Parameter

Default Value in Isaac Sim

Default Value in Isaac Gym Preview Releases

Linear Damping

0.00

0.00

Angular Damping

0.05

0.0

Max Linear Velocity

inf

1000

Max Angular Velocity

5729.58008 (degree/s)

64.0 (rad/s)

Max Contact Impulse

inf

1e32

../_images/isaac_sim_rl_transfer_rigidbody_api.png

Parameters in the JointAPI as well as the DriveAPI could be altered as well. Note that the Isaac Sim UI assumes the unit of angle to be degrees. It is particularly worth noting that the Damping and Stiffness paramters in the DriveAPI have the unit of 1/deg in the Isaac Sim UI but 1/rad in Isaac Gym Preview Releases.

Joint Parameter

Default Value in Isaac Sim

Default Value in Isaac Gym Preview Releases

Maximum Joint Velocity

1000000.0 (deg)

100.0 (rad)

../_images/isaac_sim_rl_transfer_joint_api.png

5.6. Summary

This tutorial covered the following topics:

  1. Differences in quaternion convention between Isaac Gym and Isaac Sim

  2. Differences in joint ordering between Isaac Gym and Isaac Sim

  3. Differences in physics parameters of assets between Isaac Gym and Isaac Sim