Motion Policy Extension [omni.isaac.motion_generation]¶

Motion Policy Interface¶

class PolicyType(value)¶

An enumeration.

VELOCITY = 1¶

POSITION = 2¶

class MotionPolicy(_stage: pxr.Usd.Stage, policy_type: omni.isaac.motion_generation.motion_policy_interface.PolicyType)¶

Interface for implementing motion policies for compatibility with MotionGenerator interface.

set_initialized() → None¶

Set self.initialized to True after a successful initialization.

Returns: None

update(updated_obstacles: Optional[List[pxr.Usd.Prim]] = None) → None¶

Applies all necessary updates to the internal world/robot state.

Parameters: updated_obstacles (list, optional) – If provided, only the given obstacles will have their poses updated. For motion policies that use obstacle poses relative to the robot base (e.g. Lula based planners), this list will be ignored if the robot base has moved because all object poses will have changed relative to the robot. Defaults to None.
Returns: None

get_joint_velocity_targets(joint_positions: numpy.array, joint_velocities: numpy.array, frame_duration: float) → numpy.array¶

Compute new velocity targets based on robot state.

This function will be used by MotionGenerator to set a velocity target at every frame. This function only needs to be implemented if the MotionPolicy has the PolicyType VELOCITY_POLICY

Parameters

joint_positions (np.array) – (m x 1) vector with position of each active robot joint.
joint_velocities (np.array) – (m x 1) vector with velocity of each active robot joint.
frame_duration (float) – Duration of a single frame of simulation in seconds.

Returns

(m x 1) vector with velocity target for each active joint.

Return type

np.array

get_joint_position_targets(joint_positions: numpy.array, joint_velocities: numpy.array, frame_duration: float) → numpy.array¶

Compute new position targets based on robot state.

This function will be used by MotionGenerator to set a position target at every frame. This function only needs to be implemented if the MotionPolicy has the PolicyType POSITION_POLICY

Parameters

joint_positions (np.array) – (m x 1) vector with position of each active robot joint.
joint_velocities (np.array) – (m x 1) vector with velocity of each active robot joint.
frame_duration (float) – Duration of a single frame of simulation in seconds.

Returns

(m x 1) vector with position target for each active joint.

Return type

np.array

get_active_joints() → List[str]¶

Return names of active joints.

Some articulated robot joints may be ignored by some policies. E.g., the gripper of the Franka arm is not used to follow targets, and the RMPflow config files excludes the joints in the gripper from the list of articulated joints.

Returns: names of active joints.
Return type: list of str

set_cspace_target(target: numpy.array) → None¶

Set configuration space target for the robot.

Parameters: target (np.array) – Desired configuration for the robot as (m x 1) vector where m is the number of active joints.
Returns: None

set_end_effector_target(target_prim: pxr.Usd.Prim, position_only: bool = False) → None¶

Set end effector target.

Parameters

target_prim (Usd.Prim) – USD prim of the target. target_prim may also be None, in which case it is up to the policy to specify the desired behavior of the robot. Some policies store a default c-space configuration in their config files and drive the robot to that position when there is no target specified.
position_only (bool, optional) – When True, the policy will use only the position (not orientation) of the target_prim as the target. Defaults to False.

Returns

None

get_end_effector_pose(joint_positions: numpy.array) → Tuple[numpy.array, numpy.array, numpy.array]¶

Return pose of the end effector at the given end effector position.

Parameters: joint_positions (np.array) – (m x 1) vector of joint positions for which to compute forward kinematics.
Returns: End effector pose returned as translation vector (3 x 1) and rotation matrix (3 x 3).
Return type: Tuple[np.array, np.array]

get_prim_pose(prim: pxr.Usd.Prim, default_trans: numpy.array = array([0.0, 0.0, 0.0]), default_rot: numpy.array = array([[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]])) → bool¶

Return pose of prim.

USD prims that lack translational information are placed on the stage at the point (0,0,0). USD prims that lack rotational information are placed on the stage with the identity rotation. Reading translation/rotation information from prims directly yields a 4x4 transform matrix, which necessitates filling in missing information with default information. This method allows the caller to know what information is actually present in the USD prims by passing in None for the defaults.

Parameters

prim (Usd.Prim) – Prim for which pose will be returned.
default_trans (np.array, optional) – Translation component of pose to be returned if the prim has no translational information. Defaults to np.zeros(3).
default_rot (cp.array, optional) – Rotational component of pose to be returned if the prim contains no rotational information. Defaults to np.eye(3).

Returns

Prim pose returned as translation vector (3 x 1) and rotation matrix (3 x 3).

Return type

Tuple[np.array, np.array]

create_cube(block_prim: pxr.Usd.Prim, side_length: Optional[float] = None, static: bool = False) → bool¶

Create a cube obstacle.

Parameters

block_prim (Usd.Prim) – USD prim representing the cube. Must have pose information.
side_length (float, optional) – [description]. Length of each side of the cube. If not specified, side_length is read from ‘size’ attribute of block_prim. Defaults to None.
static (bool, optional) – If True, indicate that cube will never change pose, and may be ignored in internal world updates. Defaults to False.

Returns

Return True if block_prim exists and has required attributes (i.e., size must be specified somewhere, either in block_prim or via side_length param).

Return type

bool

create_block(block_prim: pxr.Usd.Prim, dimensions: Optional[numpy.array] = None, static: bool = False) → bool¶

Create a block obstacle.

Parameters

block_prim (Usd.Prim) – USD prim representing the block. Must have pose information.
dimensions (np.array, optional) – Length of block in (x,y,z) dimensions. If not specified, prim must have ‘xformOp:scale’ and “size” attribute. Defaults to None.
static (bool, optional) – If True, indicate that cube will never change pose, and may be ignored in internal world updates. Defaults to False.

Returns

Return True if block_prim exists and has required attributes (i.e., side lengths must be specified somewhere, either in block_prim or via dimensions param).

Return type

bool

create_sphere(sphere_prim: pxr.Usd.Prim, radius: Optional[float] = None, static: bool = False) → bool¶

Create a sphere obstacle.

Parameters

sphere_prim (Usd.Prim) – USD prim representing the sphere. Must have pose information.
radius (float, optional) – Radius of the sphere. If not specified, radius is read from ‘radius’ attribute of sphere_prim. Defaults to None.
static (bool, optional) – If True, indicate that cube will never change pose, and may be ignored in internal world updates. Defaults to False.

Returns

Return True if sphere_prim exists and has required attributes (i.e., radius must be specified somewhere, either in sphere_prim or via radius param).

Return type

bool

create_capsule(capsule_prim: pxr.Usd.Prim, radius: Optional[float] = None, height: Optional[float] = None, static: bool = False) → bool¶

Create a capsule obstacle.

Parameters

capsule_prim (Usd.Prim) – USD prim representing the capsule. Must have pose information.
radius (float, optional) – Radius of the capsule. If not specified, radius is read from ‘radius’ attribute of capsule_prim. Defaults to None.
height (float, optional) – Height of the capsule. If not specified, height is read from ‘height’ attribute of capsule_prim. Defaults to None.
static (bool, optional) – If True, indicate that cube will never change pose, and may be ignored in internal world updates. Defaults to False.

Returns

Return True if capsule_prim exists and has required attributes (i.e., radius and height must be specified somewhere, either in capsule_prim or via radius and height params).

Return type

bool

disable_obstacle(obstacle_prim: pxr.Usd.Prim) → bool¶

Disable collision avoidance for obstacle.

Parameters: obstacle_prim (Usd.Prim) – USD prim for obstacle to be disabled.
Returns: Return true if obstacle was identified and successfully disabled.
Return type: bool

enable_obstacle(obstacle_prim: pxr.Usd.Prim) → bool¶

Enable collision avoidance for obstacle.

Parameters: obstacle_prim (Usd.Prim) – USD prim for obstacle to be enabled.
Returns: Return true if obstacle was identified and successfully enabled.
Return type: bool

remove_obstacle(obstacle_prim: pxr.Usd.Prim) → bool¶

Remove obstacle from collision avoidance. Obstacle cannot be re-enabled via enable_obstacle() after removal.

Parameters: obstacle_prim (Usd.Prim) – USD prim for obstacle to be removed.
Returns: Return true if obstacle was identified and successfully removed.
Return type: bool

Motion Generator¶

class MotionGenerator(stage: pxr.Usd.Stage)¶

Interface for running MotionPolicy on simulated robots.

initialize(policy_config: dict, robot_prim: pxr.Usd.Prim, sim_fps: float, velocity_control_damping: float = 100000000.0) → None¶

Initialize MotionGenerator.

Parameters

policy_config (dict) – Dictionary containing the necessary configurations for a motion policy. policy_config must specify “policy_type” to select a specific motion policy such as “RMPflow”. All other specifications are policy dependent.
robot_prim (Usd.Prim) – USD prim for robot.
sim_fps (float) – Frequency at which Isaac Sim updates the world, typically 60 Hz
velocity_control_damping (float, optional) – Damping gain if using velocity control. Defaults to 1e8.

Returns

None

is_initialized() → bool¶

Indicates whether MotionGenerator has been initialized.

Returns: Return True if MotionGenerator has been initialized. Else return False.
Return type: bool

move(updated_obstacles: Optional[List[pxr.Usd.Prim]] = None) → None¶

Generate new joint targets using MotionPolicy and apply joint targets to the robot.

Parameters: updated_obstacles (list, optional) – List of obstacles that need to have their poses updated. If provided, only these obstacles will have their poses updated. Defaults to None.
Returns: None

get_joint_states() → Tuple[numpy.array, numpy.array, numpy.array]¶

Return the states (position, velocity and acceleration) of all robot joints.

Returns: States (position, velcocity and acceleration) for all robot joints.
Return type: Tuple[np.array, np.array, np.array]

get_active_joint_states() → Tuple[numpy.array, numpy.array, numpy.array]¶

Return the states (position, velocity and acceleration) of active robot joints. Active joints are joints controlled by the underlying MotionPolicy.

Returns: States (position, velcocity and acceleration) for active robot joints.
Return type: Tuple[np.array, np.array, np.array]

get_end_effector_pose() → Tuple[numpy.array, numpy.array, numpy.array]¶

Return current pose of the end effector.

Returns: End effector pose returned as translation vector (3 x 1) and rotation matrix (3 x 3).
Return type: Tuple[np.array, np.array]

set_cspace_target(target: numpy.array) → None¶

Set configuration space target for the robot.

Parameters: target (np.array) – Desired configuration for the robot as (m x 1) vector where m is the number of active joints.
Returns: None

set_end_effector_target(target_prim: pxr.Usd.Prim, position_only: bool = False) → None¶

Set end effector target.

Parameters

target_prim (Usd.Prim) – USD prim of the target. target_prim may also be None, in which case it is up to the policy to specify the desired behavior of the robot. Some policies store a default c-space configuration in their config files and drive the robot to that position when there is no target specified.
position_only (bool, optional) – When True, the policy will use only the position (not orientation) of the target_prim as the target. Defaults to False.

Returns

None

create_cube(block_prim: pxr.Usd.Prim, side_length: Optional[float] = None, static: bool = False) → bool¶

Create a cube obstacle.

Parameters

block_prim (Usd.Prim) – USD prim representing the cube. Must have pose information.
side_length (float, optional) – [description]. Length of each side of the cube. If not specified, side_length is read from ‘size’ attribute of block_prim. Defaults to None.
static (bool, optional) – If True, indicate that cube will never change pose, and may be ignored in internal world updates. Defaults to False.

Returns

Return True if block_prim exists and has required attributes (i.e., size must be specified somewhere, either in block_prim or via side_length param).

Return type

bool

create_block(block_prim: pxr.Usd.Prim, dimensions: Optional[numpy.array] = None, static: bool = False) → bool¶

Create a block obstacle.

Parameters

block_prim (Usd.Prim) – USD prim representing the block. Must have pose information.
dimensions (np.array, optional) – Length of block in (x,y,z) dimensions. If not specified, prim must have ‘xformOp:scale’ and “size” attribute. Defaults to None.
static (bool, optional) – If True, indicate that cube will never change pose, and may be ignored in internal world updates. Defaults to False.

Returns

Return True if block_prim exists and has required attributes (i.e., side lengths must be specified somewhere, either in block_prim or via dimensions param).

Return type

bool

create_sphere(sphere_prim: pxr.Usd.Prim, radius: Optional[float] = None, static: bool = False) → bool¶

Create a sphere obstacle.

Parameters

sphere_prim (Usd.Prim) – USD prim representing the sphere. Must have pose information.
radius (float, optional) – Radius of the sphere. If not specified, radius is read from ‘radius’ attribute of sphere_prim. Defaults to None.
static (bool, optional) – If True, indicate that cube will never change pose, and may be ignored in internal world updates. Defaults to False.

Returns

Return True if sphere_prim exists and has required attributes (i.e., radius must be specified somewhere, either in sphere_prim or via radius param).

Return type

bool

create_capsule(capsule_prim: pxr.Usd.Prim, radius: Optional[float] = None, height: Optional[float] = None, static: bool = False) → bool¶

Create a capsule obstacle.

Parameters

capsule_prim (Usd.Prim) – USD prim representing the capsule. Must have pose information.
radius (float, optional) – Radius of the capsule. If not specified, radius is read from ‘radius’ attribute of capsule_prim. Defaults to None.
height (float, optional) – Height of the capsule. If not specified, height is read from ‘height’ attribute of capsule_prim. Defaults to None.
static (bool, optional) – If True, indicate that cube will never change pose, and may be ignored in internal world updates. Defaults to False.

Returns

Return True if capsule_prim exists and has required attributes (i.e., radius and height must be specified somewhere, either in capsule_prim or via radius and height params).

Return type

bool

get_prim_pose(prim: pxr.Usd.Prim, default_trans: numpy.array = array([0.0, 0.0, 0.0]), default_rot: numpy.array = array([[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]])) → bool¶

Return pose of prim.

USD prims that lack translational information are placed on the stage at the point (0,0,0). USD prims that lack rotational information are placed on the stage with the identity rotation. Reading translation/rotation information from prims directly yields a 4x4 transform matrix, which necessitates filling in missing information with default information. This method allows the caller to know what information is actually present in the USD prims by passing in None for the defaults.

Parameters

prim (Usd.Prim) – Prim for which pose will be returned.
default_trans (np.array, optional) – Translation component of pose to be returned if the prim has no translational information. Defaults to np.zeros(3).
default_rot (cp.array, optional) – Rotational component of pose to be returned if the prim contains no rotational information. Defaults to np.eye(3).

Returns

Prim pose returned as translation vector (3 x 1) and rotation matrix (3 x 3).

Return type

Tuple[np.array, np.array]

disable_obstacle(obstacle_prim: pxr.Usd.Prim) → bool¶

Disable collision avoidance for obstacle.

Parameters: obstacle_prim (Usd.Prim) – USD prim for obstacle to be disabled.
Returns: Return true if obstacle was identified and successfully disabled.
Return type: bool

enable_obstacle(obstacle_prim: pxr.Usd.Prim) → bool¶

Enable collision avoidance for obstacle.

Parameters: obstacle_prim (Usd.Prim) – USD prim for obstacle to be enabled.
Returns: Return true if obstacle was identified and successfully enabled.
Return type: bool

remove_obstacle(obstacle_prim: pxr.Usd.Prim) → bool¶

Remove obstacle from collision avoidance. Obstacle cannot be re-enabled via enable_obstacle() after removal.

Parameters: obstacle_prim (Usd.Prim) – USD prim for obstacle to be removed.
Returns: Return true if obstacle was identified and successfully removed.
Return type: bool

get_joint_velocity_targets() → numpy.array¶

Compute and return joint velocity targets using underlying MotionPolicy.

Returns: (m x 1) vector of velocity targets, where m is the number of active joints.
Return type: np.array

get_joint_position_targets() → numpy.array¶

Compute and return joint position targets using underlying MotionPolicy.

Active joint targets are computed by MotionPolicy. Position targets for non-active joints are kept identical to those currently set in the articulation.

Returns: (m x 1) vector of position targets, where m is the total number of joints.
Return type: np.array

get_motion_policy() → omni.isaac.motion_generation.motion_policy_interface.MotionPolicy¶

It can be convenient to interact directly with the low level motion policy for testing and development, but in general the policy should be designed to function using only the motion_policy_interface.

Returns: Underlying motion policy used by MotionGenerator.
Return type: MotionPolicy

RMPFlow Base Controller¶

class RMPFlowController(name: str, robot_prim_path: str, policy_map_path: List[str], physics_dt: float = 0.016666666666666666)¶

[summary]

Parameters

name (str) – [description]
robot_prim_path (str) – [description]
policy_map_path (List[str]) – [description]
physics_dt (float, optional) – [description]. Defaults to 1.0/60.0.

add_cube_obstacle(cube_prim: pxr.Usd.Prim) → None¶

[summary]

Parameters: cube_prim (Usd.Prim) – [description]

forward(target_end_effector_position: numpy.ndarray, target_end_effector_orientation: Optional[numpy.ndarray] = None) → omni.isaac.core.utils.types.ArticulationAction ¶

[summary]

Parameters

target_end_effector_position (np.ndarray) – [description]
target_end_effector_orientation (Optional[np.ndarray], optional) – [description]. Defaults to None.

Returns

[description]

Return type

ArticulationAction

get_motion_generation() → omni.isaac.motion_generation.motion_generation.MotionGenerator¶

[summary]

Returns: [description]
Return type: MotionGenerator

remove_cube_obstacle(cube_prim: pxr.Usd.Prim) → None¶

[summary]

Parameters: cube_prim (Usd.Prim) – [description]

reset() → None¶: [summary]

Pick Place Controller¶

class PickPlaceController(name, cspace_controller: omni.isaac.core.controllers.base_controller.BaseController, gripper_controller: omni.isaac.core.controllers.base_gripper_controller.BaseGripperController, start_picking_height: Optional[float] = None, event_velocities: Optional[List[float]] = None)¶

Phase 0: Move end_effector above the cube center.
Phase 1: Lower end_effector down to encircle the target cube
Phase 2: close grip.
Phase 3: Move end_effector up again, keeping the grip tight (lifting the block).
Phase 4: Smoothly move the end_effector toward the goal xy, keeping the height constant.
Phase 5: Move end_effector vertically toward goal height.
Phase 6: loosen the grip.
Phase 7: Move end_effector vertically up again
Phase 8: Move end_effector towards the old xy position.

Parameters

name ([type]) – [description]
cspace_controller (BaseController) – [description]
gripper_controller (GripperController) – [description]
start_picking_height (typing.Optional[float], optional) – [description]. Defaults to None.
event_velocities (typing.Optional[typing.List[float]], optional) – [description]. Defaults to None.

Raises

Exception – [description]
Exception – [description]

forward(picking_position: numpy.ndarray, placing_position: numpy.ndarray, current_joint_positions: numpy.ndarray, end_effector_offset: Optional[numpy.ndarray] = None, end_effector_orientation: Optional[numpy.ndarray] = None) → omni.isaac.core.utils.types.ArticulationAction ¶

[summary]

Parameters

picking_position (np.ndarray) – [description]
placing_position (np.ndarray) – [description]
current_joint_positions (np.ndarray) – [description]
end_effector_offset (typing.Optional[np.ndarray], optional) – [description]. Defaults to None.
end_effector_orientation (typing.Optional[np.ndarray], optional) – [description]. Defaults to None.

Returns

[description]

Return type

ArticulationAction

get_current_event() → int¶

[summary]

Returns: [description]
Return type: int

is_done() → bool¶

[summary]

Returns: [description]
Return type: bool

is_paused() → bool¶

[summary]

Returns: [description]
Return type: bool

pause() → None¶: [summary]

reset(start_picking_height: Optional[float] = None, event_velocities: Optional[List[float]] = None) → None¶

[summary]

Parameters

start_picking_height (typing.Optional[float], optional) – [description]. Defaults to None.
event_velocities (typing.Optional[typing.List[float]], optional) – [description]. Defaults to None.

Raises

Exception – [description]
Exception – [description]

resume() → None¶: [summary]

Stacking Controller¶

class StackingController(name: str, pick_place_controller: omni.isaac.motion_generation.pick_place_controller.PickPlaceController, picking_order_cube_names: List[str], robot_observation_name: str)¶

[summary]

Parameters

name (str) – [description]
pick_place_controller (PickPlaceController) – [description]
picking_order_cube_names (typing.List[str]) – [description]
robot_observation_name (str) – [description]

forward(observations: dict, end_effector_orientation: Optional[numpy.ndarray] = None, end_effector_offset: Optional[numpy.ndarray] = None) → omni.isaac.core.utils.types.ArticulationAction ¶

A controller should take inputs and returns an ArticulationAction to be then passed to the: ArticulationController.

Parameters: observations (dict) – [description]
Raises: NotImplementedError – [description]
Returns: [description]
Return type: ArticulationAction

is_done() → bool¶

[summary]

Returns: [description]
Return type: bool

reset(picking_order_cube_names: Optional[List[str]] = None) → None¶

[summary]

Parameters: picking_order_cube_names (typing.Optional[typing.List[str]], optional) – [description]. Defaults to None.

Wheel Base Pose Controller¶

class WheelBasePoseController(name: str, open_loop_wheel_controller: omni.isaac.core.controllers.base_controller.BaseController, is_holonomic: bool = False)¶

[summary]

Parameters

name (str) – [description]
open_loop_wheel_controller (BaseController) – A controller that takes in a command of [longitudinal velocity, steering angle] and returns the ArticulationAction to be applied to the wheels if non holonomic. and [longitudinal velocity, latitude velocity, steering angle] if holonomic.
is_holonomic (bool, optional) – [description]. Defaults to False.

forward(start_position: numpy.ndarray, start_orientation: numpy.ndarray, goal_position: numpy.ndarray, lateral_velocity: float = 20.0, yaw_velocity: float = 0.5, heading_tol: float = 0.05, position_tol: float = 4.0) → omni.isaac.core.utils.types.ArticulationAction ¶

[summary]

Parameters

start_position (np.ndarray) – [description]
start_orientation (np.ndarray) – [description]
goal_position (np.ndarray) – [description]
lateral_velocity (float, optional) – [description]. Defaults to 20.0.
yaw_velocity (float, optional) – [description]. Defaults to 0.5.
heading_tol (float, optional) – [description]. Defaults to 0.05.
position_tol (float, optional) – [description]. Defaults to 4.0.

Returns

[description]

Return type

ArticulationAction

reset() → None¶: [summary]