RL [omni.isaac.gym]

Base Environment Wrapper

class SimulationApp(launch_config: Optional[dict] = None, experience: str = '')

Helper class to launch Omniverse Toolkit.

Omniverse loads various plugins at runtime which cannot be imported unless the Toolkit is already running. Thus, it is necessary to launch the Toolkit first from your python application and then import everything else.

Usage:

# At top of your application
from omni.isaac.kit import SimulationApp
config = {
     width: "1280",
     height: "720",
     headless: False,
}
simulation_app = SimulationApp(config)

# Rest of the code follows
...
simulation_app.close()

Note

The settings in DEFAULT_LAUNCHER_CONFIG are overwritten by those in config.

Parameters

  • config (dict) – A dictionary containing the configuration for the app. (default: None)

  • experience (str) – Path to the application config loaded by the launcher (default: “”, will load app/omni.isaac.sim.python.kit if left blank)

DEFAULT_LAUNCHER_CONFIG = {'active_gpu': None, 'anti_aliasing': 3, 'denoiser': True, 'display_options': 3094, 'headless': True, 'height': 720, 'livesync_usd': None, 'max_bounces': 4, 'max_specular_transmission_bounces': 6, 'max_volume_bounces': 4, 'memory_report': False, 'multi_gpu': True, 'open_usd': None, 'renderer': 'RayTracedLighting', 'samples_per_pixel_per_frame': 64, 'subdiv_refinement_level': 0, 'sync_loads': True, 'width': 1280, 'window_height': 900, 'window_width': 1440}

The config variable is a dictionary containing the following entries

Parameters

  • headless (bool) – Disable UI when running. Defaults to True

  • active_gpu (int) – Specify the GPU to use when running, set to None to use default value which is usually the first gpu, default is None

  • multi_gpu (bool) – Set to true to enable Multi GPU support, Defaults to true

  • sync_loads (bool) – When enabled, will pause rendering until all assets are loaded. Defaults to True

  • width (int) – Width of the viewport and generated images. Defaults to 1280

  • height (int) – Height of the viewport and generated images. Defaults to 720

  • window_width (int) – Width of the application window, independent of viewport, defaults to 1440,

  • window_height (int) – Height of the application window, independent of viewport, defaults to 900,

  • display_options (int) – used to specify whats visible in the stage by default. Defaults to 3094 so extra objects do not appear in synthetic data. 3286 is another good default, used for the regular isaac-sim editor experience

  • subdiv_refinement_level (int) – Number of subdivisons to perform on supported geometry. Defaults to 0

  • renderer (str) – Rendering mode, can be RayTracedLighting or PathTracing. Defaults to PathTracing

  • anti_aliasing (int) – Antialiasing mode, 0: Disabled, 1: TAA, 2: FXAA, 3: DLSS, 4:RTXAA

  • samples_per_pixel_per_frame (int) – The number of samples to render per frame, increase for improved quality, used for PathTracing only. Defaults to 64

  • denoiser (bool) – Enable this to use AI denoising to improve image quality, used for PathTracing only. Defaults to True

  • max_bounces (int) – Maximum number of bounces, used for PathTracing only. Defaults to 4

  • max_specular_transmission_bounces (int) – Maximum number of bounces for specular or transmission, used for PathTracing only. Defaults to 6

  • max_volume_bounces (int) – Maximum number of bounces for volumetric materials, used for PathTracing only. Defaults to 4

  • open_usd (str) – This is the name of the usd to open when the app starts. It will not be saved over. Default is None and an empty stage is created on startup.

  • livesync_usd (str) – This is the location of the usd that you want to do your interactive work in. The existing file is overwritten. Default is None

  • memory_report (bool) – Set to true to print a memory usage report on exit. Default is False

property app

omniverse kit application object

Type

omni.kit.app.IApp

close() → None

Close the running Omniverse Toolkit.

property context

the current USD context

Type

omni.usd.UsdContext

is_exiting() → bool

bool: True if close() was called previously, False otherwise

is_running() → bool

bool: convenience function to see if app is running. True if running, False otherwise

reset_render_settings()

Reset render settings to those in config.

Note

This should be used in case a new stage is opened and the desired config needs to be re-applied.

set_setting(setting: str, value) → None

Set a carbonite setting

Parameters

  • setting (str) – carb setting path

  • value – value to set the setting to, type is used to properly set the setting.

update() → None

Convenience function to step the application forward one frame

class VecEnvBase(headless: bool)

This class provides a base interface for connecting RL policies with task implementations. APIs provided in this interface follow the interface in gym.Env. This class also provides utilities for initializing simulation apps, creating the World, and registering a task.

action_space = None
close() → None

Closes simulation.

metadata = {'render.modes': []}
property num_envs

Retrieves number of environments.

Returns

Number of environments.

Return type

num_envs(int)

observation_space = None
render(mode='human') → None

Step the renderer.

Parameters

mode (str) – Select mode of rendering based on OpenAI environments.

reset()

Resets the task and updates observations.

reward_range = (-inf, inf)
seed(seed=- 1)

Sets a seed. Pass in -1 for a random seed.

Parameters

seed (int) – Seed to set. Defaults to -1.

Returns

Seed that was set.

Return type

seed (int)

set_task(task, backend='numpy', sim_params=None, init_sim=True) → None
Creates a World object and adds Task to World.

Initializes and registers task to the environment interface. Triggers task start-up.

Parameters

  • task (RLTask) – The task to register to the env.

  • backend (str) – Backend to use for task. Can be “numpy” or “torch”. Defaults to “numpy”.

  • sim_params (dict) – Simulation parameters for physics settings. Defaults to None.

  • init_sim (Optional[bool]) – Automatically starts simulation. Defaults to True.

spec = None
step(actions)
Basic implementation for stepping simulation.

Can be overriden by inherited Env classes to satisfy requirements of specific RL libraries. This method passes actions to task for processing, steps simulation, and computes observations, rewards, and resets.

Parameters

actions (Union[numpy.ndarray, torch.Tensor]) – Actions buffer from policy.

Returns

Buffer of observation data. rewards(Union[numpy.ndarray, torch.Tensor]): Buffer of rewards data. dones(Union[numpy.ndarray, torch.Tensor]): Buffer of resets/dones data. info(dict): Dictionary of extras data.

Return type

observations(Union[numpy.ndarray, torch.Tensor])

property unwrapped

Completely unwrap this env.

Returns

The base non-wrapped gym.Env instance

Return type

gym.Env

abstractmethod(funcobj)

A decorator indicating abstract methods.

Requires that the metaclass is ABCMeta or derived from it. A class that has a metaclass derived from ABCMeta cannot be instantiated unless all of its abstract methods are overridden. The abstract methods can be called using any of the normal ‘super’ call mechanisms.

Usage:

class C(metaclass=ABCMeta):

@abstractmethod def my_abstract_method(self, …):

…

Multi-Threaded Environment Wrapper

exception TaskStopException

Exception class for signalling task termination.

args
with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

class TrainerMT

A base abstract trainer class for controlling starting and stopping of RL policy.

abstract run()

Runs RL loop in a new thread

abstract stop()

Stop RL thread

class VecEnvBase(headless: bool)

This class provides a base interface for connecting RL policies with task implementations. APIs provided in this interface follow the interface in gym.Env. This class also provides utilities for initializing simulation apps, creating the World, and registering a task.

action_space = None
close() → None

Closes simulation.

metadata = {'render.modes': []}
property num_envs

Retrieves number of environments.

Returns

Number of environments.

Return type

num_envs(int)

observation_space = None
render(mode='human') → None

Step the renderer.

Parameters

mode (str) – Select mode of rendering based on OpenAI environments.

reset()

Resets the task and updates observations.

reward_range = (-inf, inf)
seed(seed=- 1)

Sets a seed. Pass in -1 for a random seed.

Parameters

seed (int) – Seed to set. Defaults to -1.

Returns

Seed that was set.

Return type

seed (int)

set_task(task, backend='numpy', sim_params=None, init_sim=True) → None
Creates a World object and adds Task to World.

Initializes and registers task to the environment interface. Triggers task start-up.

Parameters

  • task (RLTask) – The task to register to the env.

  • backend (str) – Backend to use for task. Can be “numpy” or “torch”. Defaults to “numpy”.

  • sim_params (dict) – Simulation parameters for physics settings. Defaults to None.

  • init_sim (Optional[bool]) – Automatically starts simulation. Defaults to True.

spec = None
step(actions)
Basic implementation for stepping simulation.

Can be overriden by inherited Env classes to satisfy requirements of specific RL libraries. This method passes actions to task for processing, steps simulation, and computes observations, rewards, and resets.

Parameters

actions (Union[numpy.ndarray, torch.Tensor]) – Actions buffer from policy.

Returns

Buffer of observation data. rewards(Union[numpy.ndarray, torch.Tensor]): Buffer of rewards data. dones(Union[numpy.ndarray, torch.Tensor]): Buffer of resets/dones data. info(dict): Dictionary of extras data.

Return type

observations(Union[numpy.ndarray, torch.Tensor])

property unwrapped

Completely unwrap this env.

Returns

The base non-wrapped gym.Env instance

Return type

gym.Env

class VecEnvMT(headless: bool)

This class provides a base interface for connecting RL policies with task implementations in a multi-threaded fashion. RL policies using this class will run on a different thread than the thread simulation runs on. This can be useful for interacting with the UI before, during, and after running RL policies. Data sharing between threads happen through message passing on multi-threaded queues.

action_space = None
clear_queues()

Clears all queues.

close() → None

Closes simulation.

get_actions(block=True)

Retrieves actions from policy by waiting for actions to be sent to the queue from the RL thread.

Parameters

block (Optional[bool]) – Whether to block thread when waiting for data.

Returns

actions buffer retrieved from queue.

Return type

actions (Union[np.ndarray, torch.Tensor, None])

get_data(block=True)

Retrieves data from task by waiting for data dictionary to be sent to the queue from the simulation thread.

Parameters

block (Optional[bool]) – Whether to block thread when waiting for data.

Returns

data dictionary retrieved from queue.

Return type

actions (Union[np.ndarray, torch.Tensor, None])

initialize(action_queue, data_queue, timeout=30)

Initializes queues for sharing data across threads.

Parameters

  • action_queue (queue.Queue) – Queue for passing actions from policy to task.

  • data_queue (queue.Queue) – Queue for passing data from task to policy.

  • timeout (Optional[int]) – Seconds to wait for data when queue is empty. An exception will be thrown when the timeout limit is reached. Defaults to 30 seconds.

metadata = {'render.modes': []}
property num_envs

Retrieves number of environments.

Returns

Number of environments.

Return type

num_envs(int)

observation_space = None
render(mode='human') → None

Step the renderer.

Parameters

mode (str) – Select mode of rendering based on OpenAI environments.

reset()

Resets the task and updates observations.

reward_range = (-inf, inf)
run(trainer)

Main loop for controlling simulation and task stepping. This method is responsible for starting simulation, stepping task and simulation, collecting buffers from task, sending data to policy, and retrieving actions from policy. It also deals with the case when the policy terminates on completion and continues the simulation thread so that UI does not get affected.

Parameters

trainer (TrainerMT) – A Trainer object that implements APIs for starting and stopping RL thread.

seed(seed=- 1)

Sets a seed. Pass in -1 for a random seed.

Parameters

seed (int) – Seed to set. Defaults to -1.

Returns

Seed that was set.

Return type

seed (int)

send_actions(actions, block=True)

Sends actions from RL thread to simulation thread by adding actions to queue.

Parameters

  • actions (Union[np.ndarray, torch.Tensor]) – actions buffer to be added to queue.

  • block (Optional[bool]) – Whether to block thread when writing to queue.

send_data(data, block=True)

Sends data from task thread to RL thread by adding data to queue.

Parameters

  • data (dict) – Dictionary containing task data.

  • block (Optional[bool]) – Whether to block thread when writing to queue.

set_task(task, backend='numpy', sim_params=None, init_sim=True) → None
Creates a World object and adds Task to World.

Initializes and registers task to the environment interface. Triggers task start-up.

Parameters

  • task (RLTask) – The task to register to the env.

  • backend (str) – Backend to use for task. Can be “numpy” or “torch”. Defaults to “numpy”.

  • sim_params (dict) – Simulation parameters for physics settings. Defaults to None.

  • init_sim (Optional[bool]) – Automatically starts simulation. Defaults to True.

spec = None
step(actions)
Basic implementation for stepping simulation.

Can be overriden by inherited Env classes to satisfy requirements of specific RL libraries. This method passes actions to task for processing, steps simulation, and computes observations, rewards, and resets.

Parameters

actions (Union[numpy.ndarray, torch.Tensor]) – Actions buffer from policy.

Returns

Buffer of observation data. rewards(Union[numpy.ndarray, torch.Tensor]): Buffer of rewards data. dones(Union[numpy.ndarray, torch.Tensor]): Buffer of resets/dones data. info(dict): Dictionary of extras data.

Return type

observations(Union[numpy.ndarray, torch.Tensor])

property unwrapped

Completely unwrap this env.

Returns

The base non-wrapped gym.Env instance

Return type

gym.Env