Camera Control

Overview

Omni.Replicator.Agent (ORA) provides randomization for the cameras. By default, the cameras are spawned and point to the origin of the world system. There is also an option to spawn the cameras that looks at the characters in the scene. Additionally, ORA includes a camera calibration extension that provides the intrinsic and extrinsic information of the virtual fixed cameras in the ORA simulation.

Aim Camera Toward a Character

Introduction

This feature aims to automate the camera layout adjustment progress. Before clicking “Set up Simulation” button, adjust the camera’s heights, viewing angles, and the distances to characters via adjusting several carb.settings values.

Activate the Feature

  1. To activate the feature, set aim_camera_to_character to True.

  2. Configure the variables.

import carb
my_setting = carb.settings.get_settings()
target_value = True
aim_camera_to_character = "persistent/exts/omni.replicator.agent/aim_camera_to_character"
my_setting.set(aim_camera_to_character, target_value)

Attributes

Setting Attributes

In the camera calibration extension, camera attributes are values used to adjust the camera’s properties including height above ground, distance between character, view angle, and FOV. Carb setting values are used to store those values.

To edit those camera attributes using a UI slot:

  1. Enable “kit debug settings extension” from the Window -> Extension. A DebugSetting panel appears.

  2. Search omni.replicator.agent in the searching slot, then go to “persistent” category to see all the settings related to the camera randomization feature.

an illustration for where to find the setting

Attributes

Illustration for the attributes:

an illustration for what each attribute represent for the camera settings

  • Max Camera Height

    • Usage: Camera’s max height value (translation’s z axis value) range

    • Default Value (meter): 3

  • Min Camera Height

    • Usage: Camera’s min height value (translation’s z axis value) range

    • Default Value (meter): 2

  • Camera Focus Height

    • Usage: This attribute determines where the camera looks. The camera’s focus point is: “Camera Focus Height” value above the character’s root position

    • Default Value (meter): 0.7

  • Camera Max Look Down Angle

    • The Max angle between camera and the ground

    • Default Value (degree): 60

  • Camera Min Look Down Angle

    • The Max angle between camera and the ground

    • Default Value (degree): 60

  • Camera Min Look Down Angle

    • The Min angle between camera and the ground

    • Default Value (degree): 0

Note

Camera Height must be higher than the character focus height.

an illustration for how to set the camera

Special Cases

  • What if Max Camera Height and Min Camera Height are equal?

    • The program places cameras at same height value.

  • What if Max Camera Look Down Angle and Min Camera Look Down Angle are equal?

    • The program places all camera with the same Camera Look Down Angle.

  • What if there is no plausible position to place the cameras?

    • The default camera generation function are triggered. All the cameras are focused on the map’s origin point of (0, 0, 0).

  • What if we also need to place Lidar Cameras?

    • The Lidar Camera is automatically matched to the Camera with the same index. For example, when creating Lidar_01, it checks whether the Camera_01 is inside the scene. If it is, then Lidar_01 shares the same transform as Camera_01.

Extra Config Values

max_camera_distance and min_camera_distance are used to set camera’s max and min distance toward the character.

  • Max Camera Distance’s default value (meter): 14

  • Min Camera Distance’s default value (meter): 6.5

an illustration for the extra camera config values

Note

Max camera height must be lower than max camera distance.

Adjusting Random Camera Setting Variables

To adjust the config values, change the carb setting variables in the following file.

The folllowing example script can be run in the Window-> Script Editor.

import carb

# activate the feature by setting aim_camera_to_character to true
my_setting = carb.settings.get_settings()
target_value = True
aim_camera_to_character = "persistent/exts/omni.replicator.agent/aim_camera_to_character"
my_setting.set(aim_camera_to_character, target_value)

# please set your target value in here
target_max_camera_height = 3
target_min_camera_height = 2
target_character_focus_height = 0.7
target_max_camera_look_down_angle = 60
target_min_camera_look_down_angle = 0
target_max_camera_distance = 14
target_min_camera_distance = 6.5


# set min camera height to "target min camera height"
min_camera_height = "/persistent/exts/omni.replicator.agent/min_camera_height"
my_setting.set(min_camera_height , target_min_camera_height)
setted_min_camera_height = my_setting.get(min_camera_height)
carb.log_warn("This is the setted min height" + str(setted_min_camera_height))


# set max camera height to "target max camra height"
max_camera_height = "/persistent/exts/omni.replicator.agent/max_camera_height"
my_setting.set(max_camera_height , target_max_camera_height)
setted_max_camera_height = my_setting.get(max_camera_height)
carb.log_warn("This is the setted max height" + str(setted_max_camera_height))

# set camera focus height to "target camera focus height"
character_focus_height = "/persistent/exts/omni.replicator.agent/character_focus_height"
my_setting.set(character_focus_height, target_character_focus_height)
setted_character_focus_height = my_setting.get(character_focus_height)
carb.log_warn("This is the setted focus character height" + str(setted_character_focus_height))

# set max camera look down angle to " taget max camera look down angle"
max_camera_look_down_angle = "/persistent/exts/omni.replicator.agent/max_camera_look_down_angle"
my_setting.set(max_camera_look_down_angle, target_max_camera_look_down_angle)
setted_max_camera_look_down_angle = my_setting.get(max_camera_look_down_angle)
carb.log_warn("This is the setted max camera look down" + str(setted_max_camera_look_down_angle))

# set min camera look down angle to " target min camera look down angle"
min_camera_look_down_angle = "/persistent/exts/omni.replicator.agent/min_camera_look_down_angle"
my_setting.set(min_camera_look_down_angle, target_min_camera_look_down_angle)
setted_min_camera_look_down_angle = my_setting.get(min_camera_look_down_angle)
carb.log_warn("This is the setted min camera look down" + str(setted_min_camera_look_down_angle))


# set max camera distance to " target max camera distance"
max_camera_distance = "persistent/exts/omni.replicator.agent/max_camera_distance"
my_setting.set(max_camera_distance, target_max_camera_distance)
setted_max_camera_distance = my_setting.get(max_camera_distance)
carb.log_warn("This is the setted min camera distance" + str(setted_max_camera_distance))

# set min camera distance to " target min camera distance"
min_camera_distance = "persistent/exts/omni.replicator.agent/min_camera_distance"
my_setting.set(min_camera_distance, target_min_camera_distance)
setted_min_camera_distance = my_setting.get(min_camera_distance)
carb.log_warn("This is the setted min camera distance" + str(setted_min_camera_distance))

Camera Calibration

ORA includes an extension called Omni.Replicator.Agent.Camera_calibration. The camera calibration tool provides camera information for cameras generated by ORA. It can be used to select a top-view camera within the stage to create the stage layout. Details such as camera direction, location, and FOV polygon information are also be captured and saved to a JSON file. For each camera generated by ORA, the FOV polygon is visually represented in the stage layout.