Camera Control
Overview
Omni.Replicator.Agent (ORA) provides randomization for the cameras. By default, the cameras are spawned and point to the origin of the world system. There is also an option to spawn the cameras that looks at the characters in the scene. Additionally, ORA includes a camera calibration extension that provides the intrinsic and extrinsic information of the virtual fixed cameras in the ORA simulation.
Aim Camera Toward a Character
Introduction
This feature aims to automate the camera layout adjustment progress. Before clicking “Set up Simulation” button, adjust the camera’s heights, viewing angles, and the distances to characters via adjusting several carb.settings values.
Activate the Feature
To activate the feature, set
aim_camera_to_character
toTrue
.Configure the variables.
import carb
my_setting = carb.settings.get_settings()
target_value = True
aim_camera_to_character = "persistent/exts/omni.replicator.agent/aim_camera_to_character"
my_setting.set(aim_camera_to_character, target_value)
Attributes
Setting Attributes
In the camera calibration extension, camera attributes are values used to adjust the camera’s properties including height above ground, distance between character, view angle, and FOV. Carb setting values are used to store those values.
To edit those camera attributes using a UI slot:
Enable “kit debug settings extension” from the Window -> Extension. A DebugSetting panel appears.
Search omni.replicator.agent in the searching slot, then go to “persistent” category to see all the settings related to the camera randomization feature.
Attributes
Illustration for the attributes:
Max Camera Height
Usage: Camera’s max height value (translation’s z axis value) range
Default Value (meter):
3
Min Camera Height
Usage: Camera’s min height value (translation’s z axis value) range
Default Value (meter):
2
Camera Focus Height
Usage: This attribute determines where the camera looks. The camera’s focus point is: “Camera Focus Height” value above the character’s root position
Default Value (meter):
0.7
Camera Max Look Down Angle
The Max angle between camera and the ground
Default Value (degree):
60
Camera Min Look Down Angle
The Max angle between camera and the ground
Default Value (degree):
60
Camera Min Look Down Angle
The Min angle between camera and the ground
Default Value (degree):
0
Special Cases
What if Max Camera Height and Min Camera Height are equal?
The program places cameras at same height value.
What if Max Camera Look Down Angle and Min Camera Look Down Angle are equal?
The program places all camera with the same Camera Look Down Angle.
What if there is no plausible position to place the cameras?
The default camera generation function are triggered. All the cameras are focused on the map’s origin point of
(0, 0, 0)
.
What if we also need to place Lidar Cameras?
The Lidar Camera is automatically matched to the Camera with the same index. For example, when creating
Lidar_01
, it checks whether theCamera_01
is inside the scene. If it is, thenLidar_01
shares the same transform asCamera_01
.
Extra Config Values
max_camera_distance
and min_camera_distance
are used to set camera’s max and min distance toward the character.
Max Camera Distance’s default value (meter):
14
Min Camera Distance’s default value (meter):
6.5
Note
Max camera height must be lower than max camera distance.
Adjusting Random Camera Setting Variables
To adjust the config values, change the carb
setting variables in the following file.
The folllowing example script can be run in the Window-> Script Editor.
import carb # activate the feature by setting aim_camera_to_character to true my_setting = carb.settings.get_settings() target_value = True aim_camera_to_character = "persistent/exts/omni.replicator.agent/aim_camera_to_character" my_setting.set(aim_camera_to_character, target_value) # please set your target value in here target_max_camera_height = 3 target_min_camera_height = 2 target_character_focus_height = 0.7 target_max_camera_look_down_angle = 60 target_min_camera_look_down_angle = 0 target_max_camera_distance = 14 target_min_camera_distance = 6.5 # set min camera height to "target min camera height" min_camera_height = "/persistent/exts/omni.replicator.agent/min_camera_height" my_setting.set(min_camera_height , target_min_camera_height) setted_min_camera_height = my_setting.get(min_camera_height) carb.log_warn("This is the setted min height" + str(setted_min_camera_height)) # set max camera height to "target max camra height" max_camera_height = "/persistent/exts/omni.replicator.agent/max_camera_height" my_setting.set(max_camera_height , target_max_camera_height) setted_max_camera_height = my_setting.get(max_camera_height) carb.log_warn("This is the setted max height" + str(setted_max_camera_height)) # set camera focus height to "target camera focus height" character_focus_height = "/persistent/exts/omni.replicator.agent/character_focus_height" my_setting.set(character_focus_height, target_character_focus_height) setted_character_focus_height = my_setting.get(character_focus_height) carb.log_warn("This is the setted focus character height" + str(setted_character_focus_height)) # set max camera look down angle to " taget max camera look down angle" max_camera_look_down_angle = "/persistent/exts/omni.replicator.agent/max_camera_look_down_angle" my_setting.set(max_camera_look_down_angle, target_max_camera_look_down_angle) setted_max_camera_look_down_angle = my_setting.get(max_camera_look_down_angle) carb.log_warn("This is the setted max camera look down" + str(setted_max_camera_look_down_angle)) # set min camera look down angle to " target min camera look down angle" min_camera_look_down_angle = "/persistent/exts/omni.replicator.agent/min_camera_look_down_angle" my_setting.set(min_camera_look_down_angle, target_min_camera_look_down_angle) setted_min_camera_look_down_angle = my_setting.get(min_camera_look_down_angle) carb.log_warn("This is the setted min camera look down" + str(setted_min_camera_look_down_angle)) # set max camera distance to " target max camera distance" max_camera_distance = "persistent/exts/omni.replicator.agent/max_camera_distance" my_setting.set(max_camera_distance, target_max_camera_distance) setted_max_camera_distance = my_setting.get(max_camera_distance) carb.log_warn("This is the setted min camera distance" + str(setted_max_camera_distance)) # set min camera distance to " target min camera distance" min_camera_distance = "persistent/exts/omni.replicator.agent/min_camera_distance" my_setting.set(min_camera_distance, target_min_camera_distance) setted_min_camera_distance = my_setting.get(min_camera_distance) carb.log_warn("This is the setted min camera distance" + str(setted_min_camera_distance))
Camera Calibration
ORA includes an extension called Omni.Replicator.Agent.Camera_calibration. The camera calibration tool provides camera information for cameras generated by ORA. It can be used to select a top-view camera within the stage to create the stage layout. Details such as camera direction, location, and FOV polygon information are also be captured and saved to a JSON file. For each camera generated by ORA, the FOV polygon is visually represented in the stage layout.