3. Omni.Anim.People

Warning

Omni.Anim.People is in Beta. For more information on pre-release and beta terms, refer to the Omniverse License Agreement.

Omni.Anim.People is an Extension for simulating human events in environments such as retail stores, warehouses, and, traffic intersections.

A simulation consists of a sequence of actions for each character in the scene to perform. There are six supported character actions:

• Idle

• Look around

• Queue

• Sit

• Stand

• Walk

The Extension consists of three components:

• A user interface for configuring simulation settings

• Behavior scripts that control the characters in a simulation

• Pre-configured character assets and animations

3.1. Enable Omni.Anim.People

First, enable omni.kit.window.viewport:

1. Select Window > Extensions to open the Extension Manager.

2. Search for “omni.kit.window.viewport”.

3. Select and enable the Extension.

Then, enable omni.anim.people:

1. In the Extension Manager, search for “omni.anim.people”.

2. Select and enable the Extension.

Finally, load the Extension UI in Window > Behavior Control Panel.

You use this in a future step.

Note

We recommend manually performing these steps when starting the app instead of setting the Extensions on autoload.

3.2. Get Started

3.2.1. Use a Demo Simulation

We’ve provided some working examples to help you get started.

First, enable the Navigation Mesh Extension:

1. In the Extension Manager, search for “omni.anim.navigation”.

2. Select and enable the Navigation Bundle Extension.

Next, open a demo:

1. In the Content panel, navigate to omniverse://localhost/NVIDIA/Assets/Isaac/2022.2.0/Isaac/Samples/PeopleDemo/SimpleEventSimulation.

2. Double-click simple_event_simulation.usd and select Open Original File to try out the demo.

Locate the Omni.Anim.People behavior scripts:

1. Select Window > Extensions to open the Extension Manager.

2. Search for “omni.anim.people”.

3. Click the folder icon to open the Extension files.

4. Navigate to \omni\anim\people\scripts.

5. Copy the path to this directory. You use it in the next step.

Add the behavior script to each character:

1. Expand each character in the Stage.

2. Hold CTRL to multi-select each character’s SkelRoot. (One example of a character’s SkelRoot is /World/male_adult_police_04/male_adult_police_04/ManRoot/male_adult_police_04.)

3. Open the Property panel.

4. In the Python Scripting property, select Add Asset.

5. Paste your scripts path into the url. (This should end in \omni\anim\people\scripts)

6. Select the character_behavior.py script.

7. Click Select.

Attach the command file:

1. In the Behavior Control Panel you opened earlier, click the folder icon for Command File Path.

2. Navigate to omniverse://localhost/NVIDIA/Assets/Isaac/2022.2.0/Isaac/Samples/PeopleDemo/SimpleEventSimulation, and select the commands text file.

Click Play to run the scene.

Note

In the Simple_Event_Simulation characters avoiding moving objects (like robots) can also be demoed. For this, make the carter_v2_drive_to_goal robot present on stage visible. Next, navigate to /World/carter_v2_drive_to_goal/chassis_link and Right Click On Prim > Add > Python Scripting. In the Python Scripting property select Add Asset and attach the dynamic_obstacle.py python file present in the same folder as the character_behavior.py file.

Note

When opening the samples using a Linux based machine, refer to the first point in Troubleshooting.

3.2.2. Set up a Simulation from Scratch

1. First, enable the navigation extension extension by navigating to Window > Extensions, searching for “omni.anim.navigation”, selecting Navigation Bundle and clicking on the “enable” button.

2. Load the environment needed for the simulation. Make sure the parent prim /World exists, if not create the prim /World. Save and reload the scene.

3. Next, create a navigation mesh for the stage by Right Click On Stage > Create > Navigation > NavMeshVolume.

• This should create a navigation mesh with all the obstacles cut out of the navigation mesh.

4. Adding characters to the stage -

• Navigate to the characters folder at omniverse://localhost/NVIDIA/Assets/Isaac/2022.2.0/Isaac/People/Characters. Then, drag the Biped_Setup.usd file onto the /World xform.

• Next, from the Characters folder load the characters that you would like to use by navigating to a characters folder and dragging and dropping the usd file into the /World xform.

• Move the characters to their starting positions in the scene.

5. Setting up the characters -

• Find the behavior scripts provided with the extensions by navigating to Window > Extensions, searching for “people” and selecting omni.anim.people. Next, click on the folder icon to open the extension files.

• Then navigate to \omni\anim\people\scripts and locate the character_behavior.py file.

• Expand each character on the stage menu and navigate till you find their SkelRoot. Example - /World/female_adult_police_03_new/female_adult_police_03/ManRoot/female_adult_police_03.

• Select the SkelRoot of every character by holding down control and clicking on each SkelRoot. Next navigate to SkelRoot > Right Click > Add > Animation > Animation Graph and select the only available animation graph.

• Select the SkelRoot of every character by holding down control and clicking on each SkelRoot. Select SkelRoot > Right Click > Add > Python Scripting to enable python scripting.

• Go to the property window and in the Python Scripting property, select Add Asset and attach the character_behavior.py script.

6. Next open up a text file and create commands for a simulation using the guide below. Once the command file is ready, in the Behavior Control Panel set the command file in the Command File Path property.

7. Run the scene by clicking on Play.

3.3. Command Guide

The command file for a simulation contains a sequence of commands for every character. The command structure is character_name action action_params, where character_name is the name of the SkelRoot prim. Below, we list each command available and its input format.

3.3.1. GoTo

GoTo moves the character to a location. GoTo can be followed by a single point or a sequence of points, the last point must specify the ending rotation the character should have upon reaching its destination.

command structure:
    character_name GoTo x y z end_rot

    character_name Goto x1 y1 z1 x2 y2 z2 x3 y3 z3 end_rot

example:
    female_adult_police_03 GoTo 10 10 0 90

You can use ‘_’ for ending rotation if you would prefer to not set a rotation. When a sequence of points is given, the character will cross every point. If Navmesh Navigation and Dynamic Obstacle Avoidance is on then the character will try to avoid both static and dynamic obstacles.

3.3.2. LookAround

LookAround makes the character stand in the same spot, while moving its head from left to right. LookAround takes a duration value and performs the action for that duration.

command structure:
    character_name LookAround duration

example:
    female_adult_police_03 LookAround 10

3.3.3. Idle

Idle makes the character stand still. Idle takes a duration value and performs the action for that duration.

command structure:
    character_name Idle duration

example:
    female_adult_police_03 Idle 10

3.3.4. Sit

Sit makes the character go to a prim, and attempt to sit on it. After the specified duration ends, the character will stand back up.

command structure:
    character_name Sit seat_prim duration

example:
    female_adult_police_03 Sit /World/Chair 5

Note

Sit plays a fixed animation, hence the seat_prim must be scaled to fit the animation. The seat_prim must also contain two properties “walk_to_offset” and “interact_offset”. The “walk_to_offset” must specify the distance in front of the prim that the character should walk to. The “interact_offset” must specify the offset from where the sitting animation should be played. Checkout the Multi_Event_Simulation in the samples folder for an example.

3.3.5. Queue

For the Queue command, we must first define a queue. A queue can be defined in the command file as follows -

command structure:

    # defines the queue
    Queue Example_Queue

    # Defines each spot in the queue. Structure Queue_Spot 'Queue_name' 'Spot_Index' x y z rotation
    Queue_Spot Example_Queue 0 11 0 0 90

    Queue_Spot Example_Queue 1 9 0 0 0

    Queue_Spot Example_Queue 2 7 0 0 0

    Queue_Spot Example_Queue 3 5 0 0 0

After defining the queue, characters can be moved to the queue using the following -

command structure:

    # character goes to queue
    female_adult_medical_01 Queue Example_Queue

    # action to perform when at 1st position in a queue
    female_adult_medical_01 LookAround 10

    # destination to go to when leaving the queue. In this example, the character will go to 10, 10, 0.
    female_adult_medical_01 Dequeue Example_Queue 10 10 0 _

Note

All three commands listed above are mandatory for the queue command to work correctly.

3.4. Avoid Moving Objects

Characters can be made to avoid small and slow moving objects (like robots) by attaching the dynamic_obstacle.py script to the moving object. Follow these steps -

• Find the ‘moving component’ of the object you want to avoid. This is generally the base prim of a physics articulation.

• On the prim Right Click > Add > Python Scripting.

• In the property window, find the Python Scripting property, click on Select Asset button and select the dynamic_obstacle.py script in the extension files.

To see an example, load the Simple_Event_Simulation in the samples folder. Make the carter_v2_drive_to_goal robot present on stage visible. Next, navigate to /World/carter_v2_drive_to_goal/chassis_link and Right Click On Prim > Add > Python Scripting. In the Python Scripting property select Add Asset and attach the dynamic_obstacle.py python file present in the scripts folder.

3.5. Adding New Animations

Users can add their own animations/actions to the simulation capabilities with a zero code process. Follow the steps below. -

1. Open the animation graph, by navigating to Window > Animation > Animation Graph > Edit Animation Graph.

2. Double-click on the state machine to open it. Create a new state and add your animation in it.

3. Connect the state to idle state with two transitions, from and to idle. Double click the transition arrows to modify the transitions. Right click and select Condition > ConditionCompareVariable. Attach the variable to the transition.

4. Open the ConditionCompareVariable in the property window and select the ‘Action’ variable name. For the transition going to the idle node, this value must be ‘None’. For the transition going from the idle node, the value can be anything.

5. Next go the Script folder in the extension files and use the following command to generate a new python command file. Note that the cmd_name should match the name of the node in the state machine and the transition_name should match the value used in the condition variable.

   python cmd_file_generator.py -t .\commands\simple_cmd_template.txt -n cmd_name -tr transition_name
   

6. The above command will generate a python file with the name cmd_name.py. Drag and drop this file into the commands folder.

7. You can now use the command in the command file as follows - character_name command duration.

3.6. Adding New Characters

To add new characters, they must be setup using Animation Retargeting. Next, the steps in the Set up a Simulation from Scratch section can be followed to use the characters in a simulation.

3.7. Troubleshooting

• In a linux based system the usd file, behavior scripts, and the command file have to be on the same file system (either on Nucleus or local file system). When trying the samples we recommend uploading the scripts folder onto the local Nucleus server and then attaching them to the characters.

• In case an error is produced by the behavior scripts, the scripts need to be reinitialized by reloading the stage - File > Reopen

• In case a navigation-mesh is not generated after creating a navigation volume, open Window > Navigation > Navmesh. In the NavMesh panel reset the defaults to the values shown below.

3.8. Limitations

The extension is being released in its beta version and has some limitations. Below, the currently observed limitations have been listed.

• Character dynamic avoidance is best effort and not guaranteed. In some scenarios where static and dynamic obstacles are present together, dynamic avoidance might cause erratic movement leading characters to walk into another character or into a static obstacle.

• Depending on the navigation mesh built, characters might go into static obstacles while walking. If this is observed the user should rebuild the navigation mesh with a higher agent radius.

• When constructing a queue, each queue spot must be at least 1 meter apart from each other.

• Character rotation in a fixed spot is currently done by rotating the whole prim.