Camera Manipulation#

Mouse interaction with the Viewport is provided by the omni.kit.manipulator.camera extension, which is built on top of the omni.ui.scene framework. Bindings for keyboard and mouse interaction are available as carb settings and can be remapped or customized.

import carb.settings
settings = carb.settings.get_settings()

default_bindings = settings.get("/exts/omni.kit.viewport.window/bindings/camera")

# Print the default bindings to control a camera
print(default_bindings)

# A dictionary of GestureName => Keyboard & Mouse to control the camera
# {'PanGesture': 'Any MiddleButton', 'TumbleGesture': 'Alt LeftButton', 'ZoomGesture': 'Alt RightButton', 'LookGesture': 'RightButton', 'ZoomScrollGesture': 'Any', 'FlightSpeedGesture': 'RightButton', 'FlightMode': 'RightButton'}

# Set the setting to an empty dictionary, will mean no camera manipulation will occur
settings.set("/exts/omni.kit.viewport.window/bindings/camera", {})

# Remap the Keyboard and mouse combinations
custom_bindings = {
    # Left and Right mouse buttons will cause a pan/move operation (which is MiddleButton by default)
    "PanGesture": 'LeftButton RightButton',

    # Right mouse buttons will cause a tumble operation (which is Alt + LeftButton by default)
    "TumbleGesture": 'RightButton',

    # Middle mouse with any or no modifier will invoke the look (rotate around camera)
    "LookGesture": 'Any MiddleButton',

    # Enter Flight mode by pressing alt and middle mouse (WASD keys then control camera)
    "FlightMode": 'Alt MiddleButton',

    # Keep the default of right button + scroll wheel editing the flight mode speed
    "FlightSpeedGesture": 'RightButton',

    # Keep the zoom in/out operation as the defaults are (Alt + RightButton or ScrollWheel)
    "ZoomGesture": 'Alt RightButton',
    "ZoomScrollGesture": 'Any',
}
settings.set("/exts/omni.kit.viewport.window/bindings/camera", custom_bindings)

# Print the bindings now (and interact with the Viewport to test them)
print(settings.get("/exts/omni.kit.viewport.window/bindings/camera"))

# Then restore back to the defaults
settings.set("/exts/omni.kit.viewport.window/bindings/camera", default_bindings)

# Print the bindings again (and interact with the Viewport to test them)
print(settings.get("/exts/omni.kit.viewport.window/bindings/camera"))

Binding to a custom operation#

The above code showed how to remap key and mouse combination to existing actions. You can also provide your own implementation for a custom behavior

# Import the base-class for the operation
from omni.kit.manipulator.camera import CameraGestureBase

class MyCustomOperation(CameraGestureBase):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        # Cache the model items we will be setting to
        self.__move_item = self.model.get_item("move")
        self.__tumble_item = self.model.get_item("tumble")

    def on_mouse_move(self, mouse_moved):
        # Print the mouse delta for demonstration
        print("mouse_moved", mouse_moved)

        # Get the configured move speeds from the model
        world_speed = self.world_speed
        move_speed = self.move_speed
        tumble_speed = self.tumble_speed

        # Accumulate the move from the change in x on the mouse
        self._accumulate_values(self.__move_item,
                                mouse_moved[0] * 0.5 * world_speed[0] * move_speed[0],
                                0,
                                0)

        # Accumulate a tumble from the change in y on the mouse
        self._accumulate_values(self.__tumble_item,
                                0,
                                mouse_moved[1] * tumble_speed[1] * 90,
                                0)


# NOTE: This portion is only relevant when running in the script-editor,
# as custom bindings are strings and operate on qualified names to module objects.
#
# For typical usage in an extension, the module would already exist for the extension and provide the qualified name.
#
if True:
    import importlib.util
    import sys
    from types import ModuleType

    # Create a new module dynamically, called my_custom_extension
    new_module = ModuleType("my_custom_extension")
    # Insert it into sys.modules
    sys.modules["my_custom_extension"] = new_module
    # Add the class to the module (which is then available as my_custom_extension.MyCustomOperation)
    new_module.MyCustomOperation = MyCustomOperation

# Save the default bindings so they can be restored
import carb.settings
settings = carb.settings.get_settings()
default_bindings = settings.get("/exts/omni.kit.viewport.window/bindings/camera")


# Make the Viewport only respond to LeftButton drags to manipulate the camera
custom_bindings = {
    "my_custom_extension.MyCustomOperation": 'LeftButton',
}

settings.set("/exts/omni.kit.viewport.window/bindings/camera", custom_bindings)

# Now any horizontal left-button-drag in the Viewport will pan the camera, while a vertical left-button-drag tumbles

Operations and Values#

The CameraManipulator model store’s the amount of movement according to three modes, applied in this order:

  1. Tumble

  2. Look

  3. Move (Pan)

  4. Fly (FlightMode / WASD navigation)

tumble#

Tumble values are specified in degrees as the amount to rotate around the current up-axis. These values should be pre-scaled with any speed before setting into the model. This allows for different manipulators/gestures to interpret speed differently, rather than lock to a constant speed.

# Tumble by 180 degrees around Y (as a movement across ui X would cause)
model.set_floats('tumble', [0, 180, 0])

look#

Look values are specified in degrees as the amount to rotate around the current up-axis. These values should be pre-scaled with any speed before setting into the model. This allows for different manipulators/gestures to interpret speed differently, rather than lock to a constant speed.

# Look by 90 degrees around X (as a movement across ui Y would cause)
# i.e Look straight up
model.set_floats('look', [90, 0, 0])

move#

Move values are specified in world units and the amount to move the camera by. Move is applied after rotation, so the X, Y, Z are essentially left, right, back. These values should be pre-scaled with any speed before setting into the model. This allows for different manipulators/gestures to interpret speed differently, rather than lock to a constant speed.

# Move left by 30 units, up by 60 and back by 90
model.set_floats('move', [30, 60, 90])

fly#

Fly values are the direction of flight in X, Y, Z. Fly is applied after rotation, so the X, Y, Z are essentially left, right, back. These values will be scaled with fly_speed before aplication. Because fly is a direction with fly_speed automatically applied, if a gesture/manipulator wants to fly slower without changing fly_speed globally, it must apply whatever factor is required before setting.

# Move left
model.set_floats('fly', [1, 0, 0])
# Move up
model.set_floats('fly', [0, 1, 0])

Speed#

By default the Camera manipulator will map a full mouse move across the viewport as follows:

  • Pan: A full translation of the center-of-interest across the Viewport.

  • Tumble: A 180 degree rotation across X or Y.

  • Look: A 180 degree rotation across X and a 90 degree rotation across Y.

These speed can be adjusted by setting float values into the model.

world_speed#

The Pan and Zoom speed can be adjusted with three floats set into the model as ‘world_speed’.

# Half the movement speed for both Pan and Zoom
pan_speed_x = 0.5, pan_speed_y = 0.5, zoom_speed_z = 0.5
model.set_floats('world_speed', [pan_speed_x, pan_speed_y, zoom_speed_z])

rotation_speed#

The Tumble and Look speed can be adjusted with either a scalar value for all rotation axes or per component.

# Half the rotation speed for both Tumble and Look
rot_speed_both = 0.5
model.set_floats('rotation_speed', [rot_speed_both])
# Half the rotation speed for both Tumble and Look in X and quarter if for Y
rot_speed_x = 0.5, rot_speed_y = 0.25
model.set_floats('rotation_speed', [rot_speed_x, rot_speed_y])

tumble_speed#

Tumble can be adjusted separately with either a scalar value for all rotation axes or per component. The final speed of Tumble operation is rotation_speed * tumble_speed

# Half the rotation speed for Tumble
rot_speed_both = 0.5
model.set_floats('tumble_speed', [rot_speed_both])
# Half the rotation speed for Tumble in X and quarter if for Y
rot_speed_x = 0.5, rot_speed_y = 0.25
model.set_floats('tumble_speed', [rot_speed_x, rot_speed_y])

look_speed#

Look can be adjusted separately with either a scalar value for all rotation axes or per component. The final speed of a Look operation is rotation_speed * tumble_speed

# Half the rotation speed for Look
rot_speed_both = 0.5
model.set_floats('look_speed', [rot_speed_both])
# Half the rotation speed for Tumble in X and quarter if for Y
rot_speed_x = 0.5, rot_speed_y = 0.25
model.set_floats('look_speed', [rot_speed_x, rot_speed_y])

fly_speed#

The speed at which FlightMode (WASD navigation) will fly through the scene. FlightMode speed can be adjusted separately with either a scalar value for all axes or per component.

# Half the speed in all directions
fly_speed = 0.5
model.set_floats('fly_speed', [fly_speed])
# Half the speed when moving in X or Y, but double it moving in Z
fly_speed_x_y = 0.5, fly_speed_z = 2
model.set_floats('fly_speed', [fly_speed_x_y, fly_speed_x_y, fly_speed_z])

Undo#

Because we’re operating on a unique omni.usd.UsdContext we don’t want movement in the preview-window to affect the undo-stack. To accomplish that, we’ll set the ‘disable_undo’ value to an array of 1 int; essentially saying disable_undo=True.

# Let's disable any undo for these movements as we're a preview-window
model.set_ints('disable_undo', [1])

Disabling operations#

By default the manipulator will allow Pan, Zoom, Tumble, and Look operations on a perspective camera, but only allow Pan and Zoom on an orthographic one. If we want to explicitly disable any operations again we set int values as booleans into the model.

disable_tumble#

# Disable the Tumble manipulation
model.set_ints('disable_tumble', [1])

disable_look#

# Disable the Look manipulation
model.set_ints('disable_look', [1])

disable_pan#

# Disable the Pan manipulation.
model.set_ints('disable_pan', [1])

disable_zoom#

# Disable the Zoom manipulation.
model.set_ints('disable_zoom', [1])