Add New Functionality To Our Experience#

Now that you understand client/server setup and communication, you’ll go through an example of adding a new button. First, you’ll add the button to our SwiftUI panel and add the functionality to our state and client management. Once you’re happy with your UI on the client, you’ll jump over to Omniverse and add the required ActionGraph to our stage.

Client#

On the client-side, you will develop a new button in the SwiftUI panel to trigger the randomize function. This button will send a message to the server, initiating the randomization process for variant colors and styles. This function will work once we’ve added the needed code to the following components:

  • ConfigureView

  • OmniverseStateManager

  • OmniverseClientEvents

ConfigureView#

In the ConfigureView file, replace the current var purseActions with the following code starting at line 86.

// Defines a view that contains actions related to the asset

var purseActions: some View {
    VStack {
        // toggle the visibility of the purse
        LabelButton(
            label: "Visibility",
            onText: "On",
            offText: "Off",
            toggleView: true,
            icon: AnyView(dashed),
            isOn: true
        ) { isOn in
        // Updates the state of purse visibility
        appModel.stateManager["purseVisibility"] = isOn
            ? PurseVisibility.visible
            : PurseVisibility.hidden
        } textCondition: { _ in
            // Condition to determine the text displayed on the button
            viewModel.purseVisible
        }

        // Horizontal stack to place the Rotate and Randomize buttons
        // side by side
        HStack {

            // The Rotate button to rotate the purse
            LabelButton(
                label: "Rotate",            // The button's label
                toggleView: false,          // not a toggle button
                icon: AnyView(dashed)       // Icon for the button
            ) { isOn in
                // Sends the appropriate rotation action
                if isOn {
                    appModel.stateManager.send(RotationAction.rotateCCW)
                } else {
                    appModel.stateManager.send(RotationAction.rotateCW)
                }
        }

        // The Randomize button to randomize the purse
        LabelButton(
            label: "Randomize",          // The button's label
            toggleView: false,           // not a toggle button
            icon: AnyView(dashed)        // Icon for the button
        ) { _ in
            // Sends a request to randomize the purse
            appModel.stateManager.send(Randomize.randomize)
        }
    }

    Spacer() // Adds space below the buttons to push them up
    }
}

This code inserts a new Randomize button into a VStack with our existing buttons. You may get an error that Randomize has no member, we’ll fix that next.

OmniverseClientEvents#

In the OmniverseClientEvents file, insert the following code at the end of the file, after line 267””

// Represents an input event for a randomize action
public struct RandomizeClientInputEvent: EncodableInputEvent {

    // Message data indicating the type of action
    public let message: [String: String]
    // Type of event, always set to "executeAction"
    public let type = "executeAction"
    // Initializes the event with a message indicating a randomize action
    public init(_ randomize: Randomize) {
        self.message = ["actionType": "randomize"]
    }
}
// Represents different types of randomize actions
public enum Randomize: String, OmniverseMessageProtocol {
    case randomize = "True"
    // Provides an encodable representation of the randomize action
    public var encodable: any EncodableInputEvent {
        return RandomizeClientInputEvent(self)
    }
}

This code adds a Randomize button. You can now build your application and run it in the simulator, and you should see the Randomize button added to your UI.

_images/randomize-btn.png

Omniverse#

In Omniverse, we now need to add the ActionGraph logic associated with this function. We’ll add a graph to our stage, insert some python code to randomize the variants, and listen for a message event from the client to trigger the script.

_images/message.png

Action Graph Logic Setup#

  1. Open the Purse dataset USD in Omniverse Kit Editor.

  2. Create a new action graph using one of the following methods:

    • Right-click (RMB) within the stage panel, navigate to Create > Visual Scripting and click Action Graph

    • Navigate to Window > Visual Scripting > Action Graph and click New Action Graph

  3. Right-click (RMB) on the new action graph and select “rename” from the dropdown menu. Rename it to “randomizeController”.

  4. Take note of the Nodes column to the right of the Graph, you’ll create some actionGraph nodes within this graph that will define the randomize logic.

  5. On MessageBus Event
    • Add+ an Attribute from the Property Panel.

      • Set the Attribute name to message

      • Set the Attribute Type to string

    • Set the Event name to executeAction

    • Uncheck Only Simulate On Play

  6. Script Node:
    • Add+ an Attribute from the Property Panel.

      • Set the Name to primPath.

      • Set the Port Type to input

      • Set the Data Type to token

    • Add the following prim path to the new primPath Attribute: /World/SM_LuxuryBag_A01_01

    • Add+ an Attribute from the Property Panel:

      • Set the Name to randomize

      • Set the Port Type to input

      • Set the Data Type to token

      • After selecting Randomize on the client, you should see this populate to say { "actionType":"randomize"}

    • Connect the Received event from On MessageBus Event to Exec In on the Script Node.

  7. To Token

    • Connect on_messagebus:message to input:value

    • Connect token to script_node:input:randomize

  8. Insert the following code block into the script node that you created earlier:

A script node within the randomizeController action graph which randomly selects variants for a specified USD prim.#
def setup(db: og.Database):
    pass

def cleanup(db: og.Database):
    pass

def compute(db: og.Database):
    import carb
    from omni.usd import get_context
    import random

    # Check if the randomize input is triggered
    if not db.inputs.randomize:
        return False

    # Get the USD stage
    stage = get_context().get_stage()

    # Retrieve the prim path input
    primPath = db.inputs.primPath  # primPath as string

    # Define the variant sets and variants
    variantOptions = {
        "Gold": ["Yellow", "White", "Pink"],
        "Locker": ["Triangle", "Ring"],
        "KeyRing": ["ON", "OFF"],
        "Handle": ["Round", "Square"],
        "Leather": ["Beige", "Black", "BlackEmboss", "Orange", "Tan", "White"]
    }

    # Get the prim using the prim path
    prim = stage.GetPrimAtPath(primPath)
    if not prim:
        carb.log_error(f"Not valid prim: {primPath}")
        return False

    # Get the variant sets from the prim
    variantSets = prim.GetVariantSets()
    if not variantSets:
        carb.log_error(f"Prim does not have Variant Sets: {primPath}")
        return False

    # Randomly select a variant for each variant set and set it
    for variantSetName, variants in variantOptions.items():
        variantSet = variantSets.GetVariantSet(variantSetName)
        if not variantSet:
            carb.log_error(
                f"Prim `{primPath}` does not have"
                + f" variant set: {variantSetName}"
            )
            continue

        # Randomly select a variant
        selectedVariant = random.choice(variants)
        variantSet.SetVariantSelection(selectedVariant)
        print(f"Set {variantSetName} to {selectedVariant}")

    return True

For brevity, we won’t add any ack logic to this setup, but you can if you’d like to notify the client or another event that randomize has completed.

Test#

Now we’re ready to test! You can Start AR with your modified stage in Omniverse, and you can launch your client from Xcode into the Simulator or deploy to the Vision Pro. Connect to your Omniverse server and try the Randomize button.

On this page