SceneInfoNetworkNode#

The SceneInfoNetworkNode is a specialized agent in the Chat USD system that handles scene information retrieval. It extends the NetworkNode class from the LC Agent framework and is responsible for retrieving information about the current USD scene.

Overview#

The SceneInfoNetworkNode serves as the scene information provider of the Chat USD system. It can analyze the current USD stage, extract relevant information about scene elements, and provide this information to other agents. This enables the system to generate more accurate and effective code by understanding the current state of the scene.

Implementation#

The SceneInfoNetworkNode is implemented as a Python class that extends NetworkNode:

class SceneInfoNetworkNode(NetworkNode):
    """
    Use this function to get any information about the scene.

    Use it only in the case if the user wants to write ascript and you need information about the scene.

    For example use it if you need the scene hirarchy or the position of objects.

    Always use it if the user asked for an object in the scene and didn't provide the exact name.

    Never use USDCodeInteractiveNetworkNode if the exact name is not known.
    Use SceneInfoNetworkNode to get the name first and use USDCodeInteractiveNetworkNode when the name is known.

    Use this function to get any size or position of the object.
    """

    default_node: str = "SceneInfoGenNode"
    code_interpreter_hide_items: Optional[List[str]] = None

    def __init__(
        self,
        question: Optional[str] = None,
        enable_interpreter_undo_stack=True,
        enable_rag=True,
        max_retries: Optional[int] = None,
        **kwargs,
    ):
        super().__init__(**kwargs)

        if max_retries is None:
            max_retries = 15

        self.add_modifier(SceneInfoPromoteLastNodeModifier())
        self.add_modifier(NetworkLenghtModifier(max_length=max_retries))
        self.add_modifier(CodeExtractorModifier())
        self.add_modifier(
            DoubleRunUSDCodeGenInterpreterModifier(
                success_message="This is some scene information:\n",
                hide_items=self.code_interpreter_hide_items,
                undo_stack=enable_interpreter_undo_stack,
                first_run=False,
            )
        )
        if enable_rag:
            self.add_modifier(USDCodeGenRagModifier())

        if question:
            with self:
                RunnableHumanNode(human_message=human_message.format(question=question))

The class initializes itself with various parameters that control its behavior, adds several modifiers to extend its functionality, and sets up a human message if a question is provided.

Human Message#

The SceneInfoNetworkNode uses a human message template to format questions for scene information retrieval:

human_message = """Consider the following task:
{question}

Generate a script that prints only the necessary information about the scene required to perform this task. Keep the script as short and concise as possible.
"""

This template is used to format the question for the node, instructing it to generate a script that prints only the necessary information about the scene.

Configuration Parameters#

The SceneInfoNetworkNode can be configured with various parameters to customize its behavior:

  • question: The question to ask about the scene

  • enable_interpreter_undo_stack: Whether to enable the interpreter undo stack

  • enable_rag: Whether to enable RAG for scene information retrieval

  • max_retries: Maximum number of retries for code execution

These parameters allow the node to be customized for different use cases and requirements.

Modifiers#

The SceneInfoNetworkNode uses several modifiers to extend its functionality:

  1. SceneInfoPromoteLastNodeModifier: Promotes code and error messages from subnetworks

  2. NetworkLenghtModifier: Controls the maximum length of the network

  3. CodeExtractorModifier: Extracts code snippets from responses

  4. DoubleRunUSDCodeGenInterpreterModifier: Executes code snippets and handles errors

  5. USDCodeGenRagModifier: Enhances scene information retrieval with RAG

These modifiers work together to provide a comprehensive scene information retrieval experience.

SceneInfoPromoteLastNodeModifier#

The SceneInfoPromoteLastNodeModifier promotes code and error messages from subnetworks to the main network. This ensures that the user sees the final code and any error messages, even if they were generated by a subnetwork.

NetworkLenghtModifier#

The NetworkLenghtModifier controls the maximum length of the network. This prevents the network from growing too large and consuming too many resources.

CodeExtractorModifier#

The CodeExtractorModifier extracts code snippets from responses. It identifies code blocks in the response and extracts them for execution.

DoubleRunUSDCodeGenInterpreterModifier#

The DoubleRunUSDCodeGenInterpreterModifier executes code snippets and handles errors. It is configured with a success message and to skip the first run, focusing on the second run for execution.

USDCodeGenRagModifier#

The USDCodeGenRagModifier enhances scene information retrieval with Retrieval-Augmented Generation (RAG). It retrieves relevant information from a knowledge base to improve the quality of the generated code.

Scene Information Retrieval Process#

The scene information retrieval process in SceneInfoNetworkNode follows these steps:

  1. Query Analysis: The user’s query is analyzed to understand what information is needed

  2. Script Generation: A script is generated to retrieve the necessary information

  3. Script Execution: The script is executed to retrieve the information

  4. Information Formatting: The retrieved information is formatted for presentation

  5. Response Generation: A response is generated with the formatted information

This process ensures that the retrieved information is accurate, relevant, and presented in a useful format.

Usage in Chat USD#

In Chat USD, the SceneInfoNetworkNode is registered with the node factory and configured with specific parameters:

get_node_factory().register(
    SceneInfoNetworkNode,
    name="ChatUSD_SceneInfo",
    enable_interpreter_undo_stack=False,
    max_retries=1,
    enable_rag=need_rags,
    hidden=True,
)

This registration makes the node available for use by the ChatUSDNetworkNode, which routes scene information queries to it.

Integration with Other Agents#

The SceneInfoNetworkNode is designed to work closely with other agents in the Chat USD system, particularly the USDCodeInteractiveNetworkNode. The typical workflow is:

  1. Use SceneInfoNetworkNode to retrieve information about the scene

  2. Use USDCodeInteractiveNetworkNode to generate and execute code based on this information

This integration ensures that the generated code is accurate and effective, as it is based on the current state of the scene.

Importance in the Chat USD System#

The SceneInfoNetworkNode plays a critical role in the Chat USD system, as it provides the foundation for accurate and effective code generation. By retrieving information about the current state of the scene, it enables the system to generate code that is tailored to the specific scene, rather than generic code that may not work correctly.

The system message for the ChatUSDSupervisorNode emphasizes the importance of the SceneInfoNetworkNode:

3. ChatUSD_SceneInfo [CRITICAL FOR SCENE OPERATIONS]
   - Maintains current scene state knowledge
   - Must be consulted FIRST for any scene manipulation tasks
   - Required for:
     * Any operation where prim name is not explicitly provided
     * Any attribute manipulation without explicit values
     * Operations requiring knowledge of:
       - Prim existence or location
       - Prim properties (size, position, rotation, scale)
       - Prim hierarchy
       - Prim type or nature
       - Current attribute values
       - Scene structure
       - Available materials
       - Relationship between prims
       - Bounds or extents
       - Layer structure
       - Stage metadata
   - Provides scene context for other functions
   - Should be used before USD code generation for scene operations
   - Cannot generate complex code but provides essential scene data

This emphasis ensures that the SceneInfoNetworkNode is used appropriately in the system, leading to more accurate and effective code generation.