Integrating an Authoring Layer#

In this article, a toy example using the Execution Framework is used to describe an online bakery. While the simplistic subject matter of the example is contrived, the concepts demonstrated in the example have real-world applications.

The article is structured such that the example starts simple, and new concepts are introduced piecemeal.

The Authoring Layer#

The Execution Framework, in particular the execution graph, is a common language to describe execution across disparate software components. It is the job of each component (or an intermediary) to populate the execution graph based on some internal description. We call this per-component, internal description the authoring layer. It is common to have multiple different authoring layers contribute to a single execution graph.

This example demonstrate a single authoring layer that describes several online bakeries. The data structures used by this authoring layer is as follows:

Listing 38 Data structures used in the authoring layer to describe each bakery.#
struct BakedGood
{
    unsigned int bakeMinutes;
    std::string name;
};

struct Order
{
    std::string customer;
    std::vector<BakedGood> bakedGoods;
};

struct Bakery
{
    std::string name;
    std::vector<Order> orders;
};

The example starts by describing two bakeries at the authoring layer:

Listing 39 Authoring layer code that describes the orders for two bakeries.#
std::vector<Bakery> bakeries {
    Bakery {
        "The Pie Hut", // bakery name
        {
            Order {
                "Tracy", // customer
                {
                    BakedGood { 20, "applePie" },
                    BakedGood { 30, "chickenPotPie" }
                }
            },
            Order {
                "Kai", // customer
                {
                    BakedGood { 22, "peachPie" },
                }
            }
        }
    },
    Bakery {
        "Sam's Bakery", // bakery name
        {
            Order {
                "Alex", // customer
                {
                    BakedGood { 20, "blueberryPie" },
                }
            }
        }
    }
};

Setting Up the Execution Graph#

With the authoring layer defined, the following code is then used to populate the execution graph based on the authoring layer description:

Listing 40 Top-level code to populate the execution graph from the authoring layer’s description.#
// this example manually creates an execution graph.  in most applications (e.g. kit-based applications) this will
// already be created for you
GraphPtr graph = Graph::create("exec.graph");

// as mentioned above, the builder context, pass pipeline, and builder will likely already be created for you in
// real-world scenarios.
GraphBuilderContextPtr builderContext{ GraphBuilderContext::create(graph, PassPipeline::create()) };
GraphBuilderPtr builder{ GraphBuilder::create(builderContext) };

// ef relies on the user to maintain a reference (i.e. a call omni::core::Object::acquire()) on each node in a graph
// definition.  this can be done by simply holding an array of NodePtr objects in your definition.  in this case,
// since we're populating the top-level graph definition, we simply store the NodePtrs here.
std::vector<NodePtr> nodes;

for (auto& bakery : bakeries) // for each bakery
{
    auto node = Node::create(
        graph, // this makes the node a part of the execution graph's top-level graph definition
        BakeryGraphDef::create(builder, bakery), // bakery's definition (i.e. work description)
        carb::fmt::format("node.bakery.{}", bakery.name)
    );

    // connect the bakery to the root of the execution graph's definition so that it will be executed.  only nodes
    // in a graph definition that can reach the definition's root node will be executed.
    builder->connect(graph->getRoot(), node);

    nodes.emplace_back(std::move(node));
}

The execution graph can be visualized as follows:

flowchart LR 00000261A3F90560(( )) 00000261A3F90560-->00000261A0498170 00000261A3F90560-->00000261A3FB0260 00000261A0498170(node.bakery.The Pie Hut) 00000261A0498170-.->00000261A3F8DA50 00000261A3FB0260(node.bakery.Sam's Bakery) 00000261A3FB0260-.->00000261A3F8E750 subgraph 00000261A3F8DA50[def.bakery] direction LR style 00000261A3F8DA50 fill:#FAFAFA,stroke:#777777 00000261A3F90CE0(( )) 00000261A3F90CE0-->00000261A3F91000 00000261A3F90CE0-->00000261A3F913C0 00000261A3F90CE0-->00000261A3F90600 00000261A3F90CE0-->00000261A3F90920 00000261A3F91000(node.bakery.The Pie Hut.preHeatOven) 00000261A3F91000-.->00000261A3DC44C0 00000261A3F91000-->00000261A3F90E20 00000261A3F91000-->00000261A3F906A0 00000261A3F91000-->00000261A3F90A60 00000261A3F913C0(node.bakedGood.prepare.applePie) 00000261A3F913C0-.->00000261A3D77160 00000261A3F913C0-->00000261A3F90E20 00000261A3F90600(node.bakedGood.prepare.chickenPotPie) 00000261A3F90600-.->00000261A3D76B60 00000261A3F90600-->00000261A3F906A0 00000261A3F90920(node.bakedGood.prepare.peachPie) 00000261A3F90920-.->00000261A3D767A0 00000261A3F90920-->00000261A3F90A60 00000261A3F90E20(node.bakedGood.bake.applePie) 00000261A3F90E20-.->00000261A3CA6BF0 00000261A3F90E20-->00000261A3F909C0 00000261A3F90E20-->00000261A3F911E0 00000261A3F906A0(node.bakedGood.bake.chickenPotPie) 00000261A3F906A0-.->00000261A3CA5210 00000261A3F906A0-->00000261A3F909C0 00000261A3F906A0-->00000261A3F911E0 00000261A3F90A60(node.bakedGood.bake.peachPie) 00000261A3F90A60-.->00000261A3CA53C0 00000261A3F90A60-->00000261A3F90EC0 00000261A3F90A60-->00000261A3F911E0 00000261A3F909C0(node.bakedGood.ship.Tracy) 00000261A3F909C0-.->00000261A3DC4510 00000261A3F911E0(node.bakery.The Pie Hut.turnOffOven) 00000261A3F911E0-.->00000261A3DC3F20 00000261A3F90EC0(node.bakedGood.ship.Kai) 00000261A3F90EC0-.->00000261A3DC45B0 end 00000261A3CA5210{{def.bakedGood.bake}} 00000261A3CA53C0{{def.bakedGood.bake}} 00000261A3CA6BF0{{def.bakedGood.bake}} subgraph 00000261A3D767A0[def.bakedGood.prepare] direction LR style 00000261A3D767A0 fill:#FAFAFA,stroke:#777777 00000261A3F90F60(( )) 00000261A3F90F60-->00000261A3F90740 00000261A3F90740(node.bakedGood.gather.peachPie) 00000261A3F90740-.->00000261A3CA52A0 00000261A3F90740-->00000261A3F907E0 00000261A3F907E0(node.bakedGood.assemble.peachPie) 00000261A3F907E0-.->00000261A3CA5330 end 00000261A3CA52A0{{def.bakedGood.gatherIngredients}} 00000261A3CA5330{{def.bakedGood.assemble}} subgraph 00000261A3D76B60[def.bakedGood.prepare] direction LR style 00000261A3D76B60 fill:#FAFAFA,stroke:#777777 00000261A3F91140(( )) 00000261A3F91140-->00000261A3F90BA0 00000261A3F90BA0(node.bakedGood.gather.chickenPotPie) 00000261A3F90BA0-.->00000261A3CA7100 00000261A3F90BA0-->00000261A3F904C0 00000261A3F904C0(node.bakedGood.assemble.chickenPotPie) 00000261A3F904C0-.->00000261A3CA73D0 end 00000261A3CA7100{{def.bakedGood.gatherIngredients}} 00000261A3CA73D0{{def.bakedGood.assemble}} subgraph 00000261A3D77160[def.bakedGood.prepare] direction LR style 00000261A3D77160 fill:#FAFAFA,stroke:#777777 00000261A3F91320(( )) 00000261A3F91320-->00000261A3F90B00 00000261A3F90B00(node.bakedGood.gather.applePie) 00000261A3F90B00-.->00000261A3CA6B60 00000261A3F90B00-->00000261A3F90D80 00000261A3F90D80(node.bakedGood.assemble.applePie) 00000261A3F90D80-.->00000261A3CA6530 end 00000261A3CA6530{{def.bakedGood.assemble}} 00000261A3CA6B60{{def.bakedGood.gatherIngredients}} 00000261A3DC3F20{{def.oven.turnOff}} 00000261A3DC44C0{{def.oven.preHeat}} 00000261A3DC4510{{def.order.ship}} 00000261A3DC45B0{{def.order.ship}} subgraph 00000261A3F8E750[def.bakery] direction LR style 00000261A3F8E750 fill:#FAFAFA,stroke:#777777 00000261A3FB0B20(( )) 00000261A3FB0B20-->00000261A3FAFFE0 00000261A3FB0B20-->00000261A3FB0940 00000261A3FAFFE0(node.bakery.Sam's Bakery.preHeatOven) 00000261A3FAFFE0-.->00000261A3DC3A20 00000261A3FAFFE0-->00000261A3FB0760 00000261A3FB0940(node.bakedGood.prepare.blueberryPie) 00000261A3FB0940-.->00000261A3D76CE0 00000261A3FB0940-->00000261A3FB0760 00000261A3FB0760(node.bakedGood.bake.blueberryPie) 00000261A3FB0760-.->00000261A3CA60B0 00000261A3FB0760-->00000261A3FAF720 00000261A3FB0760-->00000261A3FB0DA0 00000261A3FAF720(node.bakedGood.ship.Alex) 00000261A3FAF720-.->00000261A3DC3F70 00000261A3FB0DA0(node.bakery.Sam's Bakery.turnOffOven) 00000261A3FB0DA0-.->00000261A3DC4560 end 00000261A3CA60B0{{def.bakedGood.bake}} subgraph 00000261A3D76CE0[def.bakedGood.prepare] direction LR style 00000261A3D76CE0 fill:#FAFAFA,stroke:#777777 00000261A3FAF680(( )) 00000261A3FAF680-->00000261A3FAFA40 00000261A3FAFA40(node.bakedGood.gather.blueberryPie) 00000261A3FAFA40-.->00000261A3CA59F0 00000261A3FAFA40-->00000261A3FB0120 00000261A3FB0120(node.bakedGood.assemble.blueberryPie) 00000261A3FB0120-.->00000261A3CA6020 end 00000261A3CA59F0{{def.bakedGood.gatherIngredients}} 00000261A3CA6020{{def.bakedGood.assemble}} 00000261A3DC3A20{{def.oven.preHeat}} 00000261A3DC3F70{{def.order.ship}} 00000261A3DC4560{{def.oven.turnOff}}

Figure 20 The execution graph showing both bakeries. Arrows with solid lines represent orchestration ordering while arrows with dotted lines represent the definition a node is using.#

You can see the execution graph has several types of entities:

  • Nodes are represented by rounded boxes. Their name starts with “node.”.

  • Opaque Definitions are represented by angled boxes. Their name starts with “def.”.

  • Graph Definitions are represented by shaded boxes. Their name, at the top of the box, starts with “def.”.

  • Root Nodes are represented by circles. Their name is not shown.

  • Edges, represented by an arrow with a solid line, show the orchestration ordering between nodes.

  • Each node points to a definition, either an opaque definition or a graph definition. This relationship is represented by an arrow with a dotted line. Note, definitions can be pointed to by multiple nodes, though this example does not utilize the definition sharing feature of EF.

To simplify the example, this article focuses on a single bakery. Below, you can see a visualization of only The Pie Hut’s graph definition:

flowchart LR 00000261A3F913C0(( )) 00000261A3F913C0-->00000261A3F91320 00000261A3F913C0-->00000261A3F90B00 00000261A3F913C0-->00000261A3F911E0 00000261A3F913C0-->00000261A3F90E20 00000261A3F91320(node.bakery.The Pie Hut.preHeatOven) 00000261A3F91320-.->00000261A3DC3930 00000261A3F91320-->00000261A3F90920 00000261A3F91320-->00000261A3F90F60 00000261A3F91320-->00000261A3F90EC0 00000261A3F90B00(node.bakedGood.prepare.applePie) 00000261A3F90B00-.->00000261A3D76CE0 00000261A3F90B00-->00000261A3F90920 00000261A3F911E0(node.bakedGood.prepare.chickenPotPie) 00000261A3F911E0-.->00000261A3D77160 00000261A3F911E0-->00000261A3F90F60 00000261A3F90E20(node.bakedGood.prepare.peachPie) 00000261A3F90E20-.->00000261A3D767A0 00000261A3F90E20-->00000261A3F90EC0 00000261A3F90920(node.bakedGood.bake.applePie) 00000261A3F90920-.->00000261A3CA6530 00000261A3F90920-->00000261A3F909C0 00000261A3F90920-->00000261A3F904C0 00000261A3F90F60(node.bakedGood.bake.chickenPotPie) 00000261A3F90F60-.->00000261A3CA7070 00000261A3F90F60-->00000261A3F909C0 00000261A3F90F60-->00000261A3F904C0 00000261A3F90EC0(node.bakedGood.bake.peachPie) 00000261A3F90EC0-.->00000261A3CA6B60 00000261A3F90EC0-->00000261A3F91000 00000261A3F90EC0-->00000261A3F904C0 00000261A3F909C0(node.bakedGood.ship.Tracy) 00000261A3F909C0-.->00000261A3DC44C0 00000261A3F904C0(node.bakery.The Pie Hut.turnOffOven) 00000261A3F904C0-.->00000261A3DC4880 00000261A3F91000(node.bakedGood.ship.Kai) 00000261A3F91000-.->00000261A3DC3D40 00000261A3CA6530{{def.bakedGood.bake}} 00000261A3CA6B60{{def.bakedGood.bake}} 00000261A3CA7070{{def.bakedGood.bake}} subgraph 00000261A3D767A0[def.bakedGood.prepare] direction LR style 00000261A3D767A0 fill:#FAFAFA,stroke:#777777 00000261A3F907E0(( )) 00000261A3F907E0-->00000261A3F90880 00000261A3F90880(node.bakedGood.gather.peachPie) 00000261A3F90880-.->00000261A3CA73D0 00000261A3F90880-->00000261A3F90BA0 00000261A3F90BA0(node.bakedGood.assemble.peachPie) 00000261A3F90BA0-.->00000261A3CA6260 end 00000261A3CA6260{{def.bakedGood.assemble}} 00000261A3CA73D0{{def.bakedGood.gatherIngredients}} subgraph 00000261A3D76CE0[def.bakedGood.prepare] direction LR style 00000261A3D76CE0 fill:#FAFAFA,stroke:#777777 00000261A3F90CE0(( )) 00000261A3F90CE0-->00000261A3F90560 00000261A3F90560(node.bakedGood.gather.applePie) 00000261A3F90560-.->00000261A3CA57B0 00000261A3F90560-->00000261A3F90600 00000261A3F90600(node.bakedGood.assemble.applePie) 00000261A3F90600-.->00000261A3CA6C80 end 00000261A3CA57B0{{def.bakedGood.gatherIngredients}} 00000261A3CA6C80{{def.bakedGood.assemble}} subgraph 00000261A3D77160[def.bakedGood.prepare] direction LR style 00000261A3D77160 fill:#FAFAFA,stroke:#777777 00000261A3F90D80(( )) 00000261A3F90D80-->00000261A3F906A0 00000261A3F906A0(node.bakedGood.gather.chickenPotPie) 00000261A3F906A0-.->00000261A3CA5E70 00000261A3F906A0-->00000261A3F90740 00000261A3F90740(node.bakedGood.assemble.chickenPotPie) 00000261A3F90740-.->00000261A3CA6FE0 end 00000261A3CA5E70{{def.bakedGood.gatherIngredients}} 00000261A3CA6FE0{{def.bakedGood.assemble}} 00000261A3DC3930{{def.oven.preHeat}} 00000261A3DC3D40{{def.order.ship}} 00000261A3DC44C0{{def.order.ship}} 00000261A3DC4880{{def.oven.turnOff}}

Figure 21 Above, only the graph definition (def.bakery) describing “The Pie Hut” is shown. To simplify the example, only this part of the execution graph will be shown in the following figures.#

Building a Graph Definition#

When creating the execution graph, most of the work is done in BakeryGraphDef, which is defined as follows:

Listing 41 An implementation of INodeGraphDef that represents a bakery.#
class BakeryGraphDef : public NodeGraphDef // NodeGraphDef is an ef provided implementation of INodeGraphDef
{
public:
    // implementations of ef interfaces are encouraged to define a static create() method.  this method returns an
    // ObjectPtr which correctly manages the reference count of the returned object.
    //
    // when defining api methods like create(), the use of ObjectParam<> to accept ONI object is encouraged.  below,
    // ObjectParam<IGraphBuilder> is a light-weight object that will accept either a raw IGraphBuilder* or a
    // GraphBuilderPtr.
    static omni::core::ObjectPtr<BakeryGraphDef> create(
        omni::core::ObjectParam<IGraphBuilder> builder,
        const Bakery& bakery) noexcept
    {
        // the pattern below of creating an graph definition followed by calling build() is common.  in libraries like
        // OmniGraph, all definitions are subclassed from a public interface that specifies a build_abi() method.  since
        // the build method is virtual (in OG, not here), calling build_abi() during the constructor would likely lead
        // to incorrect behavior (i.e. calling a virtual method on an object that isn't fully instantiated is an
        // anti-pattern). by waiting to call build() after the object is fully instantiated, as below, the proper
        // build_abi() will be invoked.
        auto def = omni::core::steal(new BakeryGraphDef(builder->getGraph(), bakery));
        def->build(builder);
        return def;
    }

    // build() (usually build_abi()) is a method often seen in ef definitions.  it usually serves two purposes:
    //
    // - build the graph definition's graph
    //
    // - update the graph definition's graph when something has changed in the authoring layer
    //
    // note, this example doesn't consider updates to the authoring layer.
    void build(omni::core::ObjectParam<IGraphBuilder> parentBuilder) noexcept
    {
        // when building a graph definition, a *dedicated* builder must be created to handle connecting nodes and
        // setting the node's definitions.
        //
        // below, notice the use of 'auto'.  use of auto is highly encouraged in ef code.  in many ef methods, it is
        // unclear if the return type is either a raw pointer or a smart ObjectPtr.  by using auto, the caller doesn't
        // need to care and "The Right Thing (TM)" will happen.
        auto builder{ GraphBuilder::create(parentBuilder, this) };

        // when using the build method to repspond to update in the authoring layer, we clear out the old nodes (if
        // any). a more advanced implementation may choose to reuse nodes to avoid memory thrashing.
        m_nodes.clear();

        if (m_bakery.orders.empty())
        {
            // no orders to bake.  don't turn on the oven.
            return; // LCOV_EXCL_LINE
        }

        m_preheatOven = Node::create(
            getTopology(), // each node must be a part of a single topology.  getTopology() returns this defs topology.
            PreHeatOvenNodeDef::create(m_bakery),
            carb::fmt::format("node.bakery.{}.preHeatOven", m_bakery.name)
        );

        // connecting nodes in a graph must go through the GraphBuilder object created to construct this graph
        // definition
        builder->connect(getRoot(), m_preheatOven);

        m_turnOffOven = Node::create(
            getTopology(),
            TurnOffOvenNodeDef::create(m_bakery),
            carb::fmt::format("node.bakery.{}.turnOffOven", m_bakery.name)
        );

        for (auto& order : m_bakery.orders) // for each order
        {
            if (!order.bakedGoods.empty()) // make sure the order isn't empty
            {
                auto ship = Node::create(
                    getTopology(),
                    ShipOrderNodeDef::create(order),
                    carb::fmt::format("node.bakedGood.ship.{}", order.customer)
                );

                for (const BakedGood& bakedGood : order.bakedGoods) // for each item in the order
                {
                    auto prepare = Node::create(
                        getTopology(),
                        PrepareBakedGoodGraphDef::create(builder, bakedGood),
                        carb::fmt::format("node.bakedGood.prepare.{}", bakedGood.name)
                    );

                    auto bake = Node::create(
                        getTopology(),
                        NodeDefLambda::create( // NodeDefLambda is an opaque def which uses a lambda to perform work
                            "def.bakedGood.bake",
                            [&bakedGood = bakedGood](ExecutionTask& info)
                            {
                                log("baking {} for {} minutes", bakedGood.name, bakedGood.bakeMinutes);
                                return Status::eSuccess;
                            },
                            SchedulingInfo::eParallel
                        ),
                        carb::fmt::format("node.bakedGood.bake.{}", bakedGood.name)
                    );

                    builder->connect(getRoot(), prepare);

                    builder->connect(prepare, bake);
                    builder->connect(bake, ship);

                    builder->connect(m_preheatOven, bake);
                    builder->connect(bake, m_turnOffOven);

                    // ef expects graph definitions to maintain a ref (i.e. call omni::core::IObject::acquire()) on any
                    // nodes in the graph definition.  an easy way to do this is to keep an array of NodePtr objects in
                    // the graph definition.  NodePtr objects correctly call omni::core::IObject::acquire() and
                    // omni::core::IObject::release() at the proper times.
                    m_nodes.emplace_back(std::move(prepare));
                    m_nodes.emplace_back(std::move(bake));
                }

                m_nodes.emplace_back(std::move(ship));
            }
        }
    }

private:
    BakeryGraphDef(IGraph* graph, const Bakery& bakery) noexcept
        : NodeGraphDef(graph, "def.bakery"), m_bakery(bakery)
    {
    }

    const Bakery& m_bakery;
    NodePtr m_preheatOven;
    NodePtr m_turnOffOven;
    std::vector<NodePtr> m_nodes;
};

In Figure 21, you can see:

  • Preparation of the ingredients can run concurrently with pre-heating the oven.

  • Once pre-heating and ingredient preparation has completed, the goods are baked in parallel.

  • Once all goods in an order have baked, they are shipped to the customer.

  • Once all goods across all orders have finished baking, the oven is turned off. This action can happen in parallel with shipping the goods.

There are two types of definitions seen in the graph: graph definitions and opaque definitions.

Opaque Definitions#

Opaque definitions are represented by angled boxes. An example of an opaque definition is the definition used to pre-heat the oven:

Listing 42 An example of an opaque definition used to pre-heat the oven.#
class PreHeatOvenNodeDef : public NodeDef
{
public:
    static omni::core::ObjectPtr<PreHeatOvenNodeDef> create(const Bakery& bakery) noexcept
    {
        auto def = omni::core::steal(new PreHeatOvenNodeDef(bakery));
        return def;
    }

protected:
    Status execute_abi(ExecutionTask* info) noexcept override
    {
        log("pre-heating oven at {}", m_bakery.name);
        return Status::eSuccess;
    };

    SchedulingInfo getSchedulingInfo_abi(const ExecutionTask* info) noexcept override
    {
        return SchedulingInfo::eParallel;
    }

private:
    PreHeatOvenNodeDef(const Bakery& bakery) noexcept
        : NodeDef("def.oven.preHeat"), m_bakery(bakery)
    { }

    const Bakery& m_bakery;
};

The two main methods to overload are INodeDef::execute() and INodeDef::getSchedulingInfo(). INodeDef::execute() performs the definition’s work while INodeDef::getSchedulingInfo() provides hints to EF about how to schedule the work.

Because this is an example, the work performed in PreHeatOvenNodeDef is trivial. When performing uncomplicated work, developers are often better served using NodeDefLambda::create() which accepts a lambda, a SchedulingInfo object, and produces an INodeDef object. See the bake node in BakeryGraphDef for an example use of NodeDefLambda.

Creating an opaque definition by sub-classing from NodeDef (e.g. PreHeatOvenNodeDef) is useful when:

  • The developer wishes to provide additional methods on the definition.

  • The opaque definition needs to store authoring data whose ownership and lifetime can’t be adequately captured in the lambda provided to NodeDefLambda.

Graph Definitions#

The second type of definitions seen in Figure 21 are graph definitions. Graph definitions are represented by shaded boxes. Each graph definition has a root node, represented by a circle. In Figure 21, there is one type of graph definition: PrepareBakedGoodGraphDef.

Here, you can see the code behind PrepareBakedGoodGraphDef:

Listing 43 An example of a graph definition used to prepare a baked good.#
class PrepareBakedGoodGraphDef : public NodeGraphDefT<INodeGraphDef, INodeGraphDefDebug, IPrivateBakedGoodGetter>
{
public:
    static omni::core::ObjectPtr<PrepareBakedGoodGraphDef> create(
        omni::core::ObjectParam<IGraphBuilder> builder,
        const BakedGood& bakedGood) noexcept
    {
        auto def = omni::core::steal(new PrepareBakedGoodGraphDef(builder->getGraph(), bakedGood));
        def->build(builder);
        return def;
    }

    void build(omni::core::ObjectParam<IGraphBuilder> parentBuilder) noexcept
    {
        auto builder = GraphBuilder::create(parentBuilder, this);

        m_nodes.clear();

        auto gather = Node::create(
            getTopology(),
            NodeDefLambda::create("def.bakedGood.gatherIngredients",
                [this](ExecutionTask& info)
                {
                    log("gather ingredients for {}", m_bakedGood.name);
                    return Status::eSuccess;
                },
                SchedulingInfo::eParallel
            ),
            carb::fmt::format("node.bakedGood.gather.{}", m_bakedGood.name)
        );

        auto assemble = Node::create(
            getTopology(),
            NodeDefLambda::create("def.bakedGood.assemble",
                [this](ExecutionTask& info)
                {
                    log("assemble {}", m_bakedGood.name);
                    return Status::eSuccess;
                },
                SchedulingInfo::eParallel
            ),
            carb::fmt::format("node.bakedGood.assemble.{}", m_bakedGood.name)
        );

        builder->connect(getRoot(), gather);
        builder->connect(gather, assemble);

        m_nodes.emplace_back(std::move(gather));
        m_nodes.emplace_back(std::move(assemble));
    }

    const BakedGood& getBakedGood() const noexcept override
    {
        return m_bakedGood;
    }

private:
    using BaseType = NodeGraphDefT<INodeGraphDef, INodeGraphDefDebug, IPrivateBakedGoodGetter>;
    PrepareBakedGoodGraphDef(IGraph* graph, const BakedGood& bakedGood) noexcept
        : BaseType(graph, "def.bakedGood.prepare"), m_bakedGood(bakedGood)
    { }

    const BakedGood& m_bakedGood;
    std::vector<NodePtr> m_nodes;
};

PrepareBakedGoodGraphDef creates a simple graph with two opaque nodes, one which gathers all of the ingredients of the baked good and another which assembles the baked good.

Population Passes#

A powerful feature in EF are passes. Passes are user created chunks of code that transform the graph during graph construction. As an example, an oft performed transformation is one in which a generic graph definition is replaced with an optimized user defined graph definition.

Figure 21 shows node.bakedGood.prepare.chickenPotPie has a fairly generic graph definition. An opportunity exists to replace this definition with one which can prepare ingredients in parallel. To do this, a population pass is used:

Listing 44 The PopulateChickenPotPiePass is a population pass which replaces the generic baked good preparation definition with one which optimizes the preparation of chicken pot pie.#
class PopulateChickenPotPiePass : public omni::graph::exec::unstable::Implements<IPopulatePass>
{
public:
    static omni::core::ObjectPtr<PopulateChickenPotPiePass> create(omni::core::ObjectParam<IGraphBuilder> builder) noexcept
    {
        return omni::core::steal(new PopulateChickenPotPiePass(builder.get()));
    }

protected:
    PopulateChickenPotPiePass(IGraphBuilder*) noexcept
    {
    }

    void run_abi(IGraphBuilder* builder, INode* node) noexcept override
    {
        // a common pattern when constructing the execution graph is for population passes to want to access the
        // authoring data associated with a defintion.  to do this, a custom interface which understands the authoring
        // layer's data is used.  here we see if the currently attached definition understands retrieving BakedGood
        // objects.  if so, we get a pointer to the part of the defintion that implements the retrival interface.
        auto bakedGoodGetter = omni::graph::exec::unstable::cast<IPrivateBakedGoodGetter>(node->getDef());
        if (!bakedGoodGetter)
        {
            // either the node or def matched the name we're looking for, but the def doesn't implement our private
            // interface to access the baked good. so, this isn't a def we can populate. bail.
            return; // LCOV_EXCL_LINE
        }

        // we can now grab the baked good from the definition and use it to create a better definition for chicken pot
        // pie
        const BakedGood& bakedGood = bakedGoodGetter->getBakedGood();
        builder->setNodeGraphDef(
            node,
            ChickenPotPieGraphDef::create(builder, bakedGood)
        );
    }
};

Population passes are registered via OMNI_GRAPH_EXEC_REGISTER_POPULATE_PASS(). The registration processes requires supplying a name the pass will match. In this example, the name to match when registering PopulateChickenPotPiePass is “node.bakedGood.prepare.chickenPotPie”.

Passes are run by the application’s PassPipeline. The default PassPipeline will match the population pass against the node’s name. If a node’s name does not match, the PassPipeline will check if a graph definition is attached to the node and see if the graph definition’s name matches. For more details on how passes are applied, see Pass Concepts.

Private Interfaces#

Above, the population pass first checks if the given node’s definition implements the IPrivateBakedGoodGetter interface. That interface is defined as follows:

Listing 45 The IPrivateBakedGoodGetter allows the bakery library to safely access private implementation details via EF.#
class IPrivateBakedGoodGetter : public omni::core::Inherits<IBase, OMNI_TYPE_ID("example.IPrivateBakedGoodGetter")>
{
public:
    virtual const BakedGood& getBakedGood() const noexcept = 0;
};

IPrivateBakedGoodGetter is an example of a private interface. Private interfaces are often used in graph construction and graph execution to safely access non-public implementation details.

To understand the need of private interfaces, consider what the run_abi() method is doing in Listing 44. The purpose of the method is to replace the generic “prepare” graph definition with a higher-fidelity graph definition specific to the preparation of chicken pot pie. In order to build that new definition, parameters in the chicken pot pie’s BakedGood object are needed. Therein lies the problem: EF has no concept of a “baked good”. The population pass is only given a pointer to an INode EF interface. With that pointer, the pass is able to get yet another EF interface, IDef. Neither of these interfaces have a clue what a BakedGood is. So, how does one go about getting a BakedGood from an INode?

The answer lies in the type casting mechanism Omniverse Native Interfaces provides. The idea is simple, when creating a definition (e.g. PrepareBakedGoodGraphDef or ChickenPotPieGraphDef), do the following:

To demonstrate the latter point, consider the code used to define the ChickenPotPieGraphDef class used in the population pass:

Listing 46 EF’s NodeGraphDefT provides a default implementation of INodeGraphDef and INodeGraphDefDebug while allowing users to implement additional interfaces.#
class ChickenPotPieGraphDef : public NodeGraphDefT<INodeGraphDef, INodeGraphDefDebug, IPrivateBakedGoodGetter>

NodeGraphDefT is an implementation of INodeGraphDef and INodeGraphDefDebug. NodeGraphDefT’s template arguments allow the developer to specify all of the interfaces the subclass will implement, including interfaces EF has no notion about (e.g. IPrivateBakedGoodGetter). Recall, much like ChickenPotPieGraphDef, PrepareBakedGoodGraphDef’s implementation also inherits from NodeGraphDefT and specifies IPrivateBakedGoodGetter as a template argument.

To access the BakedGood from the given INode, the population pass calls omni::graph::exec::unstable::cast() on the node’s definition. If the definition implements IPrivateBakedGoodGetter, a valid pointer is returned, on which getBakedGood() can be called. With the BakedGood in hand, it can be used to create the new ChickenPotPieGraphDef graph definition, which the builder attaches to the node, replacing the generic graph definition.

Why Not Use dynamic_cast?#

A valid question that may arise from the code above is, “Why not use dynamic_cast?” There are two things to note about dynamic_cast:

  • dynamic_cast is not ABI-safe. Said differently, different compilers may choose to implement its ABI differently.

  • dynamic_cast relies on runtime type information (RTTI). When compiling C++, RTTI is an optional feature that can be disabled.

In Listing 44, run_abi()’s’ INode pointer (and its attached IDef) may point to an object implemented by a DLL different than the DLL that implements the population pass. That means if dynamic_cast is called on the pointer, the compiler will assume the pointer utilizes its dynamic_cast ABI contract. However, since the pointer is from another DLL, possibly compiled with a different ABI and compiler settings, that assumption may be bad, leading to undefined behavior.

In contrast to dynamic_cast, omni::graph::exec::unstable::cast() is ABI-safe due to its use of Omniverse Native Interfaces (ONI). ONI defines iron-clad, ABI-safe contracts that work across different compiler tool chains. Calling omni::graph::exec::unstable::cast() on an ONI object (e.g. IDef) has predictable behavior regardless of the compiler and compiler settings used to compile the object.

Private vs. Public Interfaces#

ONI objects are designed to be ABI-safe. However, it is clear that IPrivateBakedGoodGetter is not ABI-safe. getBakedGood() returns a reference, which is not allowed by ONI. Furthermore, the returned object, BakedGood, also is not ABI-safe due to its use of complex C++ objects like std::string and std::vector.

Surprisingly, since IPrivateBakedGoodGetter is a private interface that is defined, implemented, and only used within a single DLL, the interface can violate ONI’s ABI rules because the ABI will be consistent once omni::graph::exec::unstable::cast() returns a valid IPrivateBakedGoodGetter. If IPrivateBakedGoodGetter was able to be implemented by other DLLs, this scheme would not work due the ambiguities of the C++ ABI across DLL borders.

Private interfaces allow developers to safely access private implementation details (e.g. BakedGood) as long as the interfaces are truly private. The example above illustrates a common EF pattern, a library embedding implementation specific data in either nodes or definitions, and then defining passes which cast the generic EF INode and IDef pointers to a private interface to access the private data.

But what if the developer wants to not be limited to accessing this data in a single DLL? What if the developer wants to allow other developers, authoring their own DLLs, to access this data? In the bakery example, such a system would allow anyone to create DLLs that implement passes which can optimize the production of any baked good.

The answer to these questions are public interfaces. Unlike private interfaces, public interfaces are designed to be implemented by many DLLs. As such, public interfaces must abide by ONI’s strict ABI rules.

In the context of the example above, the following public interface can be defined to allow external developers to access baked good information:

Listing 47 An example of replacing the private IPrivateBakedGoodGetter with a public interface. Such an interface allows external developers to access baked good information in novel passes to optimize the bakery.#
class IBakedGood_abi : public omni::core::Inherits<omni::core::IObject, OMNI_TYPE_ID("example.IBakedGood")>
{
protected:
    virtual uint32_t getBakeMinutes_abi() noexcept = 0;

    virtual const char* getName_abi() noexcept = 0;
};

To utilize the public interface, definitions simply need to inherit from it and implement its methods. For example:

Listing 48 This version of PrepareBakedGoodGraphDef is similar to the previous one, but now inherits and implements (not shown) the public inteface IBakedGood rather than the private IPrivateBakedGoodGetter. IBakedGood is an API class generated by the omni.bind tool which wraps the raw IBakedGood_abi into a more friendly C++ class.#
class PrepareBakedGoodGraphDef : public NodeGraphDefT<INodeGraphDef, INodeGraphDefDebug, IBakedGood>

Using public interfaces is often more work, but unlocks the ability for external developers to improve and extend a libary’s execution graph. When deciding whether to use a public or private interface, consult the following flowchart.

flowchart TD S[Start] external{{Will external devs require data stored in your library to extend and improve your part of the execution graph?}} data{{Do you need private data for graph construction or execution?}} public[Create a public interface.] private[Create a private interface.] none[No interface is needed.] S --> external external -- Yes --> public external -- No --> data data -- Yes --> private data -- No --> none

Figure 22 Flowchart of when to use public or private interfaces.#

Back to the Population Pass#

After PopulateChickenPotPiePass runs and replaces node node.bakedGood.prepare.chickenPotPie’s generic graph definition with a new ChickenPotPieGraphDef, the bakery’s definition is as follows:

flowchart LR 00000261A3F90D80(( )) 00000261A3F90D80-->00000261A3F90880 00000261A3F90D80-->00000261A3F91320 00000261A3F90D80-->00000261A3F913C0 00000261A3F90D80-->00000261A3F90600 00000261A3F90880(node.bakery.The Pie Hut.preHeatOven) 00000261A3F90880-.->00000261A3DC4290 00000261A3F90880-->00000261A3F90E20 00000261A3F90880-->00000261A3F90F60 00000261A3F90880-->00000261A3F90740 00000261A3F91320(node.bakedGood.prepare.applePie) 00000261A3F91320-.->00000261A3D767A0 00000261A3F91320-->00000261A3F90E20 00000261A3F913C0(node.bakedGood.prepare.chickenPotPie) 00000261A3F913C0-.->00000261A3D76CE0 00000261A3F913C0-->00000261A3F90F60 00000261A3F90600(node.bakedGood.prepare.peachPie) 00000261A3F90600-.->00000261A3D77160 00000261A3F90600-->00000261A3F90740 00000261A3F90E20(node.bakedGood.bake.applePie) 00000261A3F90E20-.->00000261A3CA6FE0 00000261A3F90E20-->00000261A3F906A0 00000261A3F90E20-->00000261A3F90CE0 00000261A3F90F60(node.bakedGood.bake.chickenPotPie) 00000261A3F90F60-.->00000261A3CA7100 00000261A3F90F60-->00000261A3F906A0 00000261A3F90F60-->00000261A3F90CE0 00000261A3F90740(node.bakedGood.bake.peachPie) 00000261A3F90740-.->00000261A3CA5570 00000261A3F90740-->00000261A3F91000 00000261A3F90740-->00000261A3F90CE0 00000261A3F906A0(node.bakedGood.ship.Tracy) 00000261A3F906A0-.->00000261A3DC44C0 00000261A3F90CE0(node.bakery.The Pie Hut.turnOffOven) 00000261A3F90CE0-.->00000261A3DC3610 00000261A3F91000(node.bakedGood.ship.Kai) 00000261A3F91000-.->00000261A3DC34D0 00000261A3CA5570{{def.bakedGood.bake}} 00000261A3CA6FE0{{def.bakedGood.bake}} 00000261A3CA7100{{def.bakedGood.bake}} subgraph 00000261A3D767A0[def.bakedGood.prepare] direction LR style 00000261A3D767A0 fill:#FAFAFA,stroke:#777777 00000261A3F909C0(( )) 00000261A3F909C0-->00000261A3F90A60 00000261A3F90A60(node.bakedGood.gather.applePie) 00000261A3F90A60-.->00000261A3CA73D0 00000261A3F90A60-->00000261A3F91140 00000261A3F91140(node.bakedGood.assemble.applePie) 00000261A3F91140-.->00000261A3CA6260 end 00000261A3CA6260{{def.bakedGood.assemble}} 00000261A3CA73D0{{def.bakedGood.gatherIngredients}} subgraph 00000261A3D76CE0[def.bakedGood.pie] direction LR style 00000261A3D76CE0 fill:#FAFAFA,stroke:#777777 00000261A3FB08A0(( )) 00000261A3FB08A0-->00000261A3FB0080 00000261A3FB08A0-->00000261A3FAF220 00000261A3FB08A0-->00000261A3FAFCC0 00000261A3FB0080(node.pie.chop.carrots) 00000261A3FB0080-.->00000261A3CA5CC0 00000261A3FB0080-->00000261A3FAFC20 00000261A3FAF220(node.pie.makeCrust.chickenPotPie) 00000261A3FAF220-.->00000261A3CA5F00 00000261A3FAF220-->00000261A3FAFC20 00000261A3FAFCC0(node.pie.cook.chicken) 00000261A3FAFCC0-.->00000261A3CA6020 00000261A3FAFCC0-->00000261A3FAFC20 00000261A3FAFC20(node.pie.assemble.chickenPotPie) 00000261A3FAFC20-.->00000261A1508650 end 00000261A1508650{{def.bakedGood.assemble}} 00000261A3CA5CC0{{def.bakedGood.chop}} 00000261A3CA5F00{{def.bakedGood.makeCrust}} 00000261A3CA6020{{def.bakedGood.cook}} subgraph 00000261A3D77160[def.bakedGood.prepare] direction LR style 00000261A3D77160 fill:#FAFAFA,stroke:#777777 00000261A3F911E0(( )) 00000261A3F911E0-->00000261A3F904C0 00000261A3F904C0(node.bakedGood.gather.peachPie) 00000261A3F904C0-.->00000261A3CA6BF0 00000261A3F904C0-->00000261A3F90560 00000261A3F90560(node.bakedGood.assemble.peachPie) 00000261A3F90560-.->00000261A3CA53C0 end 00000261A3CA53C0{{def.bakedGood.assemble}} 00000261A3CA6BF0{{def.bakedGood.gatherIngredients}} 00000261A3DC34D0{{def.order.ship}} 00000261A3DC3610{{def.oven.turnOff}} 00000261A3DC4290{{def.oven.preHeat}} 00000261A3DC44C0{{def.order.ship}}

Figure 23 The execution graph after the PopulateChickenPotPiePass runs.#

Above, you can see node.bakedGood.prepare.chickenPotPie now points to a new graph definition which performs task such as preparing the crust and cooking the chicken in parallel.

Populating Graph Definitions#

As mentioned earlier, population passes run on either matching node names or matching graph definition names. You are encouraged to inspect the names used in Figure 23. There, node names are fairly specific. For example, node.bakedGood.prepare.chickenPotPie rather than node.bakedGood.prepare. Definitions, on the other hand, are generic. For example, def.bakedGood.prepare instead of def.bakedGood.prepare.applePie. This naming scheme allows for a clever use of the rules for population passes.

Earlier, you saw a population pass that optimized chicken pot pie orders by matching node names. Here, a new pass is created: PopulatePiePass:

Listing 49 PopulatePiePass finds definitions for pies and replaces the definition with a new definition optimized for the baking of pies.#
class PopulatePiePass : public omni::graph::exec::unstable::Implements<IPopulatePass>
{
public:
    static omni::core::ObjectPtr<PopulatePiePass> create(omni::core::ObjectParam<IGraphBuilder> builder) noexcept
    {
        return omni::core::steal(new PopulatePiePass(builder.get()));
    }

protected:
    PopulatePiePass(IGraphBuilder*) noexcept
    {
    }

    void run_abi(IGraphBuilder* builder, INode* node) noexcept override
    {
        auto bakedGoodGetter = omni::graph::exec::unstable::cast<IPrivateBakedGoodGetter>(node->getDef());
        if (!bakedGoodGetter)
        {
            // either the node or def matches the name we're looking for, but the def doesn't implement our private
            // interface to access the baked good, so this isn't a def we can populate. bail.
            return; // LCOV_EXCL_LINE
        }

        const BakedGood& bakedGood = bakedGoodGetter->getBakedGood();

        // if the baked good ends with "Pie" attach a custom def that knows how to bake pies
        if (!omni::extras::endsWith(bakedGood.name, "Pie"))
        {
            // this baked good isn't a pie. do nothing.
            return; // LCOV_EXCL_LINE
        }

        builder->setNodeGraphDef(
            node,
            PieGraphDef::create(builder, bakedGood)
        );
    }
};

PopulatePiePass’s purpose is to better define the process of baking pies. This is achieved by registering PopulatePiePass with a matching name of “def.bakedGood.prepare”. Any graph definition matching “def.bakedGood.prepare” with be given to PopulatePiePass. Above, PopulatePiePass’s run_abi() method first checks if the currently attached definition can provide the associated BakedGood. If so, the name of the baked good is checked. If the name of the baked good ends with “Pie” the node’s definition is replaced with a new PieGraphDef, which is a graph definition that better describes the preparation of pies. The resulting bakery graph definition is as follows:

flowchart LR 00000261A3F91320(( )) 00000261A3F91320-->00000261A3F91140 00000261A3F91320-->00000261A3F90560 00000261A3F91320-->00000261A3F91000 00000261A3F91320-->00000261A3F909C0 00000261A3F91140(node.bakery.The Pie Hut.preHeatOven) 00000261A3F91140-.->00000261A3DC37F0 00000261A3F91140-->00000261A3F907E0 00000261A3F91140-->00000261A3F911E0 00000261A3F91140-->00000261A3F90B00 00000261A3F90560(node.bakedGood.prepare.applePie) 00000261A3F90560-.->00000261A3D76E60 00000261A3F90560-->00000261A3F907E0 00000261A3F91000(node.bakedGood.prepare.chickenPotPie) 00000261A3F91000-.->00000261A3D767A0 00000261A3F91000-->00000261A3F911E0 00000261A3F909C0(node.bakedGood.prepare.peachPie) 00000261A3F909C0-.->00000261A3D76B60 00000261A3F909C0-->00000261A3F90B00 00000261A3F907E0(node.bakedGood.bake.applePie) 00000261A3F907E0-.->00000261A3CA6B60 00000261A3F907E0-->00000261A3F90CE0 00000261A3F907E0-->00000261A3F913C0 00000261A3F911E0(node.bakedGood.bake.chickenPotPie) 00000261A3F911E0-.->00000261A3CA5330 00000261A3F911E0-->00000261A3F90CE0 00000261A3F911E0-->00000261A3F913C0 00000261A3F90B00(node.bakedGood.bake.peachPie) 00000261A3F90B00-.->00000261A3CA5600 00000261A3F90B00-->00000261A3F90600 00000261A3F90B00-->00000261A3F913C0 00000261A3F90CE0(node.bakedGood.ship.Tracy) 00000261A3F90CE0-.->00000261A3DC3980 00000261A3F913C0(node.bakery.The Pie Hut.turnOffOven) 00000261A3F913C0-.->00000261A3DC3930 00000261A3F90600(node.bakedGood.ship.Kai) 00000261A3F90600-.->00000261A3DC3D40 00000261A3CA5330{{def.bakedGood.bake}} 00000261A3CA5600{{def.bakedGood.bake}} 00000261A3CA6B60{{def.bakedGood.bake}} subgraph 00000261A3D767A0[def.bakedGood.pie] direction LR style 00000261A3D767A0 fill:#FAFAFA,stroke:#777777 00000261A3FB0A80(( )) 00000261A3FB0A80-->00000261A3FB04E0 00000261A3FB0A80-->00000261A3FB0580 00000261A3FB0A80-->00000261A3FB0C60 00000261A3FB04E0(node.pie.chop.carrots) 00000261A3FB04E0-.->00000261A3CA6A40 00000261A3FB04E0-->00000261A3FB0EE0 00000261A3FB0580(node.pie.makeCrust.chickenPotPie) 00000261A3FB0580-.->00000261A3CA7100 00000261A3FB0580-->00000261A3FB0EE0 00000261A3FB0C60(node.pie.cook.chicken) 00000261A3FB0C60-.->00000261A1508650 00000261A3FB0C60-->00000261A3FB0EE0 00000261A3FB0EE0(node.pie.assemble.chickenPotPie) 00000261A3FB0EE0-.->00000261A0496AA0 end 00000261A0496AA0{{def.bakedGood.assemble}} 00000261A1508650{{def.bakedGood.cook}} 00000261A3CA6A40{{def.bakedGood.chop}} 00000261A3CA7100{{def.bakedGood.makeCrust}} subgraph 00000261A3D76B60[def.bakedGood.pie] direction LR style 00000261A3D76B60 fill:#FAFAFA,stroke:#777777 00000261A3FB0300(( )) 00000261A3FB0300-->00000261A3FAFB80 00000261A3FB0300-->00000261A3FB0620 00000261A3FAFB80(node.pie.chop.peachPie) 00000261A3FAFB80-.->00000261A3CA52A0 00000261A3FAFB80-->00000261A3FB09E0 00000261A3FB0620(node.pie.makeCrust.peachPie) 00000261A3FB0620-.->00000261A3CA57B0 00000261A3FB0620-->00000261A3FB09E0 00000261A3FB09E0(node.pie.assemble.peachPie) 00000261A3FB09E0-.->00000261A3FB40D0 end 00000261A3CA52A0{{def.bakedGood.chop}} 00000261A3CA57B0{{def.bakedGood.makeCrust}} 00000261A3FB40D0{{def.bakedGood.assemble}} subgraph 00000261A3D76E60[def.bakedGood.pie] direction LR style 00000261A3D76E60 fill:#FAFAFA,stroke:#777777 00000261A3FAF9A0(( )) 00000261A3FAF9A0-->00000261A3FAF4A0 00000261A3FAF9A0-->00000261A3FAFAE0 00000261A3FAF4A0(node.pie.chop.applePie) 00000261A3FAF4A0-.->00000261A3CA5F00 00000261A3FAF4A0-->00000261A3FAF2C0 00000261A3FAFAE0(node.pie.makeCrust.applePie) 00000261A3FAFAE0-.->00000261A3CA60B0 00000261A3FAFAE0-->00000261A3FAF2C0 00000261A3FAF2C0(node.pie.assemble.applePie) 00000261A3FAF2C0-.->00000261A3CA6260 end 00000261A3CA5F00{{def.bakedGood.chop}} 00000261A3CA60B0{{def.bakedGood.makeCrust}} 00000261A3CA6260{{def.bakedGood.assemble}} 00000261A3DC37F0{{def.oven.preHeat}} 00000261A3DC3930{{def.oven.turnOff}} 00000261A3DC3980{{def.order.ship}} 00000261A3DC3D40{{def.order.ship}}

Figure 24 The execution graph after the PopulatePiePass runs.#

It is important to note that EF’s default pass pipeline only matches population passes with either node names or graph definition names. Opaque definition names are not matched.

The example above shows that knowing the rules of the application’s pass pipeline can help EF developers name their nodes and definitions in such as way to make more effective use of passes.

Conclusion#

The bakery example is trivial in nature, but shows several of the patterns and concepts found in the wild when using the Execution Framework. An inspection of OmniGraph’s use of EF will reveal the use of all of the patterns outlined above.

A full source listing for the example can be found at source/extensions/omni.graph.exec/tests.cpp/TestBakeryDocs.cpp.