Omniverse Documentation System

Unified tool for generating Omniverse documentation.

Features at a Glance

The Omniverse documentation system supports the following:

  • The generation of uniform, on-brand documentation with minimal setup.

  • A documentation tool chain that is easy to integrate and update.

  • Extracting C++ and Python API documentation into the final docs as a first-class citizen. API documentation and overview documentation are able to easily reference each other.

  • Easily embed rich media such as equations, videos, images, tables, and graphs in a WYSIWYG markup.

  • Built-in search (including API documentation).

  • Building multiple projects in a single repository.

  • Building multiple “flavors/configs” of documentation (e.g. “internal”, “public”). Documents and chunks of a document can be conditionally included into the output based on the “flavor”.

  • Easy and predictable linking of documentation to other projects.

  • Easy publishing of documentation for multiple purposes (e.g. review, internally, externally).

  • Fast iteration paths that avoid rebuilding unchanged documentation.

  • Built on popular Open Source technologies like Markdown, reStructuredText, Sphinx, Doxygen, and Docstrings.

Initial Setup

repo_docs is a part of the repo suite of tools. In order to use repo_docs, you must first enable the repo suite in your repository. Information on setting up the repo suite can be found at:

https://gitlab-master.nvidia.com/omniverse/repo/repo_minimal

Once the repo suite is installed, add repo_docs as an available tool by adding the following to deps/repo-deps.packman.xml:

  <dependency name="repo_docs" linkPath="../_repo/deps/repo_docs">
    <package name="repo_docs" version="0.10.5" />
  </dependency>

Above, 0.10.5 should be the version number located in this repository’s VERSION file.

Next, enable and configure the tool in your repository’s repo.toml file.

[repo_docs]

enabled = true

# the name of you project. should be a single word. used to name files.
project = "carbonite"

# a slightly more descriptive name for your project. user visible. keep it short.
#
# defaults to the value specified in 'project'
name = "Carbonite SDK"

# first year in your project's copyright.
copyright_start = 2018

Finally, create an index.rst in the root of your repository. It’s recommended that all other overview documentation is placed in docs/.

All settings in the remainder in this document, unless otherwise noted, are assumed to appear under the [repo_docs] table of repo.toml.

Enabling C++ Documentation

Embeddedd C++ documentation can be extracted into your documentation with the following settings:

# list of files from which to extract documentation.  if a directory is specified,
# it will be recursively searched.
#
# paths are relative to ${docs_root}.
#
# defaults to an empty list.
doxygen_input = [
    "include/carb/Framework.h",
    "include/carb/Interface.h",
    "include/carb/PluginCoreUtils.h",
    "include/carb/PluginUtils.h",
    "include/carb/StartupUtils.h",
    "source/examples/example.doxygen/ExampleDoxygen.h",
]

# Using wildcards is also supported in `doxygen_input`. Assuming there are no other `.h` files
# in the `include/carb` directory, the above may also be specified as:
# doxygen_input = [
#     "include/carb/*.h",
#     "source/examples/example.doxygen/ExampleDoxygen.h",
# ]

# doxygen allows the creation of custom commands to ease in the documentation process.
# for example, this adds a @carb_framework_overview command which creates a link
# to a reStructuredText document. defaults to an empty list.
#
# more information on the format can be found at:
#     https://www.doxygen.nl/manual/config.html#cfg_aliases
doxygen_aliases = [
    "carb_framework_overview=\\rstref{Framework Overview <carb_framework>}",
]

# doxygen sometimes gets confused by macros. the array below allows the user to
# tell doxygen how to expand a macro. defaults to an empty list.
#
# more information on the format can be found at:
#     https://www.doxygen.nl/manual/config.html#cfg_predefined
doxygen_predefined = [
    "OMNI_EXPORT=",
    "M_PI=3",
]

# title of the page which will contain an overview of the C++ documentation.
#
# defaults to "${name} API"
api_title = "World's Best API"

Enabling Python Documentation

The documentation system is able to load .py and .pyd files, inspect each function/class, and extract their docstrings. To add the directories containing the .py and .pyd files to the tools search path, set python_paths as follows:

# path to .py and .pyd files whose docstring should be extracted.
#
# relative paths are relative to ${docs_root}. Wilcards are supported.
#
# defaults to empty list
python_paths = [
    "${root}/_build/${platform_target}/${config}/bindings-python"
]

Binary python modules can require other shared libraries to be imported. Use library_paths to add paths to PATH (windows) / LD_LIBRARY_PATH (linux):

# paths to be added to `PATH` (windows) / `LD_LIBRARY_PATH` (linux)
#
# relative paths are relative to ${docs_root}. Wilcards are supported.
#
# defaults to empty list
library_paths = [
    "${root}/_build/${platform_target}/${config}/libs/*"
]

Note that repo_docs has already configured Sphinx with popular modules such as sphinx.ext.autodoc and sphinx.ext.napoleon for generating Python API documentation. All that is left to do is add one or more rst files that structure the Python documentation and provide Sphinx with directives for generating it.

Python API Example
==================

example.widgets
---------------
.. automodule:: example.widgets
    :platform: Windows-x86_64, Linux-x86_64
    :members:
    :undoc-members:

Configuring Sphinx

The documentation system uses Sphinx to read .rst and .md files to generate HTML. Sphinx will search ${docs_root} for .rst and .md files. A subset of .rst and .md files can be excluded with sphinx_exclude_patterns:

# paths relative to ${docs_root} to be excluded from the documentation system.
#
# if a directory is given, its contents will be recursively excluded.
#
# defaults to an empty list
sphinx_exclude_patterns = [
    "docs/rfcs",
    "source",
    "include",
    "tools",
    "README.md",
]

To force include a subfolder after its parent folder was excluded with sphinx_exclude_patterns, use sphinx_dont_exclude_patterns:

sphinx_exclude_patterns = [
    "_build"
]

sphinx_dont_exclude_patterns = [
    "_build/generated/cool/page"
]

Note, sphinx_dont_exclude_patterns is unable to match paths that contained a wildcard (e.g. “*”) in sphinx_exclude_patterns.

Additional Settings

The following optional settings can be used to further customize the documentation output.

[repo_docs]

# directory in which tools should be run and to which all paths are relative
# unless otherwise noted).
#
# if the supplied path is not absolute, it is relative to ${root}.
#
# defaults to ${root}, which is the repository's root.
docs_root = "my-sub-project"

# directory where output is placed.
#
# if the supplied path is not absolute, it is relative to ${docs_root}.
#
# defaults to ${root}/_build/docs
docs_dst = "${root}/_build/more-docs"

# when using .md files as input, any relative file link will be prepended with the
# following url. defaults to "".
repo_url = "https://gitlab-master.nvidia.com/omniverse/carbonite/-/blob/master"

# if you want to be 😎 you need a 'gram, 'tube, github, and the like. repo_docs has you
# covered. use ${social_media_set} to populate the docs with a set of icons to your team's
# social media sites.
#
# you can override individual site urls with ${social_media} (see below).
#
# known values are: "omniverse"
#
# defaults to "omniverse"
#
# to disable this feature, set to ""
social_media_set = "omniverse"

# overrides and extends values from ${social_media_set}.
#
# known values are: "gitlab", "github", "youtube", "twitter", "instagram", "www",
#                   "linkedin", "twitch", "facebook", "pinterest", "bitbucket"
#
# if the first item of a pair is not a known site, its value will be used as the content
# of the anchor
#
# defaults to []
social_media = [
    # defines a gitlab icon
    [ "gitlab", "https://gitlab-master.nvidia.com/omniverse/repo/repo_docs" ],

    # overrides the default youtube button for "omniverse"
    [ "youtube", "https://www.youtube.com/watch?v=mN717odnm5Q" ],

    # defines a link (with an E icon) to etsy
    #
    # the first value is a unicode value defined in the following font face:
    #
    #   https://fontawesome.com/v4.7/icons/
    [ "&#xf2d7", "https://www.etsy.com/search?q=nvidia" ],
]

# path to the python executable.
#
# if the path is not absolute, it is considered relative to ${docs_root}.
#
# defaults to using the python executable used by packman.
python_path = "/bin/python"

# sets the version of doxygen to use.
#
# defaults to a supported version of doxygen.
doxygen_version = "1.8.18-${platform}"

# sets the version of sphinx to use.
#
# defaults to a supported version of sphinx.
sphinx_version = "3.5.3.4-py3.7-${platform}"

# some warnings within doxygen just can't be fixed. here, the user can define a
# list of regexes to run on the output of doxygen.  if a line in doxygen's output
# matches, it will be removed.  this helps reduce visual clutter.
doxygen_output_exclude_regexes = [
  "^warning: Can't find reference to F.*oo"
]

# doxygen's output directory.
#
# relative paths are relative to ${docs_dst}.
#
# defaults to ${docs_dst}/${project}/latest${build}/doxygen.
doxygen_dst = "/tmp"

# copyright owner. defaults to "NVIDIA". NOTE: this setting is marked for deprecation,
# prefer to use the more powerful `copyright` and `author` settings instead. See
# How the Omniverse Documentation System Works for details.
copyright_owner = "Nathan Cournia"

# the version of the documentation is chosen from the first defined source below:
#
#   - CI_OPEN_MERGE_REQUESTS in the environment (defined by GitLab CI)
#
#   - BUILD_NUMBER in the environment (defined by TeamCity)
#
#   - this variable
#
#   - ${docs_root}/VERSION file
#
#   - ${root}/VERSION file
#
# the current build (e.g. internal, public) is implicitly appended to the version
# number (e.g. ${version}${build})
version = "1.0"

# logo that appears in the upper-left corner of the docs.
#
# relative paths are relative to ${docs_root}
#
# defaults to the omniverse logo
logo = "docs/images/logo.png"

# logo that appears in the web browser's title/tab bar
#
# relative paths are relative to ${docs_root}
#
# defaults to the omniverse logo
favicon = "docs/images/favicon.ico"

# enable a darkmode theme switcher
# disabled by default until better validated
enable_dark_theme_switcher = true

# list of documentation bundles this repo depends upon
#
# each entry in the array is a pair. the first item in the pair is a name you can use
# to refer to the dependency.  for example, in your .rst, you can reference repo_docs'
# main page with:
#
#  :ref:`repo_docs:omni_doc_system`
#
# above, "repo_doc" is the name of the dependency and "omni_doc_system" is the reference
# name.
#
# the second item in the pair is a directory containing the repo's "objects.inv" and
# "VERSION".
#
# note, ${build} can be used to specify the current build flavor (e.g. "-internal")
deps = [
    [ "repo_docs", "${root}/_repo/deps/repo_docs/_build/docs/repo_docs/latest"],
    [ "carb",      "${root}/_repo/deps/carbonite/_build/docs/carbonite/latest{build}"],
]

# extra text to be placed in sphinx's conf.py file. this can be used to greatly change
# sphinx's behavior and should be used sparingly as it may interfere with future releases.
#
# defaults to ""
sphinx_conf_py_extra = """
    import os
    import sys
    print("hi")
"""

# sphinx_conf_py_extra accepts a list of event injections for attaching to the Sphinx
# event system for debugging and enhancement purposes.
# https://www.sphinx-doc.org/en/master/extdev/appapi.html#sphinx-core-events
# e.g:
sphinx_conf_py_extra = """
def autodoc_skip_member_handler(app, what, name obj, skip, options):
    exclusions = ("test", "tests")
    exclude = name in exclusions
    return exclude or skip
sphinx_event_handlers += [
    ("autodoc-skip-member", autodoc_skip_member_handler)
]
"""

# extra text to be placed into doxygen's config file. this can greatly change doxygen's
# behavior and should be used sparingly as it may interfere with future releases.
#
# defaults to ""
doxygen_conf_extra = """
   SEARCH_INCLUDES = YES
   INCLUDE_PATH = _build/target-deps/carb_sdk_plugins/include
"""

Running the Tool

To run the tool:

repo docs

By default, the previous built output is removed and rebuilt. See:

repo docs -h

for a full list of flags that can be passed to the tool. Clever use of the tool’s flag can greatly decrease iteration time.

Decreasing Iteration Time

By default, the following command cleans the output directory and then performs a full build:

repo docs

However, once a full build has been produced, stages of the build can be disabled (i.e. the old artifacts can be reused). For example, if you only want to see changes in the .rst files you’re authoring, you can avoid the clean, doxygen, and exhale stages with:

repo docs --stage sphinx

Likewise, if you want to clean the current output and just run Doxygen:

repo docs --stage clean --stage doxygen

See the user guide for more details on how stages work in the tool.

Updating the Tool

Updating the tool is simple. First ensure that you haven’t “source linked” to a locally cloned copy of the tool:

# you don't have to run this unless you ran `repo source link repo_docs ...` in the past
repo source unlink repo_docs

Now update the tool:

# this updates deps/repo-deps.packman.xml
repo update repo_docs

# commit
git commit -a -m "Updated repo_docs to X.X.X"

Development

Note that this project includes submodules.

Either clone with the --recurse-submodules option, or run git submodule update --init.

License

This tool is licensed under a standard NVIDIA license.

This project will download and install additional third-party open source software projects. Review the license terms of these open source projects before use.

See the License Summary page for more information.

The following projects are vendored by repo_docs:

  • exhale - BSD 3-Clause License

  • Lunr - MIT License

In addition to those pip packages listed in the requirements/*.txt files, the following projects may be installed:

  • Doxygen

  • rclone