Changelog#

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[1.71.1] - 2024-12-05#

fixed poor test assumption, now properly shove tooling deps into the temporary packman cache
Check if caching paths are a relative path to ${ROOT}, and prepend ${ROOT}. Otherwise use provided absolute paths.

[1.71.0] - 2024-11-22#

added repo cache tool for caching repo tools, packman packages, and user declared dependencies
synced scripts/repoman.py and tools/repoman/repoman.py with cache utilization changes
modified repo.sh and repo.bat to set the Packman package cache prior to bootstrapping Packman

[1.70.3] - 2024-11-01#

fix: container_utils will never detect less than one core even if mem_per_core is more than available memory

[1.70.2] - 2024-10-28#

omni_config: fix a typo in cache path on linux when XDG_DATA_HOME is not set

[1.70.1] - 2024-10-18#

fix: detect powershell.exe before attempting to use it

[1.70.0] - 2024-10-14#

add: standard OV data/cache/logs dirs as API and as tokens

[1.69.3] - 2024-10-09#

fix: build number and get_git_branch to use a safe name for git tags in CI

[1.69.2] - 2024-10-08#

Remove platform_target token, set platform_host instead

[1.69.1] - 2024-10-01#

Attempt to use new powershell if available before falling back to old

[1.69.0] - 2024-09-23#

Utilize pip’s user-level cache with a file-lock guard to locally cache the uv python wheel

[1.68.1] - 2024-09-20#

Slight tweak to the --abi argument. If --abi= is passed and a default repo.tokens.abi is set, platform_target_abi token == platform_host rather than abi-declarative.

[1.68.0] - 2024-09-20#

Set platform_target and platform_target_abi tokens.
platform_target == host_platform. Overridden in repo_build if --platform-target flag set.
platform_target_abi == abi-declarative host_platform if abi token also set via --abi flag. Overridden in repo_build if --platform-target flag set.

[1.67.0] - 2024.09-03#

Configuration filtering is now possible against tokens and envvars.
Equality checks can be performed against token and envvars for filtering.
Added abi token that can be set by the –abi CLI arg

[1.66.1] - 2024-08-02#

Fix --set-token and --ci-env. Selective tool loading optimization introduced another parser missing them.

[1.66.0] - 2024-07-31#

container_utils.py log.error -> log.info to avoid confusing error-level log messages on console
container_utils.get_available_cpu_count supports a Windows path via powershell

[1.65.2] - 2024-07-30#

Remove gtl remote from packman config

[1.65.1] - 2024-07-24#

Added windows-aarch64 platform to PLATFORM_MAPPING

[1.65.0] - 2024-07-24#

Bump uv 0.1.45 -> 0.2.28 to patch out a statically linked zlib CVE

[1.64.2] - 2024-07-19#

OMPE-15376: uv on Windows is not multi-writer safe, guard all uv actions with a filelock until uv is patched

[1.64.1] - 2024-07-09#

make the tools/repoman/repoman.py and scripts/repoman.py match better
make sure both support linked directories (see 1.63.5)

[1.64.0] - 2024-07-17#

remove unused telemetry gathering (was internal only)

[1.63.5] - 2024-07-09#

better support for paths which include linked directories

[1.63.4] - 2024-06-27#

support sorting of OVFlowBuildIdentifier

[1.63.3] - 2024-06-18#

fix: build nubmer and get_git_branch to not replace branch with the tag on Gitlab CI (when tag is available)
fix: sanitize dot (.) in branch name out from build_number generation

[1.63.2] - 2024-06-06#

fix: remove the error message when calling repo.sh via a relative path (e.g., kit/repo.sh in the Kit repo)

[1.63.1] - 2024-06-05#

Added the ${path_sep} token that can be used in repo.toml configuration and replaced with a platform specific path separator

[1.63.0] - 2024-05-30#

OMPE-2613: Change NVDF omni-devplat-metrics endpoints from ES to Opensearch

[1.62.0] - 2024-05-29#

move repotool_min_version_check and raise_on_minimum_failure into version.py for reuse as a check in tool bootstrapping

[1.61.0] - 2024-05-28#

Add argument for repo Python dependency loading to require hashes and not fetch undeclared dependencies.
Provide legacy fallback to pip for py37 and/or uv unsupported platform.

[1.60.2] - 2024-05-23#

More explict typing for container utils

[1.60.1] - 2024-05-24#

fix: repo_man started discovering tool without entry_point after 1.59.0

[1.60.0] - 2024-05-16#

Swap in uv instead of pip for repo Python dependency loading.
repo will now use a multiprocessing safe cache in the OS default cache directory e.g. ~/.cache/uv/
Python dependencies will be symlinked against expanded wheels in the cache.

[1.59.6] - 2024-05-23#

Improved container utils exception handling

[1.59.5] - 2024-05-22#

Fixed a container utils issue where memory_limit could return None instead of 0

[1.59.4] - 2024-05-21#

Fixed another unhandled exception in container utils

[1.59.3] - 2024-05-21#

Fixed an unhandled exception in container utils

[1.59.2] - 2024-05-17#

Fix Gitlab CI build number on release branches.

[1.59.1] - 2024-05-16#

Prevent error if repo.toml does not exist.

[1.59.0] - 2024-05-14#

Startup optimization: only import tool that runs

[1.58.5] - 2024-05-10#

Add container utils to collect container data to be accessed by other tools

[1.58.4] - 2024-04-29#

Enable repo_poetry to point at a different pyproject.toml than the one in ${root}
Sync sentry-sdk version between pyproject.toml and vendor.py

[1.58.3] - 2024-04-25#

Add a –wait-for-debugger option to allow for repo tool script debugging

[1.58.2] - 2024-04-04#

Remove old DEBUG_MODE from update.py

[1.58.1] - 2024-04-24#

Quotes around system called file ops

[1.58.0] - 2024-04-24#

Removed repo_build_docs tool as it has been superseded by repo_docs.

[1.57.13] - 2024-04-23#

Addressed many small clarity and security items

[1.57.12] - 2024-04-12#

Fixes recursion error when an invalid tool is called

[1.57.11] - 2024-04-11#

When a bad argument is passed to a tool, usage for that tool is now displayed

[1.57.10] - 2024-04-10#

License as MIT

[1.57.9] - 2024-04-09#

Change exception message output for external users

[1.57.8] - 2024-03-27#

Usage provides additional information about tools

[1.57.7] - 2024-03-27#

Add mermaid diagram to docs for sample invocation to execution flow

[1.57.6] - 2024-03-27#

make exit code reported via telemetry match actual exit code returned by repo_man

[1.57.5] - 2024-03-27#

Disable sentry and telemetry for external users

[1.57.4] - 2024-03-27#

Add missing base exception KeyboardInterrupt handling in run_process functions.

[1.57.3] - 2024-03-26#

Add Exception handling in run_process functions, will catch the decoding errors that can happen when the process is killed.

[1.57.2] - 2024-03-25#

chore: add poetry licensing verbiage to readme

[1.57.1] - 2024-03-15#

lock poetry deps to urllib-1.26.18 for CVE-2023-43804

[1.57.0] - 2024-03-14#

Allow requirements file locations to be overridden by repo tools.

[1.56.5] - 2024-03-13#

Prevent the teamcity log publishing from running outside of teamcity.

[1.56.4] - 2024-03-13#

Conform all current copyrights to use SPDX
Introduce and supply standard NVIDIA license into final package
Update packman, update repo deps
Fix a repo_test upgrade bug and e2e test failure residual from pytest migration

[1.56.3] - 2024-03-11#

Add a utility function is_git_tracked() and relevant unit test

[1.56.2] - 2024-03-08#

add requests package for nicer HTTP calls

[1.56.1] - 2024-03-07#

clean_folders util now normalizes folder paths

[1.56.0] - 2024-02-28#

nv_env.in_nvidia / nvenv.in_corpnet functions for helping to avoid wasting time on retryerrors etc

[1.55.2] - 2024-02-27#

Move all tests to pytest

[1.55.1] - 2024-02-21#

Reduce gitlab ci_message-ing

[1.55.0] - 2024-02-20#

Added omni.repo.man.deps.install_dependencies to replace by the former omni.repo.deps.verify_vendor_integrity that involved a heavy hashing verification.
Added hash_directory arg to omni.repo.man.deps.validate_dependencies so users can opt-in for dependency verification. Defaults to false.

[1.54.0] - 2024-02-16#

Quick refactor of repo_poetry to instead ingest tools/poetry/pyproject.toml
- This converts poetry from a static vendor-requirements.txt in the downstream project directory structure to a packaged pyproject.toml.

[1.53.2] - 2024-02-16#

Propagate block disabling behavior to non-generic blocks
Uniqify gitlab message blocks to decouple section-linkage in log window

[1.53.1] - 2024-02-15#

ci_message/block can now filter by ci platform

[1.53.0] - 2024-02-12#

Add config arg repo.file_logging_max_bytes config option for controlling size of RotatingFileHandler logfile.
Update exception_utils._push_logfile_to_teamcity() to push logfile + optional rotated file to TeamCity.

[1.52.0] - 2024-02-09#

Add PushEnv, a commonly reused OS environ context manager, to utils

[1.51.10] - 2024-02-08#

Make sure the py.typed marker actually makes it into the package

[1.51.9] - 2024-02-08#

Add py.typed marker for PEP-561 compliance

[1.51.8] - 2024-02-06#

Store original sys.argv on the options for use in other repo_tools.

[1.51.7] - 2024-02-05#

output get_git_branch, get_git_hash git call errors as info, it is callers responsibility to handle None return value

[1.51.6] - 2024-01-24#

Fixed tests/pytest/test_e2e.py
Default test suite switch to ‘pytest’ suite, as it contains for more tests than ‘unittest’ suite

[1.51.5] - 2024-01-31#

allow packman lookup in find_new_version to be optimized with a version string

[1.51.4] - 2024-01-25#

Fix bad type error on Python 3.7 in configuration classes.

[1.51.3] - 2024-01-23#

avoid deprecated distutils usage

[1.51.2] - 2024-01-19#

build_number uses ci_message and repo_man’s own version is detectable by build_number

[1.51.1] - 2024-01-18#

env-var DISABLE_CI_MESSAGE_BLOCK being truthy will short-circuit ci_message_block

[1.51.0] - 2024-01-17#

Added configuration classes using Pydantic for improved configuration handling.

[1.50.15] - 2024-01-17#

Fix mutable default in dependencies md5 calculation incorrectly invalidating package installs on multiple calls.

[1.50.14] - 2024-01-12#

Fix urllib version to 1.26.16 so that repo test will work again

[1.50.13] - 2024-01-11#

Revert MR !187 that breaks repo update repo.

[1.50.12] - 2024-01-11#

Fix bug where there was a dependency on dateutil from packman. Dependency removed. This will allow usage with packman 7.17.

[1.50.11] - 2024-01-10#

Add missing export for QuietExpectedError.

[1.50.10] - 2024-01-10#

Add QuietExpectedError that will not print anything to the console or file log when raised.

[1.50.9] - 2024-01-03#

Set pip timeout to 60 seconds and retries to 5.

[1.50.8] - 2023-12-20#

Added config arg to repo_packman tool (for release/debug tokens)
Forward repo.tokens to repo_packman tool

[1.50.7] - 2023-12-15#

Don’t open/close ci blocks from a background thread.

[1.50.6] - 2023-12-14#

Swapped the base_delay and growth_rate values for omni.repo.man.deps._retry_rename to correctly backoff.

[1.50.5] - 2023-12-13#

Added omni.repo.man.utils.exponential_backoff_retry decorator for easy exponential retry of functions.
Exponential retry for the move operation in omni.repo.man.deps for moving repo Python deps install folder. This potentially addresses the PermissionError issue encountered on Windows.

[1.50.4] - 2023-12-11#

Changed copy and link time diagnostics to print in milliseconds instead of seconds

[1.50.3] - 2023-12-07#

In repo.sh prefix the exec command with the setter for OMNI_REPO_ROOT rather than using export and polluting the user’s shell.
In repo.bat use SETLOCAL and ENDLOCAL to constrain OMNI_REPO_ROOT to the batch file scope.

[1.50.2] - 2023-12-01#

Use print instead of print_log in deps.py

[1.50.1] - 2023-12-01#

Add an option to bypass rich print in print_log

[1.50.0] - 2023-11-30#

Brief refactor of build_docs.py, build_number.py, cmd_config_overrider.py, configuration.py, debugging.py, and deps.py.
sys.exit and base Exception swapped for either omni.repo.man.exceptions.RepoToolError or built-in Exception subclasses where it makes sense.

[1.49.2] - 2023-11-30#

Sanitize your inputs. The OMNI_REPO_ROOT envvar for repo Python dependency loading included extra quotes and pathlib.Path explodes on py37 but silently fails on py310. Neither behavior is desirable, strip quotes instead.

[1.49.1] - 2023-11-28#

Patch omni.repo.man.deps.pip_install_requirements to emit the stdout/stderr coming out of pip in the event of failure, even if the quiet argument is passed.
Retry omni.repo.man.deps.pip_install_requirements 3 times with 15 second delays to provide a little buffer for our unreliable network.

[1.49.0] - 2023-11-09#

Implement a decorator which can be used to enforce a minimum version of a given repo-tool for function execution with a clear exception. e.g. repotool_min_version(“repo_man”, “1.50.0”)

[1.48.1] - 2023-11-07#

Corrected a bug where logger propagation was set to False in omni.repo.man.exceptions_utils.emit_stack_to_logfile. This caused the helpful exception related messaging to not get pushed to the console leading to confusion and tedium referencing logfiles.

[1.48.0] - 2023-11-03#

Permanently mute repo_telemetry whoami call to avoid error level log messages in Linbuild.
Refactor omni.repo.man.exceptions to extract all helper utilities into omni.repo.man.exceptions_utils to avoid circular dependency loading.

[1.47.2] - 2023-10-31#

Import repo-tool template yml for CI

[1.47.1] - 2023-10-31#

Add repo logs artifacts (CI only)

[1.47.0] - 2023-10-30#

Update .gitlab-ci.yml, add new jobs for more validation.
Fixed tests that were not working on Gitlab CI during merge requests pipelines
Renamed CHANGES.md to CHANGELOG.md
Renamed mytest suite to unittest

[1.46.6] - 2023-10-30#

Emit TeamCity buildProblem status message first to set the build log anchor point, then emit the log messages stating an error has occured.

[1.46.6] - 2023-10-25#

Fix exception name to be correct for any exception
Print repo tools errors to the console

[1.46.5] - 2023-10-18#

Added the exception error message for the general repo on-exit error reporting. This is added context for scenarios where the repo.log logfile does not get saved off, and a stack is not emitted to console.

[1.46.4] - 2023-10-16#

Fix unnecessary cache re-population in the edge case where requirements.txt is more recent than the existing cache, but the regenerated cache is identical to the existing cache.

[1.46.3] - 2023-10-12#

Silence git errors from repo telemetry. If telemetry fails it should not spam the console, it isn’t a critical path tool.

[1.46.2] - 2023-09-26#

Added a file lock around Python dependency operations. Turns out my testing against Kit missed rapid-thrash projects that 100% stomped each other.

[1.46.1] - 2023-09-22#

MacOS is missing realpath. Swap to using pwd -P when setting OMNI_REPO_ROOT envvar.

[1.46.0] - 2023-09-21#

Refactored repo tool Python dependency caching:
- Tool dependencies will now be stored in the path ${root}/_repo/deps/${tool_name}_deps/${platform}/${python_version}/${requirements.txt hash}
- The intention is that the execution of tools with multiple python versions against one repository will not reuse dependencies for another python version.
- This sets up our dependencies for purposeful shared caching as well with rigid validation that shared dependencies have not changed.
- Parallel attempts to pull the dependencies should no longer clash. A tempdir is used and a final fight will occur during the copy with retries, validation, and acceptance if another process wins.
Simplified the entrypoint for Python dependency caching within omni.repo.man.__init__.py

[1.45.0] - 2023-09-19#

Add import_optional_configs to be able to import configs only if exist

[1.44.1] - 2023-09-11#

rich: Document tool settings and support older python runtimes by catching the unprotected rich.print call

[1.44.0] - 2023-09-05#

Refactor on-exit exception handling for repo tools. Current behavior should not change much, but this sets up tooling to start raising precise exceptions rather than exiting non-zero or raising a generic Exception.
New exit code rules:
- exit code 1 == an unexpected error was encountered, e.g. compiler error
- exit code 2 == an expected error was encountered, e.g. a test failed or repo format --verify failed.
omni.repo.man.exceptions contains an exception base class and several subclasses to act as a start.
tracebacks_on_exit has been deprecated as a configuration file option. It in general was not used, and moving forward we should be deprecating our sys.exit(1) calls, thereby nullifying this exit path.

[1.43.2] - 2023-09-04#

fix long dormant bug in OVFlowBuildIdentifier

[1.43.1] - 2023-08-28#

fix typing issue when repo_man used with python-3.7

[1.43.0] - 2023-08-25#

new nvgitlab.py module, new utils for ci_message / ci_message_block to help with teamcity-to-gitlab migrations
moved some ci-independent utils from nvteamcity to utils

[1.42.5] - 2023-08-16#

repo update - don’t match versions with a different build metadata parts, e.g. kit package with .cp39 in the version

[1.42.4] - 2023-08-11#

fix get_git_branch on Gitlab

[1.42.3] - 2023-07-27#

rich: replace_bultin_print default to false, as all print("##teamcity...") statements stop working
rich: leave print_log to use rich.print

[1.42.2] - 2023-07-27#

rich: disable wrapping for a print function
rich: disable custom logger when running on CI

[1.42.1] - 2023-07-27#

Use GITLAB_CI to detect if we run on Gitlab CI - more accurate than using CI_PIPELINE_ID

[1.42.0] - 2023-07-27#

add rich module dependency and integration

[1.41.0] - 2023-07-13#

RepoTelemetry timeout is now customizable at object instantiation. The index version is also customizable at object instantiation.

[1.40.2] - 2023-07-11#

Fix parse_packman_config for packman 7 API

[1.40.1] - 2023-07-07#

Bump nspect version and allow changing it with repo.nspect_version key in config

[1.40.0] - 2023-07-03#

Added support for a global repo config file located at ~/.nvidia-omniverse/config/global_repo.toml
This config file is the second to final layer for our repo configs, and has been documented in docs/configuration.md.
Added support for per-project config file located in ${repo_root}/user.repo.toml
This config file is the final layer for our repo configs, and has been documented in docs/configuration.md.
A print_log statement will emit the global repo config path and per-project config path if one is detected.

[1.39.0] - 2023-06-30#

OSEC-774: add basic API to run omni-nspect

[1.38.1] - 2023-06-30#

make it possible to override tokens and interpolations with CLI

[1.38.0] - 2023-06-23#

Add setting to make repo_publish exit with a failure when no packages are located
Fixed repo build_number (and subsequently package naming) for Gitlab CI Merge Requests

[1.37.1] - 2023-06-22#

Fixed regression in run_process when it won’t exit even with exit_on_error=True if the executable doesn’t exist

[1.37.0] - 2023-06-12#

Add support for “++” syntax in configuration to merge arrays.

[1.36.1] - 2023-06-05#

fixed a bug in repo update that failed to update repo man correctly due to a recursion CLI argument breadcrumb being interpreted as a package to be updated.
tokens are now resolved in packman packages that are updated via repo update. This enabled tokens to be set in a package name. Hopefully a sane default for the token is provided by the user.

[1.36.0] - 2023-06-01#

add py_ver token that provides the Python interpreter version major + minor e.g. 3.10.4 == py310.

[1.35.2] - 2023-06-01#

target fixation happened, missed the scenario where no tokens are defined on the CLI, yet referenced within the repo.toml. We now load the combined repo.toml + repo_tools.toml merged config, set any discovered tokens, and then load the file again. This avoids a high impact refactor within the token code to correctly handle token escaping.

[1.35.1] - 2023-05-31#

Add repo poetry tool (moved from custom command batch scripts) to update poetry.lock files for repo tools

[1.35.0] - 2023-05-31#

Follow-on to 1.31.0 that moves command line custom token resolution earlier in bootstrapping such that config files are then loaded with these tokens set before we build the massive argparse parser. The prior solution only resolved tokens after the parser was built, which skipped scenarios where a token was referenced within a subtool argparse argument.
Adds the in_ci boolean token that reflects if running on gitlab, teamcity, or local. Utilizes the --ci-env flag as well so this can be faked.

[1.34.1] - 2023-05-26#

Fixed missing argument to update_packman

[1.34.0] - 2023-05-26#

Add tomli dependency and switch to using it to load toml files (faster)

[1.33.2] - 2023-05-23#

Add timeout support in run_process

[1.33.1] - 2023-05-22#

Fix a typo in repo_update to properly match repo_man and packman wildcards.

[1.33.0] - 2023-05-19#

Add the –ci-env argument to repo_man to pretend to be a CI environment for testing purposes.

[1.32.1] - 2023-05-18#

update default repo_package used by repo upload to “5.8.6”

[1.32.0] - 2023-05-12#

teach repo update to update repo.bat/sh scripts and packman
add process_args_to_cmd

[1.31.0] - 2023-05-09#

Users can now define custom tokens at the command line. This is ideally paired with a default token value in repo.toml. docs/configuration.md has more information on how this can be used.

[1.30.0] - 2023-05-09#

Added a timer context manager to RepoTelemetry that can be used within other tools to push duration metrics.
Slight refactor of our RepoTelemetry metrics gathering object to limit what set_exit_code does. It no longer sets ts_end_time and l_duration keys in the payload. The expected behavior is to now use the timer context manager.

[1.29.0] - 2023-05-04#

Refactored how repo_man presents the --config argument along with providing a secondary helper action so that repo_build’s alternate --release and --debug flags also cooperate.
This unified handling of config also enables the ${config} token to resolve correctly in all scenarios when a single config value is passed in.

[1.28.8] - 2023-04-26#

WAR for Teamcity exception on Windows -> UnicodeEncodeError: ‘charmap’ codec can’t encode character ‘\uf06a’

[1.28.5] - 2023-04-24#

Add the default to the help message in add_config_arg().

[1.28.4] - 2023-04-24#

add encoding="utf-8" to subprocess.Popen in run_process_return_output to avoid the exception: ‘charmap’ codec can’t encode character
FileNotFoundError was unreachable in both run_process and run_process_return_output

[1.28.3] - 2023-04-21#

support argparse config being a single entry list or a string, and setting a complimentary config token correctly.
map in stdin in our subprocess.Popen wrappers to enable live debugging within nested repo calls due to python_exectuable being set.

[1.28.2] - 2023-04-19#

repo_publish - remove separator from the config for package mask to support both dot and dash separators.

[1.28.1] - 2023-04-19#

Setting a user defined Python interpreter on Windows had an odd behavior. Visually it looked correct, but in reality the child process with the correct executable was running in parallel on the console with the parent process. Instead we subprocess.Popen on Windows. Unix is unchanged.

[1.28.0] - 2023-04-19#

Improves omni.repo.man.utils.run_process_return_output().
- You are now able to omit stderr from the returned output.
- You can now choose whether to print the process output to stdout. If stderr was ommitted from the output, you can choose whehter to print that to stderr.
Prevents potential crashes from several internal subprocess calls.

[1.27.0] - 2023-04-13#

Support user defined Python interpreter via either system path, packman dependency defined in source controls packman.xml file, or via packman dependency defined in repo.toml.
This will hot reload the interpreter to whatever the user defines, ideal for projects that need to release build artifacts across a range of Python versions.
It is possible this will break some tooling, please report issues you encounter.

[1.26.1] - 2023-04-13#

Fix edge case where a symlink exists in the destination directory
Fix FileNotFoundError from occurring when run_process() is called on a nonexistent program.

[1.26.0] - 2023-04-07#

Refactored entry a little bit to move the run_tool block into a discrete function to make it a little easier to test it.
Stack traces are now always dumped into the logfile, which by default is located at ${root}/_repo/repo.log.
Stacks are dumped to the console if either --verbose or --tracebacks.
The logfile is pushed to TeamCity as a build artifact in case the logging isn’t sufficient to debug.
Maybe this will be a suitable compromise for debugging purposes vs confusing stacks to console.

[1.25.5] - 2023-04-07#

Bugfix logging.info

[1.25.4] - 2023-03-30#

Bugfix Retry of http request

[1.25.3] - 2023-03-29#

Update RepoTelemetry class to allow sending multiple payloads to NVDataFlow in a single request.
Expand the environment metadata collected by RepoTelemetry.
Refactor RepoTelemetry to require an extra step to collect environment metadata.
Update logic for determining which NVDataFlow index to post telemetry data to.

[1.25.2] - 2023-03-28#

Include commit hash in repo telemetry
Include flag indicating local repo run in repo telemetry

[1.25.1] - 2023-03-27#

Improve repo build_number for use with formatted md files

[1.25.0] - 2023-03-24#

Add utils.is_running_on_ci() to check if you’re running on CI vs local.

[1.24.1] - 2023-03-22#

Fix bug where add_config_arg() was setting the default to a string when allow_multiple was set to True.

[1.24.0] - 2023-03-21#

Remove the require_build_config and add add_config_arg instead
Add get_tool_config to get the current config for a tool

[1.23.0] - 2023-03-20#

Add utils.set_niceness(): This increases the niceness of your process so that other processes get more CPU and IO time. CPU niceness is implemented on Linux and Mac right now. IO niceness is only implemented on Linux.

[1.22.1] - 2023-03-06#

Silence warnings about git operations failing when running in linbuild

[1.22.0] - 2023-03-17#

Adds the require_build_config setting to each repo tool, so that a tool can specify that it uses the config value from its options. This adds the ability for users to specify default_build_config in repo.toml to have release or debug set when config wasn’t specified by the user. This also sets the ${config} token.

[1.21.6] - 2023-03-15#

Fixed bug that alters first value of sys.argv

[1.21.5] - 2023-03-14#

Silence git errors when gathering telemetry data
Added OS type, CPU architecture, and sys.argv values to telemetry
Fixed test_report_error()
Fixed version numbers in changelog

[1.21.4] - 2023-03-13#

Fixed python typing issues
Added CONTRIB.md to index.rst

[1.21.3] - 2023-03-13#

Fixed an issue with wrong types in http handling
Fixed python typing issues
Fixed issue with not checknig return value for none

[1.21.2] - 2023-03-08#

fixed an issue where run_process could result in a python error: [WinError 6] The handle is invalid.

[1.21.1] - 2023-03-06#

optimize telemetry to gather data on a separate thread and only do that if enabled.

[1.21.0] - 2023-03-01#

collect telemetry about how long each repo tool takes to run.
look for settings in the merged repo config to enable/disable sending telemetry data to NVDataFlow.

[1.20.5] - 2023-02-28#

push repo.log as a TeamCity artifact on failure
print the exception message on non-verbose executions of repo

[1.20.4] - 2023-02-23#

tool version retrieval with version.get_tool_version within entry.report_error expected the imported subtool to be a true module with a path attribute. Turns out that is not reliable, so make a check and skip file VERSION check if said attribute is missing.

[1.20.3] - 2023-02-14#

workaround for \?\ paths in link folders allowed check for python 3.10+ on windows.

[1.20.2] - 2023-02-08#

remove stacktrace on exception dump to console. Can be enabled via --tracebacks OR flagging tracebacks_on_exit and tracebacks_on_raise in repo.toml PLUS --verbose. The tracebacks were causing some confusion.
tracebacks are still logged to the default logfile at _repo/repo.log.
cleaned up the TeamCity buildProblem on failure. Now includes an easy to copy command for the failure plus the repo_man + sub-tool’s versions to easily identify common failures.

[1.20.1] - 2023-02-01#

nvenv.TemporaryDirectory accepts the keeptemp flag as a constructor argument.

[1.20.0] - 2023-01-19#

Add support for -- to pass extra arguments to any tool using options.extra_args

[1.19.3] - 2023-01-17#

added SWIPAT links to pyproject.toml for toml + yaml libraries.

[1.19.2] - 2023-01-13#

repo update was sort of broken since 1.16.3 because of “version” variable shadowing
repo update: add --repo-root to be able to run on a different repo
properly fix platform filters from 1.16.3 version

[1.19.1] - 2023-01-06#

Had a typo in gitlab config referencing repo vs repo.sh + forgot about git config.

[1.19.0] - 2023-01-06#

CICD changes: repo format –verify on merge requests. publish packman package on merge to $CI_DEFAULT_BRANCH.
repo_test reporting coverage value for pytest suite. Push git tag on merge to $CI_DEFAULT_BRANCH.

[1.18.0] - 2023-01-04#

Rework tokens (!!):
add API to resolve tokens, set and print tokens
add repo.tokens field and use them during config resolution
support resolving tokens in list/dict/tuple
support recursive tokens resolution
add --print-tokens cmd to list all of them
update documentation for tokens

[1.17.1] - 2023-01-03#

Leverage fileutils’ ERROR_IF_NOT_EXIST flag for our linking process to enable people to opt out of link source checking.
This will enable people to create dead links, but that behavior is the old behavior and Kit is littered with this.

[1.17.0] - 2022-12-21#

More thorough handling of our _vendor directories via hashing and rebuilding on conflict.
Moved this handling from init.py into deps.verify_vendor_integrity so other tools can use it.
This is associated with the changes in 1.15.0. These changes are now ready for other tools and projects to use.

[1.16.3] - 2022-12-16#

fix platform filters for a case where we have a .release BRANCH in version, but we want all branches

[1.16.2] - 2022-12-15#

fix platform filters after adding overrides

[1.16.1] - 2022-12-15#

repo_update: try to support packages with ${platform} and ${config} in their name

[1.16.0] - 2022-12-15#

Add config overrides from CLI: –/foo/bar=123

[1.15.0] - 2022-12-13#

Add support to deps for pulling Python dependencies via pip requirements.txt with pip_install_requirements
Swapped repo_man’s vendoring setup from definitions in source to requirements.txt. repo_man continues to vendor deps at build time.

[1.14.0] - 2022-11-22#

Add renaming suffix for static libraries (staticlib_ext)

If we want to package static libs, we need rules like this or people will use very broad globbing rules.

[1.13.0] - 2022-11-07#

repo publish tool: fix behavior of fail_if_exist option

[1.12.2] - 2022-11-15#

fileutils: fail folder link when src path does not exist

[1.12.1] - 2022-11-01#

disable Sentry integration by default

[1.12.0] - 2022-10-31#

Reduce the teamcity buildProblem that gets published on unhandled exception/non-zero exit.

[1.11.5] - 2022-10-27#

repo publish tool: fix support for multiple packages.

The tool config took a list of packages to publish, but only the first was ever used.

[1.11.4] - 2022-10-20#

fix getpass.getuser() exception breaking repo_man because of Sentry

[1.11.3] - 2022-10-18#

added experimental Sentry integration

[1.11.2] - 2022-10-07#

repo update tool: fix over-agressive replacement of ie, ${config} tokens

ie, we had a kit-sdk old version of: 104.0+release.94673.4b1c9203.tc.${platform}.${config} and a new possible version of 104.0+release.94898.d9e2e7c7.tc.windows-x86_64.debug that got changed to 104.0+${config}.94898.d9e2e7c7.tc.${platform}.${config}

Now, in order to consider something a valid match, the surrounding “context” must be the same, which is the preceeding / following characters, which must be either a non-alphanumeric or the start/end of the string.

So in the above case, the original context is: (".", "") The context for “release” in the new version does not match: ("+", ".") …while the context for “debug” in the new version does: (".", "")

[1.11.1] - 2022-09-28#

Corrected the bindings_ext token for the macos-universal platform.

[1.11.0] - 2022-09-28#

Pass repo_man’s verbosity setting to packman when calling backman via repo.

[1.10.1] - 2022-09-23#

Speed up fileutils copy by moving exclude check earlier (behavior hasn’t changed)

[1.10.0] - 2022-09-21#

Separate trackback reporting to sys.exit and raise exception. This enables repo_test as the tool level to disable sys.exit traceback reporting.

[1.9.4] - 2022-09-20#

repo update tool: when inserting new version, keep ${platform} even if it doesn’t have a leading “.” - ie, 1.37.0_main.local.29536596-linux-x86_64 will now be swapped to 1.37.0_main.local.29536596-${platform}if the original version had${platform}

[1.9.3] - 2022-09-20#

repo update tool: don’t detect the platform as a prerelease this allows packages like:
```
  linbuild@1.37.0_main.local.29536596-linux-x86_64
```
…to be upgraded to, without needing --include-pre-release

[1.9.2] - 2022-09-14#

Fix running build_number not from the root of repo

[1.9.1] - 2022-09-12#

repo update tool: fix, it was broken (accidentally pushed in debug mode)

[1.9.0] - 2022-09-09#

Added a –tracebacks flag and repo.toml tracebacks key to enable feature-flagged traceback reporting.
Added traceback reporting to repo_man on sys.exit and exception raised. This is primarily for pushing tracebacks to TeamCity as a buildProblem, and optionally for traceback reporting when an underlying tool exists via sys.exit(!=0).

[1.8.6] - 2022-09-08#

repo update tool: sort by versions instead of date (add –sort-by-date for previous behavior)

[1.8.5] - 2022-09-07#

Add strip_to_first_wildcard option to copy_files

[1.8.4] - 2022-08-26#

Fixed an ordering bug with repo.toml and repo_tools.toml. Reverted back to order prior to Release 1.8.0.

[1.8.3] - 2022-08-16#

Fix bug in build_number when the gitbranch is Merge-Request with no MR number

[1.8.2] - 2022-08-16#

Fix a bug in entry.py where the variable for the subtool repo_tools.toml was used instead of the repo_man packaged repo_tools.toml.

[1.8.1] - 2022-08-16#

Fix bug in build_number where “Merge-Request” branch name is not converted to “mr” token

[1.8.0] - 2022-08-15#

Rename main.py to entry.py
Light Refactor of entry.py and added pytest tests.

[1.7.1] - 2022-08-08#

Do not raise exception when temp directory fails to be removed

[1.7.0] - 2022-08-02#

Added print_log which emits a message to the calling module’s logger and optionally prints the message if the configured root logger stdout StreamHandler would drop the message due to greater configured logging level.

[1.6.1] - 2022-07-23#

Allow nvteamcity.teamcity_message_block to be used as a decorator with no args (so it can replace deprecated teamcity_block); added tests for teamcity_message_block

[1.6.0] - 2022-07-13#

Added run_process_return_output which duplicates run_process while returning stdout/stderr in a list as a secondary return value.

[1.5.1] - 2022-07-01#

fix glob on hidden files in fileutils

[1.5.0] - 2022-06-28#

Add repo_packman - shotcut to packman

[1.4.5] - 2022-06-23#

Updates to allow fileutils (and repo package) to work with long filepaths (>=260) on windows, even if the registry setting to enable longpath support isn’t set

[1.4.4] - 2022-06-01#

Change default log location to be in _repo folder to avoid issues with log file lock when clean building

[1.4.3] - 2022-06-01#

Changing the teamcity reponse to use status rather than msg because it seems msg is now empty after th 30th of may 2022 upgrade.
Updating packman to 6.44

[1.4.2] - 2022-05-27#

Support macos-x86_64 and macos-aarch64 platform targets in PLATFORM_MAPPING. This prevents a particularly cryptic crash on launch when using mac without specifying a platform target of macos-universal. Note: you should be using macos-universal instead of these where possible.

[1.4.1] - 2022-05-25#

Change ${file:} macro to only read the first line of the file

[1.4.0] - 2022-05-06#

Add pip_install_multiple

[1.3.1] - 2022-05-06#

Improve error output for execute_git

[1.3.0] - 2022-04-14#

Add more tests for omniflow naming of versions and branches for different omniflow versions
Added ‘macos-universal’ as a supported platform.
Added detection for ‘Darwin’ platforms and changed their names to the more friendly ‘macos’.

[1.2.4] - 2022-04-05#

Fix for teamcity_message with a single unnamed arg Teamcity only allows EITHER exactly one unnamed arg, and no named args OR only named args. Before this fix, when using with single-arg messages (like publishArtifacts), it would add a timestamp named arg, which would cause teamcity to error

[1.2.3] - 2022-04-01#

Allow teamcity_message to take non-keyword args
teamcity_escape_value now properly escapes unicode characters
Add nvteamcity unit tests
Fix test_nvenv on linux when not root

[1.2.2] - 2022-03-24#

Fix file logging starting a new file every time

[1.2.1] - 2022-02-25#

Improve error report on run_process failure

[1.2.0] - 2022-02-01#

Logging: add -v and -vv to control logging
Logging: add file_logging and file_logging_path to control log to file. Log “info” level by default into “${root}/_build/repo.log”

[1.1.5] - 2022-01-27#

Fix how branch is converted to build number on older “Omniverse Flow”

[1.1.4] - 2022-01-26#

Add “changelog_file” to repo_folders guidelines

[1.1.3] - 2022-01-25#

Fix breaking change: “Omniverse Flow version to comply with updated standard” to not be enabled by default. Can be enabled with omniverse_flow_version_scheme_2 = true

[1.1.2] - 2021-11-30#

Fix breaking change introduced in 1.1.0 where version_file default changed to be without .md ext.

[1.1.1] - 2021-11-19#

Fix a file handle leak in build_number - caught by unit testing
Fix bugs in how branch names are converted to strings without slashes (compliant with Omniverse Flow); added unit tests

[1.1.0] - 2021-11-17#

Change Omniverse Flow version to comply with updated standard {software-version}+{branch}.{builder}{count}.{hash}

[1.0.0] - 2021-09-17#

Add repo_test requirement
Add Python unit test stubs to run in Gitlab-CI
Added copyright and license information

[0.4.10] - 2021-09-14#

only log info and skip when there are multiple tools with the same name present

[0.4.9] - 2021-09-13#

detect and error when there are multiple tools with the same name present

[0.4.8] - 2021-08-31#

vendor yaml and add get_yaml_module()

[0.4.7] - 2021-08-13#

only leave parallel folder link, file copy produces permission issues/races

[0.4.6] - 2021-08-13#

force vendor pip dep

[0.4.5] - 2021-08-13#

unify TC service messages usage, provide more API
parallel file copy and folder link

[0.4.4] - 2021-08-05#

repo update tool: add –include-pre-release flag, default to false

[0.4.3] - 2021-07-13#

Bug fix to first merge import configs, before actual config

[0.4.2] - 2021-07-12#

Bug fix to resolve main config with the same pipeline as a tool config

[0.4.1] - 2021-07-09#

Bug fix to only resolve string interpolation

[0.4.0] - 2021-07-09#

Add support for config interpolations, like ${conf:path.to.foo}
Add config import with repo.import_configs

[0.3.46] - 2021-06-11#

Adding a bit of support for build numbers and branches when running in gitlab.

[0.3.45] - 2021-04-21#

Added a new variable symbols_check to copy_files and when set to true makes sure there is a pdb for any dll, exe, pyd and lib for a file thats being copied.

[0.3.44] - 2021-04-13#

Patch up repo update replacing “+release” with “+${config}”.

[0.3.43] - 2021-03-29#

Add more logging to git calls

[0.3.42] - 2021-03-25#

Expose load_toml_config_with_tokens function and format code

[0.3.41] - 2021-03-10#

Fixes for 0.3.40

[0.3.40] - 2021-03-10#

Removing the dual build number stuff for now. Its become a bit of a problem for somethings and isn’t worth the effort right now.

[0.3.39] - 2021-03-08#

Update sphinx package in repo build_docs and allow overriding it with a setting

[0.3.38] - 2021-03-04#

Add find_and_extract_package, extract_archive_to_folder, short_hash

[0.3.37] - 2021-03-03#

extra_tool_paths to add more tools from other location

[0.3.36] - 2021-03-02#

Remove unused code that fixes publish of some packages with versions that triggered exception

[0.3.35] - 2021-02-19#

Change replace_file_tokens to strip file content, removing new lines at the end

[0.3.34] - 2021-02-18#

Version/OmniverseFlow - allow version only, ensure is_valid returns bool

[0.3.33] - 2021-02-16#

Add more TC block: for pip_install and for each tool run

[0.3.32] - 2021-02-16#

Minor fix to the process when checking for allowed paths.

[0.3.31] - 2021-02-15#

Code formatting.
Changed the TC blocks a little so that we get less spam in the logs and they are not doubled when they are used.

[0.3.30] - 2021-02-08#

fix applying dict filter correctly for list and tuples inside of a config dict

[0.3.29] - 2021-02-03#

move a number of modules from repo_package (mostly related to changelog generation) to here, to allow them to be used by repo_publish. Update repo publish to support changelog generation

[0.3.28] - 2021-02-03#

ignore version from other branches by default in repo update (add setting to control it)

[0.3.27] - 2021-02-02#

fixing the issue where after sorting the latest package the list is empty and an index error is raised

[0.3.26] - 2021-01-29#

Fix typo in build_number
repo update can now search just by passing start of the package name (no wildcard needed). But if match is perfect update only that one.
Small refactor in nvteamcity

[0.3.25] - 2021-01-26#

Changed the new build number stuff a little so that the dual build number is opt in for now. FINAL_BUILD_NUMBER has to be set to 0 for it to work.

[0.3.24] - 2021-01-19#

omni.repo.man.update - update_packages returns modified versions

[0.3.23] - 2021-01-14#

Minor fix for the build number order.

[0.3.21] - 2021-01-13#

Add label_regexps for publish

[0.3.20] - 2021-01-12#

Fix output build number

[0.3.19] - 2021-01-12#

repo update added control over version filtering. By default major version is switched and MR packages are ignored. CLI switches to control

[0.3.18] - 2021-01-07#

Bugfix to the store_teamcity_statistics

[0.3.17] - 2021-01-06#

Allowing a custom key for the new store_teamcity_statistics stuff.

[0.3.16] - 2021-01-04#

Removed all the test stuff from nvteamcity.py, it was mostly used during dev and had a manual verification.
Added a new decorator for teamcity statistics called teamcity_statistics_timer. You just need to pass in a string thats used as the key and the function will be timed and the key+value pair will be stored in teamcity. If nothing is passed in, then the function name will be used as the key. This will also create a Teamcity block.
Added a function store_teamcity_statistics(key, value) for the statistics to store a custom key/value if you dont want to use the wrapper. Both the key and value are required.
Added a new decorator called show_calling_function that will print out what file and line called the function that is wrapped.
Some build numbers are getting a little long and the CI systems are truncating the number, so we have added support for a shorter build number for CI and keeping the existing for packaging. This is inline with Omniverse Flow.

[0.3.15] - 2020-12-09#

Fixing the issue where we get double build numbers in the name because we check for a - and in the new Omniverse Flow scheme we dont have any. Now we check for a - and +.
Disabled a broken unittest.

[0.3.14] - 2020-12-09#

Fixing the issue where a branch might contain a / which causes confusion in with the paths.

[0.3.13] - 2020-12-07#

Allow for recurisve mappings eg $a -> $b -> $c in the prebuild.toml files section

[0.3.12] - 2020-11-18#

build_number tool, switch to version: {VERSION}+{gitbranch}.{githash}.{build_location}.{build_number}

[0.3.11] - 2020-11-18#

build_number tool, switch to version: {VERSION}[-rc.number]{[-|.]gitbranch}+{githash}.{ci}.{build_counter}

[0.3.10] - 2020-11-17#

Add resolve and print url for publish

[0.3.9] - 2020-11-08#

Add platform filter for config, e.g.: ‘platform:windows_x86-64’

[0.3.8] - 2020-11-06#

Add ‘exclude_path_patterns’ to copy_files.

[0.3.7] - 2020-11-03#

Added a check if we are doing file link instead of folder (and failure)

[0.3.6] - 2020-10-28#

Add “allowed_paths” param for copy and link file utils

[0.3.5] - 2020-10-27#

change default remote for publishing to “cloudfront” as we switching to new remote2 configuration of packman everywhere

[0.3.4] - 2020-10-23#

update upload to use latest repo_package by default

[0.3.3] - 2020-10-22#

add --output-file to build_number

[0.3.2] - 2020-10-17#

add default_remote to publish

[0.3.1] - 2020-10-14#

build_number as a tool
replace some prints with logging info and error

[0.3.0] - 2020-10-14#

New repo update tool

[0.2.9] - 2020-10-12#

Added generate_build_number() function
Formats build_number based on Omniverse Flow guidelines.

[0.2.8] - 2020-10-12#

Add extra_args to pip_install

[0.2.7] - 2020-10-07#

change default repo_deps path to be in _repo/deps

[0.2.6] - 2020-10-06#

publish: fail_if_exist

[0.2.5] - 2020-10-06#

Start producing more labels when uploading
Allow specifying any package in publish tool

[0.2.4] - 2020-10-02#

Fix vendoring if toml happen to be already installed in python’s site-packages

[0.2.3] - 2020-09-18#

Cache getting git branch and hash to optimize as they called many times often

[0.2.2] - 2020-09-18#

Fix git autoinstall from packman

[0.2.1] - 2020-09-18#

Add support for ${file:PATH} syntax in config
Switch to use VERSION file in root

[0.2] - 2020-09-11#

New functionality: ‘repo’ command as a central entry point for all repoman tools. Repoman becomes a framework to register and configure repo tools.
Introducing repo_build_docs and repo_upload builtin tools. Switching to use them.
Bring in repo_fileutils in repo_man

[0.1.22.4] - 2020-07-27#

Fixes

[0.1.22.3] - 2020-07-23#

Removed a print block used for debugging.

[0.1.22.2] - 2020-07-23#

Renamed teamcity_block to teamcity_message_block and changed it so you can pass in name and description.
Started to add teamcity blocks around the git, teamcity api and gitlab api functions.

[0.1.22.1] - 2020-07-22#

Forgot to return the output when teamcity_block is used.

[0.1.22] - 2020-07-22#

Continued work on the teamcity blocks started in 0.1.21. You can use the decorator teamcity_block to create a teamcity block around a function.
- Please see teamcity.py for some test examples. Better documentation will happen soon.

[0.1.21] - 2020-07-17#

Added a teamcity module to start collecting up functions related to teamcity usage.
- Added open_teamcity_block and close_teamcity_block which can be used to add better stats in the build log.

[0.1.20] - 2020-07-13#

Added a timeout when running git commands. Timeout default is 60 seconds and retry default is 5 times.

[0.1.19] - 2020-07-02#

Adding a timeout of 5 seconds to the gitlab api call. If it fails, the timeout will increase by 5 seconds for each try.

[0.1.18] - 2020-04-29#

Added a function to access the gitlab api. gitlab_api(url, data, gitlab_url). url is required, data (json only) and gitlab_url are optional.
Added a function to access the teamcity api. teamcity_api(url, data). url is required, data (xml only) is optional.

[0.1.17] - 2020-04-24#

Bug fix only.

[0.1.16] - 2020-04-17#

get_path_hash will return a tuple containing the githash of the path specified and the githash with the MR number if its running as a MR, ["<hash>", "<hash>-<mr>"]. If its not a merge request, then both strings will be the same hash.
trigger_teamcity_build can trigger a teamcity build if following args are passed in. The response code is returned.
- branch_name - This is the name of the branch in the repo to build.
- commit_id - This is the commit hash of the change to build.
- build_config_id - This is the build configuration ID of the job in teamcity. Its found in the general settings for a job under Build configuration ID.

[0.1.15] - 2020-04-07#

call_git and get_git_hash now accept the arg cwd, which points to the directory that the subprocess running git should run in.

[0.1.14] - 2020-03-11#

When deleting a file we now remove the read-only flag first before attempting the delete

[0.1.13] - 2019-12-17#

Add repo-deps to repo folders

[0.1.12] - 2019-11-27#

Hotfix $lib_prefix in get_platform_file_mapping

[0.1.11] - 2019-11-26#

Add gather_files_from_dict_for_platform, get_platform_os_and_arch, get_platform_file_mapping, gather_files_from_dict
Add few tests for those

[0.1.10] - 2019-11-19#

Add change_syspath
disable pip_install checks explicitly for passed folder only, that doesn’t work well with std library deps.

[0.1.9] - 2019-11-18#

Add change_syspath
pip_install checks explicitly for passed folder only (on import)

[0.1.8] - 2019-11-15#

Add module param to pip_install

[0.1.7] - 2019-11-13#

Fix few bugs, more stable

[0.1.6] - 2019-11-13#

Add change_envvar, docs

[0.1.5] - 2019-11-13#

Restructure internally: split into submodules.
Bring nvenv from packagemaker
Add git status check API and in upload script
Add explicit build_label_filename API
Add package_only flag to package_and_upload
Add basic test

[0.1.4] - 2019-11-06#

Add optional version param to pip_install

[0.1.3] - 2019-11-01#

Add package and upload API and script.
Default logging to stdout (enable_std_logging)
Add optional version param to pip_install

[0.1.0] - 2019-10-24#

First release of the repoman tool, a rebranding of the multitool