Changelog#
All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
[1.70.2] - 2024-10-28#
omni_config: fix a typo in cache path on linux when XDG_DATA_HOME is not set
[1.70.1] - 2024-10-18#
fix: detect powershell.exe before attempting to use it
[1.70.0] - 2024-10-14#
add: standard OV data/cache/logs dirs as API and as tokens
[1.69.3] - 2024-10-09#
fix: build number and get_git_branch to use a safe name for git tags in CI
[1.69.2] - 2024-10-08#
Remove
platform_target
token, setplatform_host
instead
[1.69.1] - 2024-10-01#
Attempt to use new powershell if available before falling back to old
[1.69.0] - 2024-09-23#
Utilize pip’s user-level cache with a file-lock guard to locally cache the
uv
python wheel
[1.68.1] - 2024-09-20#
Slight tweak to the
--abi
argument. If--abi=
is passed and a default repo.tokens.abi is set,platform_target_abi
token == platform_host rather than abi-declarative.
[1.68.0] - 2024-09-20#
Set
platform_target
andplatform_target_abi
tokens.platform_target
== host_platform. Overridden in repo_build if--platform-target
flag set.platform_target_abi
== abi-declarative host_platform ifabi
token also set via--abi
flag. Overridden in repo_build if--platform-target
flag set.
[1.67.0] - 2024.09-03#
Configuration filtering is now possible against tokens and envvars.
Equality checks can be performed against token and envvars for filtering.
Added abi token that can be set by the –abi CLI arg
[1.66.1] - 2024-08-02#
Fix
--set-token
and--ci-env
. Selective tool loading optimization introduced another parser missing them.
[1.66.0] - 2024-07-31#
container_utils.py log.error -> log.info to avoid confusing error-level log messages on console
container_utils.get_available_cpu_count supports a Windows path via powershell
[1.65.2] - 2024-07-30#
Remove gtl remote from packman config
[1.65.1] - 2024-07-24#
Added windows-aarch64 platform to PLATFORM_MAPPING
[1.65.0] - 2024-07-24#
Bump uv 0.1.45 -> 0.2.28 to patch out a statically linked zlib CVE
[1.64.2] - 2024-07-19#
OMPE-15376: uv on Windows is not multi-writer safe, guard all uv actions with a filelock until uv is patched
[1.64.1] - 2024-07-09#
make the
tools/repoman/repoman.py
andscripts/repoman.py
match bettermake sure both support linked directories (see 1.63.5)
[1.64.0] - 2024-07-17#
remove unused telemetry gathering (was internal only)
[1.63.5] - 2024-07-09#
better support for paths which include linked directories
[1.63.4] - 2024-06-27#
support sorting of OVFlowBuildIdentifier
[1.63.3] - 2024-06-18#
fix: build nubmer and get_git_branch to not replace branch with the tag on Gitlab CI (when tag is available)
fix: sanitize dot (.) in branch name out from build_number generation
[1.63.2] - 2024-06-06#
fix: remove the error message when calling
repo.sh
via a relative path (e.g.,kit/repo.sh
in the Kit repo)
[1.63.1] - 2024-06-05#
Added the ${path_sep} token that can be used in repo.toml configuration and replaced with a platform specific path separator
[1.63.0] - 2024-05-30#
OMPE-2613: Change NVDF
omni-devplat-metrics
endpoints from ES to Opensearch
[1.62.0] - 2024-05-29#
move
repotool_min_version_check
andraise_on_minimum_failure
intoversion.py
for reuse as a check in tool bootstrapping
[1.61.0] - 2024-05-28#
Add argument for
repo
Python dependency loading to require hashes and not fetch undeclared dependencies.Provide legacy fallback to pip for py37 and/or uv unsupported platform.
[1.60.2] - 2024-05-23#
More explict typing for container utils
[1.60.1] - 2024-05-24#
fix: repo_man started discovering tool without entry_point after 1.59.0
[1.60.0] - 2024-05-16#
Swap in
uv
instead ofpip
forrepo
Python dependency loading.repo
will now use a multiprocessing safe cache in the OS default cache directory e.g. ~/.cache/uv/Python dependencies will be symlinked against expanded wheels in the cache.
[1.59.6] - 2024-05-23#
Improved container utils exception handling
[1.59.5] - 2024-05-22#
Fixed a container utils issue where memory_limit could return None instead of 0
[1.59.4] - 2024-05-21#
Fixed another unhandled exception in container utils
[1.59.3] - 2024-05-21#
Fixed an unhandled exception in container utils
[1.59.2] - 2024-05-17#
Fix Gitlab CI build number on release branches.
[1.59.1] - 2024-05-16#
Prevent error if
repo.toml
does not exist.
[1.59.0] - 2024-05-14#
Startup optimization: only import tool that runs
[1.58.5] - 2024-05-10#
Add container utils to collect container data to be accessed by other tools
[1.58.4] - 2024-04-29#
Enable repo_poetry to point at a different pyproject.toml than the one in
${root}
Sync sentry-sdk version between pyproject.toml and
vendor.py
[1.58.3] - 2024-04-25#
Add a –wait-for-debugger option to allow for repo tool script debugging
[1.58.2] - 2024-04-04#
Remove old DEBUG_MODE from
update.py
[1.58.1] - 2024-04-24#
Quotes around system called file ops
[1.58.0] - 2024-04-24#
Removed
repo_build_docs
tool as it has been superseded byrepo_docs
.
[1.57.13] - 2024-04-23#
Addressed many small clarity and security items
[1.57.12] - 2024-04-12#
Fixes recursion error when an invalid tool is called
[1.57.11] - 2024-04-11#
When a bad argument is passed to a tool, usage for that tool is now displayed
[1.57.10] - 2024-04-10#
License as MIT
[1.57.9] - 2024-04-09#
Change exception message output for external users
[1.57.8] - 2024-03-27#
Usage provides additional information about tools
[1.57.7] - 2024-03-27#
Add mermaid diagram to docs for sample invocation to execution flow
[1.57.6] - 2024-03-27#
make exit code reported via telemetry match actual exit code returned by repo_man
[1.57.5] - 2024-03-27#
Disable sentry and telemetry for external users
[1.57.4] - 2024-03-27#
Add missing base exception
KeyboardInterrupt
handling inrun_process
functions.
[1.57.3] - 2024-03-26#
Add
Exception
handling inrun_process
functions, will catch the decoding errors that can happen when the process is killed.
[1.57.2] - 2024-03-25#
chore: add poetry licensing verbiage to readme
[1.57.1] - 2024-03-15#
lock poetry deps to urllib-1.26.18 for CVE-2023-43804
[1.57.0] - 2024-03-14#
Allow requirements file locations to be overridden by repo tools.
[1.56.5] - 2024-03-13#
Prevent the teamcity log publishing from running outside of teamcity.
[1.56.4] - 2024-03-13#
Conform all current copyrights to use SPDX
Introduce and supply standard NVIDIA license into final package
Update packman, update repo deps
Fix a repo_test upgrade bug and e2e test failure residual from pytest migration
[1.56.3] - 2024-03-11#
Add a utility function is_git_tracked() and relevant unit test
[1.56.2] - 2024-03-08#
add requests package for nicer HTTP calls
[1.56.1] - 2024-03-07#
clean_folders util now normalizes folder paths
[1.56.0] - 2024-02-28#
nv_env.in_nvidia / nvenv.in_corpnet functions for helping to avoid wasting time on retryerrors etc
[1.55.2] - 2024-02-27#
Move all tests to pytest
[1.55.1] - 2024-02-21#
Reduce gitlab ci_message-ing
[1.55.0] - 2024-02-20#
Added
omni.repo.man.deps.install_dependencies
to replace by the formeromni.repo.deps.verify_vendor_integrity
that involved a heavy hashing verification.Added
hash_directory
arg toomni.repo.man.deps.validate_dependencies
so users can opt-in for dependency verification. Defaults to false.
[1.54.0] - 2024-02-16#
Quick refactor of repo_poetry to instead ingest
tools/poetry/pyproject.toml
This converts poetry from a static vendor-requirements.txt in the downstream project directory structure to a packaged pyproject.toml.
[1.53.2] - 2024-02-16#
Propagate block disabling behavior to non-generic blocks
Uniqify gitlab message blocks to decouple section-linkage in log window
[1.53.1] - 2024-02-15#
ci_message/block can now filter by ci platform
[1.53.0] - 2024-02-12#
Add config arg
repo.file_logging_max_bytes
config option for controlling size of RotatingFileHandler logfile.Update
exception_utils._push_logfile_to_teamcity()
to push logfile + optional rotated file to TeamCity.
[1.52.0] - 2024-02-09#
Add PushEnv, a commonly reused OS environ context manager, to utils
[1.51.10] - 2024-02-08#
Make sure the py.typed marker actually makes it into the package
[1.51.9] - 2024-02-08#
Add py.typed marker for PEP-561 compliance
[1.51.8] - 2024-02-06#
Store original
sys.argv
on theoptions
for use in other repo_tools.
[1.51.7] - 2024-02-05#
output get_git_branch, get_git_hash git call errors as info, it is callers responsibility to handle None return value
[1.51.6] - 2024-01-24#
Fixed tests/pytest/test_e2e.py
Default test suite switch to ‘pytest’ suite, as it contains for more tests than ‘unittest’ suite
[1.51.5] - 2024-01-31#
allow packman lookup in find_new_version to be optimized with a version string
[1.51.4] - 2024-01-25#
Fix bad type error on Python 3.7 in configuration classes.
[1.51.3] - 2024-01-23#
avoid deprecated distutils usage
[1.51.2] - 2024-01-19#
build_number uses ci_message and repo_man’s own version is detectable by build_number
[1.51.1] - 2024-01-18#
env-var
DISABLE_CI_MESSAGE_BLOCK
being truthy will short-circuit ci_message_block
[1.51.0] - 2024-01-17#
Added configuration classes using Pydantic for improved configuration handling.
[1.50.15] - 2024-01-17#
Fix mutable default in dependencies md5 calculation incorrectly invalidating package installs on multiple calls.
[1.50.14] - 2024-01-12#
Fix urllib version to 1.26.16 so that repo test will work again
[1.50.13] - 2024-01-11#
Revert MR !187 that breaks
repo update repo
.
[1.50.12] - 2024-01-11#
Fix bug where there was a dependency on
dateutil
from packman. Dependency removed. This will allow usage with packman 7.17.
[1.50.11] - 2024-01-10#
Add missing export for
QuietExpectedError
.
[1.50.10] - 2024-01-10#
Add
QuietExpectedError
that will not print anything to the console or file log when raised.
[1.50.9] - 2024-01-03#
Set pip timeout to 60 seconds and retries to 5.
[1.50.8] - 2023-12-20#
Added config arg to repo_packman tool (for release/debug tokens)
Forward repo.tokens to repo_packman tool
[1.50.7] - 2023-12-15#
Don’t open/close ci blocks from a background thread.
[1.50.6] - 2023-12-14#
Swapped the base_delay and growth_rate values for
omni.repo.man.deps._retry_rename
to correctly backoff.
[1.50.5] - 2023-12-13#
Added
omni.repo.man.utils.exponential_backoff_retry
decorator for easy exponential retry of functions.Exponential retry for the move operation in omni.repo.man.deps for moving
repo
Python deps install folder. This potentially addresses the PermissionError issue encountered on Windows.
[1.50.4] - 2023-12-11#
Changed copy and link time diagnostics to print in milliseconds instead of seconds
[1.50.3] - 2023-12-07#
In repo.sh prefix the
exec
command with the setter for OMNI_REPO_ROOT rather than usingexport
and polluting the user’s shell.In repo.bat use
SETLOCAL
andENDLOCAL
to constrain OMNI_REPO_ROOT to the batch file scope.
[1.50.2] - 2023-12-01#
Use
print
instead ofprint_log
in deps.py
[1.50.1] - 2023-12-01#
Add an option to bypass rich print in
print_log
[1.50.0] - 2023-11-30#
Brief refactor of build_docs.py, build_number.py, cmd_config_overrider.py, configuration.py, debugging.py, and deps.py.
sys.exit
and baseException
swapped for eitheromni.repo.man.exceptions.RepoToolError
or built-inException
subclasses where it makes sense.
[1.49.2] - 2023-11-30#
Sanitize your inputs. The OMNI_REPO_ROOT envvar for
repo
Python dependency loading included extra quotes and pathlib.Path explodes on py37 but silently fails on py310. Neither behavior is desirable, strip quotes instead.
[1.49.1] - 2023-11-28#
Patch
omni.repo.man.deps.pip_install_requirements
to emit the stdout/stderr coming out of pip in the event of failure, even if thequiet
argument is passed.Retry
omni.repo.man.deps.pip_install_requirements
3 times with 15 second delays to provide a little buffer for our unreliable network.
[1.49.0] - 2023-11-09#
Implement a decorator which can be used to enforce a minimum version of a given repo-tool for function execution with a clear exception. e.g. repotool_min_version(“repo_man”, “1.50.0”)
[1.48.1] - 2023-11-07#
Corrected a bug where logger propagation was set to False in omni.repo.man.exceptions_utils.emit_stack_to_logfile. This caused the helpful exception related messaging to not get pushed to the console leading to confusion and tedium referencing logfiles.
[1.48.0] - 2023-11-03#
Permanently mute repo_telemetry
whoami
call to avoid error level log messages in Linbuild.Refactor omni.repo.man.exceptions to extract all helper utilities into omni.repo.man.exceptions_utils to avoid circular dependency loading.
[1.47.2] - 2023-10-31#
Import repo-tool template yml for CI
[1.47.1] - 2023-10-31#
Add repo logs artifacts (CI only)
[1.47.0] - 2023-10-30#
Update .gitlab-ci.yml, add new jobs for more validation.
Fixed tests that were not working on Gitlab CI during merge requests pipelines
Renamed
CHANGES.md
toCHANGELOG.md
Renamed mytest suite to unittest
[1.46.6] - 2023-10-30#
Emit TeamCity buildProblem status message first to set the build log anchor point, then emit the log messages stating an error has occured.
[1.46.6] - 2023-10-25#
Fix exception name to be correct for any exception
Print repo tools errors to the console
[1.46.5] - 2023-10-18#
Added the exception error message for the general repo on-exit error reporting. This is added context for scenarios where the
repo.log
logfile does not get saved off, and a stack is not emitted to console.
[1.46.4] - 2023-10-16#
Fix unnecessary cache re-population in the edge case where requirements.txt is more recent than the existing cache, but the regenerated cache is identical to the existing cache.
[1.46.3] - 2023-10-12#
Silence git errors from repo telemetry. If telemetry fails it should not spam the console, it isn’t a critical path tool.
[1.46.2] - 2023-09-26#
Added a file lock around Python dependency operations. Turns out my testing against Kit missed rapid-thrash projects that 100% stomped each other.
[1.46.1] - 2023-09-22#
MacOS is missing
realpath
. Swap to usingpwd -P
when settingOMNI_REPO_ROOT
envvar.
[1.46.0] - 2023-09-21#
Refactored repo tool Python dependency caching:
Tool dependencies will now be stored in the path
${root}/_repo/deps/${tool_name}_deps/${platform}/${python_version}/${requirements.txt hash}
The intention is that the execution of tools with multiple python versions against one repository will not reuse dependencies for another python version.
This sets up our dependencies for purposeful shared caching as well with rigid validation that shared dependencies have not changed.
Parallel attempts to pull the dependencies should no longer clash. A tempdir is used and a final fight will occur during the copy with retries, validation, and acceptance if another process wins.
Simplified the entrypoint for Python dependency caching within
omni.repo.man.__init__.py
[1.45.0] - 2023-09-19#
Add
import_optional_configs
to be able to import configs only if exist
[1.44.1] - 2023-09-11#
rich
: Document tool settings and support older python runtimes by catching the unprotectedrich.print
call
[1.44.0] - 2023-09-05#
Refactor on-exit exception handling for repo tools. Current behavior should not change much, but this sets up tooling to start raising precise exceptions rather than exiting non-zero or raising a generic Exception.
New exit code rules:
exit code 1 == an unexpected error was encountered, e.g. compiler error
exit code 2 == an expected error was encountered, e.g. a test failed or
repo format --verify
failed.
omni.repo.man.exceptions contains an exception base class and several subclasses to act as a start.
tracebacks_on_exit
has been deprecated as a configuration file option. It in general was not used, and moving forward we should be deprecating our sys.exit(1) calls, thereby nullifying this exit path.
[1.43.2] - 2023-09-04#
fix long dormant bug in OVFlowBuildIdentifier
[1.43.1] - 2023-08-28#
fix typing issue when repo_man used with python-3.7
[1.43.0] - 2023-08-25#
new nvgitlab.py module, new utils for ci_message / ci_message_block to help with teamcity-to-gitlab migrations
moved some ci-independent utils from nvteamcity to utils
[1.42.5] - 2023-08-16#
repo update
- don’t match versions with a different build metadata parts, e.g. kit package with.cp39
in the version
[1.42.4] - 2023-08-11#
fix
get_git_branch
on Gitlab
[1.42.3] - 2023-07-27#
rich
: replace_bultin_print default to false, as allprint("##teamcity...")
statements stop workingrich
: leaveprint_log
to userich.print
[1.42.2] - 2023-07-27#
rich
: disable wrapping for a print functionrich
: disable custom logger when running on CI
[1.42.1] - 2023-07-27#
Use
GITLAB_CI
to detect if we run on Gitlab CI - more accurate than usingCI_PIPELINE_ID
[1.42.0] - 2023-07-27#
add
rich
module dependency and integration
[1.41.0] - 2023-07-13#
RepoTelemetry
timeout is now customizable at object instantiation. The index version is also customizable at object instantiation.
[1.40.2] - 2023-07-11#
Fix parse_packman_config for packman 7 API
[1.40.1] - 2023-07-07#
Bump nspect version and allow changing it with
repo.nspect_version
key in config
[1.40.0] - 2023-07-03#
Added support for a global repo config file located at
~/.nvidia-omniverse/config/global_repo.toml
This config file is the second to final layer for our repo configs, and has been documented in docs/configuration.md.
Added support for per-project config file located in
${repo_root}/user.repo.toml
This config file is the final layer for our repo configs, and has been documented in docs/configuration.md.
A print_log statement will emit the global repo config path and per-project config path if one is detected.
[1.39.0] - 2023-06-30#
OSEC-774: add basic API to run omni-nspect
[1.38.1] - 2023-06-30#
make it possible to override tokens and interpolations with CLI
[1.38.0] - 2023-06-23#
Add setting to make
repo_publish
exit with a failure when no packages are locatedFixed
repo build_number
(and subsequently package naming) for Gitlab CI Merge Requests
[1.37.1] - 2023-06-22#
Fixed regression in run_process when it won’t exit even with
exit_on_error=True
if the executable doesn’t exist
[1.37.0] - 2023-06-12#
Add support for “++” syntax in configuration to merge arrays.
[1.36.1] - 2023-06-05#
fixed a bug in
repo update
that failed to updaterepo man
correctly due to a recursion CLI argument breadcrumb being interpreted as a package to be updated.tokens are now resolved in packman packages that are updated via
repo update
. This enabled tokens to be set in a package name. Hopefully a sane default for the token is provided by the user.
[1.36.0] - 2023-06-01#
add
py_ver
token that provides the Python interpreter version major + minor e.g. 3.10.4 == py310.
[1.35.2] - 2023-06-01#
target fixation happened, missed the scenario where no tokens are defined on the CLI, yet referenced within the repo.toml. We now load the combined repo.toml + repo_tools.toml merged config, set any discovered tokens, and then load the file again. This avoids a high impact refactor within the token code to correctly handle token escaping.
[1.35.1] - 2023-05-31#
Add
repo poetry
tool (moved from custom command batch scripts) to update poetry.lock files for repo tools
[1.35.0] - 2023-05-31#
Follow-on to
1.31.0
that moves command line custom token resolution earlier in bootstrapping such that config files are then loaded with these tokens set before we build the massive argparse parser. The prior solution only resolved tokens after the parser was built, which skipped scenarios where a token was referenced within a subtool argparse argument.Adds the
in_ci
boolean token that reflects if running on gitlab, teamcity, or local. Utilizes the--ci-env
flag as well so this can be faked.
[1.34.1] - 2023-05-26#
Fixed missing argument to
update_packman
[1.34.0] - 2023-05-26#
Add
tomli
dependency and switch to using it to load toml files (faster)
[1.33.2] - 2023-05-23#
Add timeout support in
run_process
[1.33.1] - 2023-05-22#
Fix a typo in repo_update to properly match repo_man and packman wildcards.
[1.33.0] - 2023-05-19#
Add the –ci-env argument to repo_man to pretend to be a CI environment for testing purposes.
[1.32.1] - 2023-05-18#
update default repo_package used by repo upload to “5.8.6”
[1.32.0] - 2023-05-12#
teach repo update to update repo.bat/sh scripts and packman
add process_args_to_cmd
[1.31.0] - 2023-05-09#
Users can now define custom tokens at the command line. This is ideally paired with a default token value in repo.toml. docs/configuration.md has more information on how this can be used.
[1.30.0] - 2023-05-09#
Added a
timer
context manager toRepoTelemetry
that can be used within other tools to push duration metrics.Slight refactor of our
RepoTelemetry
metrics gathering object to limit whatset_exit_code
does. It no longer setsts_end_time
andl_duration
keys in the payload. The expected behavior is to now use thetimer
context manager.
[1.29.0] - 2023-05-04#
Refactored how repo_man presents the
--config
argument along with providing a secondary helper action so that repo_build’s alternate--release
and--debug
flags also cooperate.This unified handling of config also enables the
${config}
token to resolve correctly in all scenarios when a single config value is passed in.
[1.28.8] - 2023-04-26#
WAR for Teamcity exception on Windows -> UnicodeEncodeError: ‘charmap’ codec can’t encode character ‘\uf06a’
[1.28.5] - 2023-04-24#
Add the default to the help message in
add_config_arg()
.
[1.28.4] - 2023-04-24#
add
encoding="utf-8"
tosubprocess.Popen
inrun_process_return_output
to avoid the exception: ‘charmap’ codec can’t encode characterFileNotFoundError
was unreachable in bothrun_process
andrun_process_return_output
[1.28.3] - 2023-04-21#
support argparse config being a single entry list or a string, and setting a complimentary config token correctly.
map in stdin in our subprocess.Popen wrappers to enable live debugging within nested
repo
calls due topython_exectuable
being set.
[1.28.2] - 2023-04-19#
repo_publish - remove separator from the config for package mask to support both dot and dash separators.
[1.28.1] - 2023-04-19#
Setting a user defined Python interpreter on Windows had an odd behavior. Visually it looked correct, but in reality the child process with the correct executable was running in parallel on the console with the parent process. Instead we subprocess.Popen on Windows. Unix is unchanged.
[1.28.0] - 2023-04-19#
Improves
omni.repo.man.utils.run_process_return_output()
.You are now able to omit
stderr
from the returned output.You can now choose whether to print the process output to
stdout
. Ifstderr
was ommitted from the output, you can choose whehter to print that tostderr
.
Prevents potential crashes from several internal subprocess calls.
[1.27.0] - 2023-04-13#
Support user defined Python interpreter via either system path, packman dependency defined in source controls packman.xml file, or via packman dependency defined in repo.toml.
This will hot reload the interpreter to whatever the user defines, ideal for projects that need to release build artifacts across a range of Python versions.
It is possible this will break some tooling, please report issues you encounter.
[1.26.1] - 2023-04-13#
Fix edge case where a symlink exists in the destination directory
Fix FileNotFoundError from occurring when
run_process()
is called on a nonexistent program.
[1.26.0] - 2023-04-07#
Refactored entry a little bit to move the
run_tool
block into a discrete function to make it a little easier to test it.Stack traces are now always dumped into the logfile, which by default is located at
${root}/_repo/repo.log
.Stacks are dumped to the console if either
--verbose
or--tracebacks
.The logfile is pushed to TeamCity as a build artifact in case the logging isn’t sufficient to debug.
Maybe this will be a suitable compromise for debugging purposes vs confusing stacks to console.
[1.25.5] - 2023-04-07#
Bugfix logging.info
[1.25.4] - 2023-03-30#
Bugfix Retry of http request
[1.25.3] - 2023-03-29#
Update RepoTelemetry class to allow sending multiple payloads to NVDataFlow in a single request.
Expand the environment metadata collected by RepoTelemetry.
Refactor RepoTelemetry to require an extra step to collect environment metadata.
Update logic for determining which NVDataFlow index to post telemetry data to.
[1.25.2] - 2023-03-28#
Include commit hash in repo telemetry
Include flag indicating local repo run in repo telemetry
[1.25.1] - 2023-03-27#
Improve repo build_number for use with formatted md files
[1.25.0] - 2023-03-24#
Add
utils.is_running_on_ci()
to check if you’re running on CI vs local.
[1.24.1] - 2023-03-22#
Fix bug where
add_config_arg()
was setting the default to a string whenallow_multiple
was set toTrue
.
[1.24.0] - 2023-03-21#
Remove the
require_build_config
and addadd_config_arg
insteadAdd
get_tool_config
to get the current config for a tool
[1.23.0] - 2023-03-20#
Add utils.set_niceness(): This increases the niceness of your process so that other processes get more CPU and IO time. CPU niceness is implemented on Linux and Mac right now. IO niceness is only implemented on Linux.
[1.22.1] - 2023-03-06#
Silence warnings about git operations failing when running in linbuild
[1.22.0] - 2023-03-17#
Adds the
require_build_config
setting to each repo tool, so that a tool can specify that it uses theconfig
value from its options. This adds the ability for users to specifydefault_build_config
inrepo.toml
to haverelease
ordebug
set whenconfig
wasn’t specified by the user. This also sets the${config}
token.
[1.21.6] - 2023-03-15#
Fixed bug that alters first value of sys.argv
[1.21.5] - 2023-03-14#
Silence git errors when gathering telemetry data
Added OS type, CPU architecture, and sys.argv values to telemetry
Fixed test_report_error()
Fixed version numbers in changelog
[1.21.4] - 2023-03-13#
Fixed python typing issues
Added CONTRIB.md to index.rst
[1.21.3] - 2023-03-13#
Fixed an issue with wrong types in http handling
Fixed python typing issues
Fixed issue with not checknig return value for none
[1.21.2] - 2023-03-08#
fixed an issue where
run_process
could result in a python error:[WinError 6] The handle is invalid
.
[1.21.1] - 2023-03-06#
optimize telemetry to gather data on a separate thread and only do that if enabled.
[1.21.0] - 2023-03-01#
collect telemetry about how long each repo tool takes to run.
look for settings in the merged repo config to enable/disable sending telemetry data to NVDataFlow.
[1.20.5] - 2023-02-28#
push repo.log as a TeamCity artifact on failure
print the exception message on non-verbose executions of repo
[1.20.4] - 2023-02-23#
tool version retrieval with
version.get_tool_version
withinentry.report_error
expected the imported subtool to be a true module with a path attribute. Turns out that is not reliable, so make a check and skip file VERSION check if said attribute is missing.
[1.20.3] - 2023-02-14#
workaround for \?\ paths in link folders allowed check for python 3.10+ on windows.
[1.20.2] - 2023-02-08#
remove stacktrace on exception dump to console. Can be enabled via
--tracebacks
OR flaggingtracebacks_on_exit
andtracebacks_on_raise
in repo.toml PLUS--verbose
. The tracebacks were causing some confusion.tracebacks are still logged to the default logfile at _repo/repo.log.
cleaned up the TeamCity buildProblem on failure. Now includes an easy to copy command for the failure plus the repo_man + sub-tool’s versions to easily identify common failures.
[1.20.1] - 2023-02-01#
nvenv.TemporaryDirectory accepts the keeptemp flag as a constructor argument.
[1.20.0] - 2023-01-19#
Add support for
--
to pass extra arguments to any tool usingoptions.extra_args
[1.19.3] - 2023-01-17#
added SWIPAT links to pyproject.toml for toml + yaml libraries.
[1.19.2] - 2023-01-13#
repo update was sort of broken since 1.16.3 because of “version” variable shadowing
repo update: add
--repo-root
to be able to run on a different repoproperly fix platform filters from 1.16.3 version
[1.19.1] - 2023-01-06#
Had a typo in gitlab config referencing repo vs repo.sh + forgot about git config.
[1.19.0] - 2023-01-06#
CICD changes: repo format –verify on merge requests. publish packman package on merge to
$CI_DEFAULT_BRANCH
.repo_test reporting coverage value for pytest suite. Push git tag on merge to
$CI_DEFAULT_BRANCH
.
[1.18.0] - 2023-01-04#
Rework tokens (!!):
add API to resolve tokens, set and print tokens
add
repo.tokens
field and use them during config resolutionsupport resolving tokens in list/dict/tuple
support recursive tokens resolution
add
--print-tokens
cmd to list all of themupdate documentation for tokens
[1.17.1] - 2023-01-03#
Leverage fileutils’ ERROR_IF_NOT_EXIST flag for our linking process to enable people to opt out of link source checking.
This will enable people to create dead links, but that behavior is the old behavior and Kit is littered with this.
[1.17.0] - 2022-12-21#
More thorough handling of our _vendor directories via hashing and rebuilding on conflict.
Moved this handling from init.py into deps.verify_vendor_integrity so other tools can use it.
This is associated with the changes in 1.15.0. These changes are now ready for other tools and projects to use.
[1.16.3] - 2022-12-16#
fix platform filters for a case where we have a
.release
BRANCH in version, but we want all branches
[1.16.2] - 2022-12-15#
fix platform filters after adding overrides
[1.16.1] - 2022-12-15#
repo_update: try to support packages with ${platform} and ${config} in their name
[1.16.0] - 2022-12-15#
Add config overrides from CLI: –/foo/bar=123
[1.15.0] - 2022-12-13#
Add support to deps for pulling Python dependencies via pip requirements.txt with
pip_install_requirements
Swapped repo_man’s vendoring setup from definitions in source to requirements.txt. repo_man continues to vendor deps at build time.
[1.14.0] - 2022-11-22#
Add renaming suffix for static libraries (staticlib_ext)
If we want to package static libs, we need rules like this or people will use very broad globbing rules.
[1.13.0] - 2022-11-07#
repo publish tool: fix behavior of fail_if_exist option
[1.12.2] - 2022-11-15#
fileutils: fail folder link when src path does not exist
[1.12.1] - 2022-11-01#
disable Sentry integration by default
[1.12.0] - 2022-10-31#
Reduce the teamcity buildProblem that gets published on unhandled exception/non-zero exit.
[1.11.5] - 2022-10-27#
repo publish tool: fix support for multiple packages.
The tool config took a list of packages to publish, but only the first was ever used.
[1.11.4] - 2022-10-20#
fix getpass.getuser() exception breaking repo_man because of Sentry
[1.11.3] - 2022-10-18#
added experimental Sentry integration
[1.11.2] - 2022-10-07#
repo update tool: fix over-agressive replacement of ie, ${config} tokens
ie, we had a kit-sdk old version of:
104.0+release.94673.4b1c9203.tc.${platform}.${config}
and a new possible version of104.0+release.94898.d9e2e7c7.tc.windows-x86_64.debug
that got changed to104.0+${config}.94898.d9e2e7c7.tc.${platform}.${config}
Now, in order to consider something a valid match, the surrounding “context” must be the same, which is the preceeding / following characters, which must be either a non-alphanumeric or the start/end of the string.
So in the above case, the original context is:
(".", "")
The context for “release” in the new version does not match:("+", ".")
…while the context for “debug” in the new version does:(".", "")
[1.11.1] - 2022-09-28#
Corrected the
bindings_ext
token for themacos-universal
platform.
[1.11.0] - 2022-09-28#
Pass repo_man’s verbosity setting to packman when calling backman via repo.
[1.10.1] - 2022-09-23#
Speed up fileutils copy by moving exclude check earlier (behavior hasn’t changed)
[1.10.0] - 2022-09-21#
Separate trackback reporting to sys.exit and raise exception. This enables repo_test as the tool level to disable sys.exit traceback reporting.
[1.9.4] - 2022-09-20#
repo update tool: when inserting new version, keep ${platform} even if it doesn’t have a leading “.” - ie,
1.37.0_main.local.29536596-linux-x86_64
will now be swapped to1.37.0_main.local.29536596-${platform}
if the original version had${platform}
[1.9.3] - 2022-09-20#
repo update tool: don’t detect the platform as a prerelease this allows packages like:
linbuild@1.37.0_main.local.29536596-linux-x86_64
…to be upgraded to, without needing
--include-pre-release
[1.9.2] - 2022-09-14#
Fix running build_number not from the root of repo
[1.9.1] - 2022-09-12#
repo update tool: fix, it was broken (accidentally pushed in debug mode)
[1.9.0] - 2022-09-09#
Added a –tracebacks flag and repo.toml tracebacks key to enable feature-flagged traceback reporting.
Added traceback reporting to repo_man on sys.exit and exception raised. This is primarily for pushing tracebacks to TeamCity as a buildProblem, and optionally for traceback reporting when an underlying tool exists via sys.exit(!=0).
[1.8.6] - 2022-09-08#
repo update tool: sort by versions instead of date (add –sort-by-date for previous behavior)
[1.8.5] - 2022-09-07#
Add
strip_to_first_wildcard
option tocopy_files
[1.8.4] - 2022-08-26#
Fixed an ordering bug with repo.toml and repo_tools.toml. Reverted back to order prior to Release 1.8.0.
[1.8.3] - 2022-08-16#
Fix bug in build_number when the gitbranch is Merge-Request with no MR number
[1.8.2] - 2022-08-16#
Fix a bug in
entry.py
where the variable for the subtool repo_tools.toml was used instead of the repo_man packaged repo_tools.toml.
[1.8.1] - 2022-08-16#
Fix bug in build_number where “Merge-Request” branch name is not converted to “mr” token
[1.8.0] - 2022-08-15#
Rename
main.py
toentry.py
Light Refactor of
entry.py
and added pytest tests.
[1.7.1] - 2022-08-08#
Do not raise exception when temp directory fails to be removed
[1.7.0] - 2022-08-02#
Added
print_log
which emits a message to the calling module’s logger and optionally prints the message if the configured root logger stdout StreamHandler would drop the message due to greater configured logging level.
[1.6.1] - 2022-07-23#
Allow
nvteamcity.teamcity_message_block
to be used as a decorator with no args (so it can replace deprecatedteamcity_block
); added tests forteamcity_message_block
[1.6.0] - 2022-07-13#
Added
run_process_return_output
which duplicatesrun_process
while returning stdout/stderr in a list as a secondary return value.
[1.5.1] - 2022-07-01#
fix glob on hidden files in fileutils
[1.5.0] - 2022-06-28#
Add repo_packman - shotcut to packman
[1.4.5] - 2022-06-23#
Updates to allow fileutils (and
repo package
) to work with long filepaths (>=260) on windows, even if the registry setting to enable longpath support isn’t set
[1.4.4] - 2022-06-01#
Change default log location to be in
_repo
folder to avoid issues with log file lock when clean building
[1.4.3] - 2022-06-01#
Changing the teamcity reponse to use
status
rather thanmsg
because it seemsmsg
is now empty after th 30th of may 2022 upgrade.Updating packman to 6.44
[1.4.2] - 2022-05-27#
Support macos-x86_64 and macos-aarch64 platform targets in PLATFORM_MAPPING. This prevents a particularly cryptic crash on launch when using mac without specifying a platform target of macos-universal. Note: you should be using macos-universal instead of these where possible.
[1.4.1] - 2022-05-25#
Change ${file:} macro to only read the first line of the file
[1.4.0] - 2022-05-06#
Add
pip_install_multiple
[1.3.1] - 2022-05-06#
Improve error output for execute_git
[1.3.0] - 2022-04-14#
Add more tests for omniflow naming of versions and branches for different omniflow versions
Added ‘macos-universal’ as a supported platform.
Added detection for ‘Darwin’ platforms and changed their names to the more friendly ‘macos’.
[1.2.4] - 2022-04-05#
Fix for teamcity_message with a single unnamed arg Teamcity only allows EITHER exactly one unnamed arg, and no named args OR only named args. Before this fix, when using with single-arg messages (like publishArtifacts), it would add a timestamp named arg, which would cause teamcity to error
[1.2.3] - 2022-04-01#
Allow teamcity_message to take non-keyword args
teamcity_escape_value now properly escapes unicode characters
Add nvteamcity unit tests
Fix test_nvenv on linux when not root
[1.2.2] - 2022-03-24#
Fix file logging starting a new file every time
[1.2.1] - 2022-02-25#
Improve error report on run_process failure
[1.2.0] - 2022-02-01#
Logging: add
-v
and-vv
to control loggingLogging: add
file_logging
andfile_logging_path
to control log to file. Log “info” level by default into “${root}/_build/repo.log”
[1.1.5] - 2022-01-27#
Fix how branch is converted to build number on older “Omniverse Flow”
[1.1.4] - 2022-01-26#
Add “changelog_file” to repo_folders guidelines
[1.1.3] - 2022-01-25#
Fix breaking change: “Omniverse Flow version to comply with updated standard” to not be enabled by default. Can be enabled with
omniverse_flow_version_scheme_2 = true
[1.1.2] - 2021-11-30#
Fix breaking change introduced in 1.1.0 where version_file default changed to be without .md ext.
[1.1.1] - 2021-11-19#
Fix a file handle leak in build_number - caught by unit testing
Fix bugs in how branch names are converted to strings without slashes (compliant with Omniverse Flow); added unit tests
[1.1.0] - 2021-11-17#
Change Omniverse Flow version to comply with updated standard
{software-version}+{branch}.{builder}{count}.{hash}
[1.0.0] - 2021-09-17#
Add repo_test requirement
Add Python unit test stubs to run in Gitlab-CI
Added copyright and license information
[0.4.10] - 2021-09-14#
only log info and skip when there are multiple tools with the same name present
[0.4.9] - 2021-09-13#
detect and error when there are multiple tools with the same name present
[0.4.8] - 2021-08-31#
vendor yaml and add
get_yaml_module()
[0.4.7] - 2021-08-13#
only leave parallel folder link, file copy produces permission issues/races
[0.4.6] - 2021-08-13#
force vendor pip dep
[0.4.5] - 2021-08-13#
unify TC service messages usage, provide more API
parallel file copy and folder link
[0.4.4] - 2021-08-05#
repo update tool: add –include-pre-release flag, default to false
[0.4.3] - 2021-07-13#
Bug fix to first merge import configs, before actual config
[0.4.2] - 2021-07-12#
Bug fix to resolve main config with the same pipeline as a tool config
[0.4.1] - 2021-07-09#
Bug fix to only resolve string interpolation
[0.4.0] - 2021-07-09#
Add support for config interpolations, like
${conf:path.to.foo}
Add config import with
repo.import_configs
[0.3.46] - 2021-06-11#
Adding a bit of support for build numbers and branches when running in gitlab.
[0.3.45] - 2021-04-21#
Added a new variable
symbols_check
tocopy_files
and when set to true makes sure there is a pdb for any dll, exe, pyd and lib for a file thats being copied.
[0.3.44] - 2021-04-13#
Patch up repo update replacing “+release” with “+${config}”.
[0.3.43] - 2021-03-29#
Add more logging to git calls
[0.3.42] - 2021-03-25#
Expose
load_toml_config_with_tokens
function and format code
[0.3.41] - 2021-03-10#
Fixes for 0.3.40
[0.3.40] - 2021-03-10#
Removing the dual build number stuff for now. Its become a bit of a problem for somethings and isn’t worth the effort right now.
[0.3.39] - 2021-03-08#
Update sphinx package in repo build_docs and allow overriding it with a setting
[0.3.38] - 2021-03-04#
Add
find_and_extract_package
,extract_archive_to_folder
,short_hash
[0.3.37] - 2021-03-03#
extra_tool_paths
to add more tools from other location
[0.3.36] - 2021-03-02#
Remove unused code that fixes publish of some packages with versions that triggered exception
[0.3.35] - 2021-02-19#
Change
replace_file_tokens
to strip file content, removing new lines at the end
[0.3.34] - 2021-02-18#
Version/OmniverseFlow - allow version only, ensure is_valid returns bool
[0.3.33] - 2021-02-16#
Add more TC block: for pip_install and for each tool run
[0.3.32] - 2021-02-16#
Minor fix to the process when checking for allowed paths.
[0.3.31] - 2021-02-15#
Code formatting.
Changed the TC blocks a little so that we get less spam in the logs and they are not doubled when they are used.
[0.3.30] - 2021-02-08#
fix applying dict filter correctly for list and tuples inside of a config dict
[0.3.29] - 2021-02-03#
move a number of modules from repo_package (mostly related to changelog generation) to here, to allow them to be used by repo_publish. Update repo publish to support changelog generation
[0.3.28] - 2021-02-03#
ignore version from other branches by default in
repo update
(add setting to control it)
[0.3.27] - 2021-02-02#
fixing the issue where after sorting the latest package the list is empty and an index error is raised
[0.3.26] - 2021-01-29#
Fix typo in build_number
repo update can now search just by passing start of the package name (no wildcard needed). But if match is perfect update only that one.
Small refactor in nvteamcity
[0.3.25] - 2021-01-26#
Changed the new build number stuff a little so that the dual build number is opt in for now.
FINAL_BUILD_NUMBER
has to be set to 0 for it to work.
[0.3.24] - 2021-01-19#
omni.repo.man.update - update_packages returns modified versions
[0.3.23] - 2021-01-14#
Minor fix for the build number order.
[0.3.21] - 2021-01-13#
Add
label_regexps
for publish
[0.3.20] - 2021-01-12#
Fix output build number
[0.3.19] - 2021-01-12#
repo update
added control over version filtering. By default major version is switched and MR packages are ignored. CLI switches to control
[0.3.18] - 2021-01-07#
Bugfix to the
store_teamcity_statistics
[0.3.17] - 2021-01-06#
Allowing a custom key for the new
store_teamcity_statistics
stuff.
[0.3.16] - 2021-01-04#
Removed all the test stuff from
nvteamcity.py
, it was mostly used during dev and had a manual verification.Added a new decorator for teamcity statistics called
teamcity_statistics_timer
. You just need to pass in a string thats used as the key and the function will be timed and the key+value pair will be stored in teamcity. If nothing is passed in, then the function name will be used as the key. This will also create a Teamcity block.Added a function
store_teamcity_statistics(key, value)
for the statistics to store a custom key/value if you dont want to use the wrapper. Both thekey
andvalue
are required.Added a new decorator called
show_calling_function
that will print out what file and line called the function that is wrapped.Some build numbers are getting a little long and the CI systems are truncating the number, so we have added support for a shorter build number for CI and keeping the existing for packaging. This is inline with Omniverse Flow.
[0.3.15] - 2020-12-09#
Fixing the issue where we get double build numbers in the name because we check for a
-
and in the new Omniverse Flow scheme we dont have any. Now we check for a-
and+
.Disabled a broken unittest.
[0.3.14] - 2020-12-09#
Fixing the issue where a branch might contain a
/
which causes confusion in with the paths.
[0.3.13] - 2020-12-07#
Allow for recurisve mappings eg $a -> $b -> $c in the prebuild.toml files section
[0.3.12] - 2020-11-18#
build_number tool, switch to version:
{VERSION}+{gitbranch}.{githash}.{build_location}.{build_number}
[0.3.11] - 2020-11-18#
build_number tool, switch to version:
{VERSION}[-rc.number]{[-|.]gitbranch}+{githash}.{ci}.{build_counter}
[0.3.10] - 2020-11-17#
Add resolve and print url for publish
[0.3.9] - 2020-11-08#
Add platform filter for config, e.g.: ‘platform:windows_x86-64’
[0.3.8] - 2020-11-06#
Add ‘exclude_path_patterns’ to copy_files.
[0.3.7] - 2020-11-03#
Added a check if we are doing file link instead of folder (and failure)
[0.3.6] - 2020-10-28#
Add “allowed_paths” param for copy and link file utils
[0.3.5] - 2020-10-27#
change default remote for publishing to “cloudfront” as we switching to new remote2 configuration of packman everywhere
[0.3.4] - 2020-10-23#
update upload to use latest repo_package by default
[0.3.3] - 2020-10-22#
add
--output-file
to build_number
[0.3.2] - 2020-10-17#
add
default_remote
to publish
[0.3.1] - 2020-10-14#
build_number as a tool
replace some prints with logging info and error
[0.3.0] - 2020-10-14#
New repo update tool
[0.2.9] - 2020-10-12#
Added generate_build_number() function
Formats build_number based on Omniverse Flow guidelines.
[0.2.8] - 2020-10-12#
Add
extra_args
to pip_install
[0.2.7] - 2020-10-07#
change default
repo_deps
path to be in _repo/deps
[0.2.6] - 2020-10-06#
publish: fail_if_exist
[0.2.5] - 2020-10-06#
Start producing more labels when uploading
Allow specifying any package in publish tool
[0.2.4] - 2020-10-02#
Fix vendoring if toml happen to be already installed in python’s site-packages
[0.2.3] - 2020-09-18#
Cache getting git branch and hash to optimize as they called many times often
[0.2.2] - 2020-09-18#
Fix git autoinstall from packman
[0.2.1] - 2020-09-18#
Add support for ${file:PATH} syntax in config
Switch to use VERSION file in root
[0.2] - 2020-09-11#
New functionality: ‘repo’ command as a central entry point for all repoman tools. Repoman becomes a framework to register and configure repo tools.
Introducing repo_build_docs and repo_upload builtin tools. Switching to use them.
Bring in repo_fileutils in repo_man
[0.1.22.4] - 2020-07-27#
Fixes
[0.1.22.3] - 2020-07-23#
Removed a print block used for debugging.
[0.1.22.2] - 2020-07-23#
Renamed
teamcity_block
toteamcity_message_block
and changed it so you can pass inname
anddescription
.Started to add teamcity blocks around the git, teamcity api and gitlab api functions.
[0.1.22.1] - 2020-07-22#
Forgot to return the output when
teamcity_block
is used.
[0.1.22] - 2020-07-22#
Continued work on the teamcity blocks started in 0.1.21. You can use the decorator
teamcity_block
to create a teamcity block around a function.Please see teamcity.py for some test examples. Better documentation will happen soon.
[0.1.21] - 2020-07-17#
Added a teamcity module to start collecting up functions related to teamcity usage.
Added
open_teamcity_block
andclose_teamcity_block
which can be used to add better stats in the build log.
[0.1.20] - 2020-07-13#
Added a timeout when running git commands. Timeout default is 60 seconds and retry default is 5 times.
[0.1.19] - 2020-07-02#
Adding a timeout of 5 seconds to the gitlab api call. If it fails, the timeout will increase by 5 seconds for each try.
[0.1.18] - 2020-04-29#
Added a function to access the gitlab api.
gitlab_api(url, data, gitlab_url)
. url is required, data (json only) and gitlab_url are optional.Added a function to access the teamcity api.
teamcity_api(url, data)
. url is required, data (xml only) is optional.
[0.1.17] - 2020-04-24#
Bug fix only.
[0.1.16] - 2020-04-17#
get_path_hash
will return a tuple containing the githash of the path specified and the githash with the MR number if its running as a MR,["<hash>", "<hash>-<mr>"]
. If its not a merge request, then both strings will be the same hash.trigger_teamcity_build
can trigger a teamcity build if following args are passed in. The response code is returned.branch_name
- This is the name of the branch in the repo to build.commit_id
- This is the commit hash of the change to build.build_config_id
- This is the build configuration ID of the job in teamcity. Its found in the general settings for a job underBuild configuration ID
.
[0.1.15] - 2020-04-07#
call_git
andget_git_hash
now accept the argcwd
, which points to the directory that the subprocess running git should run in.
[0.1.14] - 2020-03-11#
When deleting a file we now remove the read-only flag first before attempting the delete
[0.1.13] - 2019-12-17#
Add repo-deps to repo folders
[0.1.12] - 2019-11-27#
Hotfix $lib_prefix in get_platform_file_mapping
[0.1.11] - 2019-11-26#
Add gather_files_from_dict_for_platform, get_platform_os_and_arch, get_platform_file_mapping, gather_files_from_dict
Add few tests for those
[0.1.10] - 2019-11-19#
Add change_syspath
disable pip_install checks explicitly for passed folder only, that doesn’t work well with std library deps.
[0.1.9] - 2019-11-18#
Add change_syspath
pip_install checks explicitly for passed folder only (on import)
[0.1.8] - 2019-11-15#
Add module param to pip_install
[0.1.7] - 2019-11-13#
Fix few bugs, more stable
[0.1.6] - 2019-11-13#
Add change_envvar, docs
[0.1.5] - 2019-11-13#
Restructure internally: split into submodules.
Bring nvenv from packagemaker
Add git status check API and in upload script
Add explicit build_label_filename API
Add package_only flag to package_and_upload
Add basic test
[0.1.4] - 2019-11-06#
Add optional version param to pip_install
[0.1.3] - 2019-11-01#
Add package and upload API and script.
Default logging to stdout (enable_std_logging)
Add optional version param to pip_install
[0.1.0] - 2019-10-24#
First release of the repoman tool, a rebranding of the multitool