1.14.37

API Server

• [OMFP-xxxx] fixing a race condition in api server related to user metrics

1.14.36

API Server

• [OMFP-4121] update idl.cpp lib with race condition fix (repackaging)

• [OMFP-4124] reduce the frequency of requests to Discovery when looking up Auth service

1.14.35

API Server

• [OMFP-4120] - replace dcb18d6mfegct.cloudfront.net with content-production.omniverse.nvidia.com for /NVIDIA mount folder

• [OMFP-4121] - update idl.cpp lib with race condition fix

1.14.34

LFT server

• [OM-118407] updating Python to fix vulnerability report

1.14.33

API Server

• [OM-99538] Fix a free-after-use bug in the connection library.

1.14.32

• Updated supressions

1.14.31

• [OMFP-4086] meta-upgrade: repair reorder fixFolderChildrenCounters to be after fixVersioning

• Update all aiohttp from (3.8.3/3.6.3/3.8.1) to 3.9.0

LFT:

• Update PyInstaller

1.14.30

API Server

• [OMFP-4040] do not produce checkpoints for non-assets, non-omniobjects during copy/rename

• [OMFP-3826] Remove all remaining carb::dictionary::ItemType::eBool checks before attempting to getAsBool from Verifier

• meta-upgrade: repair now supports removing checkpoints created for channels and associated counters

• NSPECT/Pulse vulnerability scan changes

1.14.29

• Rename user will deny requests to change a non-renamable user or to change a user to a non-renamable user, these values are set in the configuration and have a default list

1.14.28

• Launcher fixes

• Image config fixes

• Metric fix

API Server

• [OM-114394] Allow large object deltas

1.14.27

API Server

• Fixing a race in messaging / channels

• Adding metrics to improve visiblity of the internal scheduler

1.14.26

API Server

• Log Processor: fixed periodic intermittent crashloops when encountering messages with direction set to –.

1.14.25

Connection libraries

• [OMFP-2984/OMFP-3281] - Protect OmniChannelPtr in SendContext by adding refcount

API server

• Fixing mutex scoping for outMsgMtx/mSubscriptionsLock.

• Resolver logging fixes for utf-8

• Minor quality fixes

• [OM-102112] Add process stats command to the service API, set it to trip when we have no available fibers

1.14.24

• [OMFP-2475] - Nucleus Apollo GDPR: Create Secure Port for Bearer Token (3106)

• Add X-Forwarded-For header handling for setting peer IP

• Fix notification order to reduce mutex holding time while notifying subscriptions

• Obfuscate “token”: “” fields in logs

1.14.23

Connection libraries

• Update libcurl to 8.4.0

API server

• [OM-95906] Structured logging

• [OM-109929] Memory leak fix

• [OMFP-1024] s3 bucket authorization is now supported by nginx resolver cache

• [OMFP-1698] messaging and connection leak fix. deadlock resolution

• Use after free bugfix

LFT

• [OMFP-1210] Do not check token validity when calling finalizeAssetCreate

• CVE-2023-4911 - exception added

1.14.22

Connection libraries

• Renamed OMNI_LFT_MTLS_CLIENT_CERTIFICATE/OMNI_LFT_MTLS_CLIENT_PRIVATE_KEY/OMNI_LFT_MTLS_CLIENT_CERTIFICATE_PASS to OMNI_MTLS_CLIENT_CERTIFICATE/OMNI_MTLS_CLIENT_PRIVATE_KEY/OMNI_MTLS_CLIENT_CERTIFICATE_PASS

• [Linux] in PEM format through OMNI_MTLS_CLIENT_CERTIFICATE/OMNI_MTLS_CLIENT_PRIVATE_KEY

• [Windows] in P12 format through OMNI_MTLS_CLIENT_CERTIFICATE/OMNI_MTLS_CLIENT_CERTIFICATE_PASS P12 certificate doesn’t need the private key to be specified separately, but might need a password to be specified (if the certificate was issued with the password)

• [OM-102988] - Bugfix in mtls.toml parsing

API server

• [OM-100900] Fix an issue when a signed URL to an Azure blob was rendered with an invalid file name

1.14.21

API server

• [OM-106509] - Update OpenSSL to version 3.0, curl, LWS

• [OM-95902] - Update zlib

• [OM-96631] rename_user: service call in non-public API to implement a portion of GDPR requirements - this scrubs the meta database and renames all mentions of a specific username

• Fixing nSpect run

• Pulse scanner - added CVEs for GHSA-w596-4wvx-j9j6 and GHSA-xqr8-7jwr-rhp7

• New gitlab build tags

1.14.20

Connection libraries

• [OM-102988] mTLS improvements (supporting client certificate password for websocket connections and mtls.toml lookup)

Verifier

• [OM-104743] option to prevent content check in Azure environment

Docker

• Added metadata snapshotting and restore functionality

1.14.19

Connection libraries

• [OM-74305] better detect certificate store paths on Linux in LFT code path

• [OM-100198] supply JWT to LFT for all requests

• [OM-91120] rename interface macros

  • OMNI_EXPORT -> OMNI_CONNLIB_EXPORT

  • OMNI_ZERO_INIT -> OMNI_CONNLIB_ZERO_INIT

  • OMNI_ABI -> OMNI_CONNLIB_ABI

  • OMNI_DEFAULT_CTOR_ZERO_INITIALIZE -> OMNI_CONNLIB_DEFAULT_CTOR_ZERO_INITIALIZE

  • OMNI_NOEXCEPT -> OMNI_CONNLIB_NOEXCEPT

  • OMNI_CALLBACK_NOEXCEPT -> OMNI_CONNLIB_CALLBACK_NOEXCEPT

API server

• [OM-98423] do not verify JWT token between LFT <-> API

• [OM-100672] fixing deadlock in Subscriptions

Docker

• [OM-95771] change icmp nucleus server check to a tcp connect

1.14.18

Fix

• PULSE_SCAN_PROJECT_VERSION to be the tag

API server

• [OM-98815] Fixed a bug in the task scheduler

1.14.17

• Feature Flag default 0

1.14.16

• Docker

• [OM-96478] improving logging in meta dumper

• [OM-96649] meta dumper memory usage optimization

1.14.15

• Docker

1.14.14

• Converted some meta validation errors from error to warning

1.14.13

• Docker

1.14.12

• Connection libraries

  • [OM-70861] Fixing support of mTLS on Windows through OMNI_LFT_MTLS_CLIENT_CERTIFICATE/OMNI_LFT_MTLS_CLIENT_PRIVATE_KEY/OMNI_LFT_MTLS_CLIENT_CERTIFICATE_PASS environment variables. The client certificate is supposed to be set in PEM format through OMNI_MTLS_CLIENT_CERTIFICATE/OMNI_MTLS_CLIENT_PRIVATE_KEY and:

    • [Linux] in PEM format through OMNI_LFT_MTLS_CLIENT_CERTIFICATE/OMNI_LFT_MTLS_CLIENT_PRIVATE_KEY (note added LFT suffix)

    • [Windows] in P12 format through OMNI_LFT_MTLS_CLIENT_CERTIFICATE/OMNI_LFT_MTLS_CLIENT_CERTIFICATE_PASS (note added LFT suffix). P12 certificate doesn’t need the private key to be specified separately, but might need a password to be specified (if the certificate was issued with the password)

1.14.11

• API server

  • [OM-81351] Fixed retry bug in omniverse_resolver and added new parameter retry_interval into resolver config

  • [OM-80569] add setting to prevent listing child on mounts

1.14.10

• API server

  • [OM-90296] Added OMNI_DELTA_APPLIER_PORT envvar for omni.delta-applier port

  • [OM-87570] added special handling in subscribeList for mounted paths

  • [OM-79192] fix list2 .empty field for list2 call when listing a folder under a mount

  • [OM-77363] Fixing Memory Leak in omnitrace (with update to 1.2 and improving integration)

  • [OM-40736] subscribe_list supports notifications from mounts

  • [OM-79970] make LFT ticket salt configurable

  • [OM-xxxxx] change delete and rename to write ACL

  • [OM-76815] do not parse environment variables into the settings dictionary

  • [OM-76200] disconnect_client service API call

  • [OM-89001] fixing meta dumper hanging on the lock

  • [OM-87570] added special handling in subscribeList for mounted paths

  • [OM-86828] Allow operators to configure Resolver in k8s deployments

• Connection libraries

  • [OM-80070] Send proper close frame when closing connection

• LFT

  • [OM-62627] Updated the python version to 3.10

• Docker

  • [OM-81466] Pulse Docker Image Scan is failing. Changed ubuntu images to refer to ubuntu-18-04-20230208

1.14.9

• API server

  • [OM-84981] implementing user agent feature in auth/authorize_token

  • [OM-77363] Fixing Memory Leak in omnitrace (with update to 1.2 and improving integration)

1.14.8

• API server

  • [OM-79192] fix list2 .empty field for list2 call when listing a folder under a mount

  • [OM-77363] Fixing Memory Leak in omnitrace (with update to 1.2 and improving integration)

  • [OM-79970] make LFT ticket salt configurable

  • [OM-xxxxx] change delete and rename to write ACL

  • [OM-76815] do not parse environment variables into the settings dictionary

• Connection libraries

  • [OM-80070] Send proper close frame when closing connection

1.14.7

TBW

1.14.6

VERSIONING CHANGE: beginning with this release, versioning of Nucleus Core is made compliant with SemVer. Effectively, an extra “dot” is added to separate hundredths from the rest of the version (ie, 114.6 -> 1.14.6).

• Connection libraries

  • [OM-65388] send close reason, log close reason received from the server

114.5

• API server

  • [OM-65386] provide close reason to the client, log close reason from the client

  • [OM-40736] subscribe_list supports notifications from mounts

  • [OM-77363] Fixing Memory Leak in omnitrace (with update to 1.2 and improving integration)

  • [OM-79970] make LFT ticket salt configurable

  • [OM-xxxxx] change delete and rename to write ACL

  • [OM-76815] do not parse environment variables into the settings dictionary

  • [OM-82126] fixing a bug in rename2

• Connection libraries

  • [OM-65388] send close reason, log close reason received from the server

• Docker

  • OpenTelemetry: enabled exporting of collectors’ metrics.

  • Fixing docker build

• Helm

  • CI pipeline and HELM Chart refactored

  • [OM-81466] Pulse Docker Image Scan is failing. Changed ubuntu images to refer to ubuntu-18-04-20230208

114.4

• API server

  • [OM-74550] [fixing a bug] do not expose metrics on workstation

114.3

• API server

  • [OM-74550] do not expose metrics on workstation

114.2

• API server

  • [OM-71089] fixing stat2 not reporting correct mtime for mounted paths

  • [OM-56846] adding database snapshotting support

114.1

• API server

  • [OM-xxxxx] Support for redirection in non-mount asset read requests

  • [OM-xxxxx] Reviewing CVEs

• Docker

  • Log processor: was incorrectly marking API log records as LFT log records in structured logs

114.0

• API server

  • [OM-55159] Bugfix for invalid zero sequence returned in update_object API command

  • [OM-46833][OM-46669] Fixed ‘empty’ field in the message bus event, fixed ‘empty’ field in the list2 response

  • [OM-xxxxx] Speeding up read* family of requests by avoiding reading the meta record twice when possible

  • [OM-31650] Updated idl.py@0.10 with strings verification on terminate symbols

  • [OM-50296] Disabled copying of checkpoints to the destination record while renaming

  • [OM-xxxxx] Support for multipart upload using S3 resolver, S3 resolver updated to version 7.0 and supports signed redirect url

  • [OM-53256] Send CheckpointChanged notification when checkpoint is deleted

  • [OM-xxxxx] copy2 now supports copying from a mount

  • [OM-xxxxx] rename2 and copy2 now support overwriting destination

  • [OM-45634] Updated the omniverse_resolver package. Added logging from resolver. Fixed issues with mounts while listing from root directory

  • [OM-54941] do not prepend checkpoint messages with “[after … ]” or “[before …]”

  • [OM-56379] implementing service_resolve_acl API call

  • [OM-52426] fixing mount/unmount not working with paths containing trailing slashes

  • [OM-52430] Fixing Coverity Bug where a The if condition is the same as the previous if condition

  • [OM-57001] Fixing Resource Leak of File Pointers in b64_encodef Function

  • [OM-54675] Fixing the issue where conversion from const char* as returned by c_str() to std::string creates an unnecessary string copy

  • [OM-53638] Fixing the issue where conversion from const char* as returned by c_str() to std::string creates an unnecessary string copy

  • [OM-53641] Fixing the redundant condition where a condition is always true

  • [OM-52816] Fixing the redundant condition where a condition is always true

  • [OM-54671] Fixing the redundant condition where a condition is always true

  • [OM-xxxxx] Fixing the issue where DeltaManager could add deleted path to ignore list

  • [OM-xxxxx] (Backups/Service interface) Adding ‘checkpointed’ flag to opaque data, provide option to enable/disable ‘checkpointed’ flag restore from opaque

  • [OM-57338] Fixing get_acl_resolved bug

  • [OM-60834] Skipped the ‘stat’ call for the search service while resolving records acls

  • [OM-62431] Restrict rename from operating on the same path as both source and destination

  • [OM-64266] Make server allow receiving empty chunk in LFT endpoint only if it is the only one chunk.

• Connection libraries

  • [OM-xxxxx] Send ‘Multipart-Chunk-Size’ header in PUT requests during upload

  • [OM-31650] Updated idl.py@0.10 with strings verification on terminate symbols Updated legacy client library to verify all input strings on terminate symbols

  • [OM-46639] Fixing a bug related to retry logic in the C++ connection library

  • [OM-44892] Support X-Cache header in RemoteDownload (Python connection library)

  • [OM-xxxxx] Fixes for RemoteDownload (incorrect ‘Host’ header, download if size is unknown)

• LFT

  • [OM-56966] Fixed the issue when LFT returns a HTTP500 error if trying to download an empty file

  • [OM-60294][OM-60877] CentOS 7: Fixing LFT server failure due to resolver and grpc modules conflict

  • [OM-51916] Send ‘Content-Length’ header from ‘/path2’ endpoint

  • [OM-45634] Updated the omniverse_resolver package. Added logging from resolver

  • [OM-52956] Allow a request task to be cancelled.

  • [OM-64266] Don’t send hash for not-first empty chunk

113.31

• API server

  • [OM-64266] Make server allow receiving empty chunk in LFT endpoint only if it is the only one chunk.

• Docker

  • [OM-63427] MonPX rewritten as aiohttp server in attempt to resolve hangups and following disapperance of metrics

113.30

• CI pipelines / compliance for Docker images

• Docker

  • Reconfigured Stack to use host mapped ports instead of overlay for Swarm

• Connection libraries

  • [FDN-211] OpenTelemetry interop with Nucleus and LFT

113.29

• Docker

  • Log Processor: fixed processing of response sizes recorded as ‘N/A’

113.28

• API server

  • [OM-53641] Fixing the redundant condition where a condition is always true

  • [OM-52816] Fixing the redundant condition where a condition is always true

  • [OM-54671] Fixing the redundant condition where a condition is always true

• LFT

  • Added support for instance name logging

  • Implemented logging response size properly

  • Bugfix: was not logging number of bytes requested correctly

• Helm

  • [OM-xxxxx] Add ingress toggles for lft, api with default enabled=true; add to values-template

• Docker

  • Log Processor: recovered metrics that went missing in 113.25

113.27

• API server:

  • [OM-44328] Bump rename version to 1 in omni1.idl.ts to indicate that server has bugfix

• LFT Server:

  • [FDN-233] OpenTelemetry tracing

• Helm:

  • Tracing configuration

  • http ingress support besides https

113.26

• Connection libraries

  • [OM-55823] fixing LFT buffer overflow in LftGetContentRequest::proceed

113.25

• Docker

  • Added support for log-processor to emit structured JSON logs

  • Added support for controlling default directories’ set

  • Added support for disabling default mounts

113.24

• Connection libraries

  • [OM-46639] fixing assert in LftGetContentRequest::proceed

113.23

• LFT server

  • Adding missing Python module in the container

113.22

• API server

  • Re-introduced custom mount resolvers support in Launcher that was accidentally “lost” post 112.4. This fixes OM-50072

113.21

• API server

  • [FDN-211] Tracing instrumentation for Opentelemetry interoperability

113.20

• LFT server

  • [OM-49456] Used cryptography manylinux package to avoid GLIBC_2.18 dependency on CentOS

113.19

• API server

  • [OM-xxxxx] fixing PLC issues, updating libcurl, zlib and openssl, resolvers 6.0

• LFT server

  • [OM-xxxxx] use Python 3.8.13 and resolvers 6.0

113.18

• API server

  • [OM-xxxxx] removed (introduced in 113.16): extending subscribe_list with an optional ‘recursive’ argument. Instead service_subscribe_list function is added with no arguments.

113.17

• Infrastructure scripts update due to password rotation

113.16

• API server

  • [OM-xxxxx] Adding configurable quotas for list & read subscriptions

  • [OM-xxxxx] extending subscribe_list with an optional ‘recursive’ argument available only for superusers

  • [OM-xxxxx] Updated libcurl to version 7.82.0 (brotli compression is no longer supported) and zlib to version 1.2.12

• Connection libraries

  • [OM-xxxxx] Updated libcurl to version 7.82.0 (brotli compression is no longer supported) and zlib to version 1.2.12

113.15

• LFT server

  • [OM-45575] Fixed access log encoding (default is utf-8, configurable using ‘OMNI_LFT_ACCESS_LOG_ENCODING’ env.var.)

113.14

• API server

  • [OM-xxxxx] Fixed access to destructed object in DeltaManager

  • [OM-xxxxx] Fixed access to destructed requests scheduler, which could happen during shutdown

113.13

• Helm

  • Changed env vars workflow

• API server

  • [OM-xxxxx] NVBUG3509359: use openssl built with no-autoload-config to avoid binaries looking for a random path where a config could be found

• Connection libraries

  • [OM-xxxxx] NVBUG3509359: use openssl built with no-autoload-config to avoid binaries looking for a random path where a config could be found

  • [OM-45887] Enable extra logging for SSL errors

113.12

• API server

  • [OM-xxxxx] Removed PBR discovery address

113.11

• Helm

  • [OM-xxxxx] Make smoketest.backoff_limit configurable

113.10

• Helm

  • [OM-xxxxx] Added “test_discover_all” test to smoketest post-install job

  • Added “test_discover_all” test to smoketest post-install job

113.9

• API server

  • [OM-44328] Fix path collision checking in rename request

  • [OM-xxxxx] fix for a bug causing incorrect folder children counts (cherry picked from MR 2099)

113.8

• Helm

  • [OM-xxxxx] Moved meta-dumper to nucleus-api pod

113.7

• Helm

  • [OM-xxxxx] Added upgrade/verify init containers

  • [OM-xxxxx] Added post-install smoketest

  • [OM-xxxxx] Made nucleus_secret name configurable

113.6

• API server

  • [OM-43083] Do not send CheckpointsChanged event in subscribe_list to clients which don’t support subscribe_list:1 and higher capability

113.5

• Helm

  • [OM-xxxxx] Added common labels

  • [OM-xxxxx] Added long_version annotation

113.4

• Helm

  • [OM-xxxxx] Added proxy-buffering annotations

  • [OM-xxxxx] Added prometheus pod discovery annotations

  • [OM-xxxxx] Added testsuite image and stress test

  • [OM-xxxxx] Fixed ingress class handling

• Docker

  • [OM-xxxxx] Added nucleus-testsuite image config for stress testing

113.3

• Helm

  • [OM-xxxxx] Fix publish scripts

113.2

• [OM-xxxxx] declare “stop” capability (version 0) when registering in the Discovery service

• [OM-xxxxx] fix “auth” setting group name collision in omni.server.app config .json files, one group is renamed to “auth_service_client”

• Docker

  • [OM-xxxxx] lft-lb: tweaked nginx parameters to support large uploads more optimally

113.1

• [empty release] to trigger CI correctly

113.0

• API server

  • [OM-xxxxx] optimized omni.objects update & KeyValueDB transaction setup

  • [OM-38586] Expose request queue and latency metrics

  • [OM-xxxxx] Simplifying list implementation, stopping support for wildcards anywhere but the end of the path

  • [OM-40482] Implemented get_acl_resolved API call

  • [OM-38765] only rename event should come to clients supporting it

  • [OM-33101] Send CheckpointsChanged event when checkpoint is created

  • [OM-xxxxx] Fix meta record with unflushed content repair routine

  • [OM-41769] added missing list subscription event on object update

• LFT server

  • [OM-xxxxx] support /path2/ endpoint for downloading a file in a branch or a checkpoint of a file

• API client

  • [FDN-142] [OM-30451] [backported from 114.0] Send stop command when the AsyncIterator in call_many commands is closed.

112.5

• Cherry-picked hot-fixes from 111.8

112.4

• API server

  • [OM-35591] atomic checkpointing of assets and omni-objects

  • [OM-xxxxx] supporting Redis as key-value storage

  • [OM-41769] added missing list subscription event on object update

• API client

  • [FDN-142] [OM-30451] Send stop command when the AsyncIterator in call_many commands is closed.

112.3

• 111.7 patch was pulled into 112.x line

• API server

  • [OM-xxxxx] fixing an issue where closeConnection was called twice during disconnect

• Docker

  • Added lft-lb: an NGINX-based load balancer in front of LFT services

  • Allowed whitelisting custom mount resolvers in nucleus-api via OMNI_ALLOWED_MOUNT_RESOLVERS env var

112.0

• API server

  • [OM-38293] output logs to stderr

  • [OM-35288] adding deep_copy_object_struct command

  • [OM-33094] Use omni1.idl.ts instead of core.idl.ts, fixes in omni1.idl.ts

  • [OM-30334] Implemented ping2 command in the service interface

  • [OM-34044] fixed an stack overflow issue happenning in the legacy wildcard code

  • [OM-35419] Extended access log with new information

  • [OM-34185] Fixing Python connection library total_bytes argument being non-optional in LFTUpload interface

  • [OM-31965] Used idl.py@0.7 package version

  • [OM-36424] Make delete/delete2 delete HEAD only when all checkpoints are deleted, set_acl_v2 can set ACLs only on HEAD, all checkpoints get ACLs same as in HEAD

  • fixed bugs in branch list management, checkpoint message deletion, fix setting rollback path in create/copy with overwrite

  • updated openssl to 1.1.1l and libcurl to 7.78.0

  • [OM-27140] Implemented support for passing user data to mount using Auth service’s user store.

  • [OM-39139] Added –lft-disable-compression option to Nucleus docker scripts

  • [OM-27540] Support for OMNI_DEFAULT_S3_PROXY_CACHE environment variable

• Connection libraries

  • [OM-36524] Avoid using the cache when accessing localhost

  • [OM-31965] Used idl.py@0.7 package version

  • [OM-27140] Updated connection library to 11.15 (update Python connection lib’s RemoteDownload to send Connection-Signature header when available, FileTransfer passes connection id signature on download when url begins with LFT address)

• Bugfixes

  • [OM-36781] Used libwebsockets v4.2.2-17-g14b610c0 package version with fixed pong payload issue

  • [OM-21145] Used idl.py@0.7 8533e18e package version with fixed close frame issue

  • [OM-37920] Cancel LFT RemoteDownload pending tasks in case of error/cancellation

111.8

• Fixed exceptions’ reporting in nucleus-meta-dumper

111.7

• Docker

  • Now, launching “nucleus-api” requires SECURITY_REVIEWED env var set to 1

111.6

• API server

  • [OM-38293] [fixed] output logs to stderr

  • [OM-34667] [fixed?] fix non-printable symbols in TSV

111.5

• API server

  • [OM-36424] [fixed] Deleting files does not delete checkpoints; causing some empty dirs to fail deletion with FOLDER_NOT_EMPTY while they are empty

  • [OM-38134] [fixed] setting new ACLs through set_acl_v2 using either HEAD or any of the checkpoints sets ACLs for all of them”

  • Log processor: now sets a debug copy of a log it’s parsing aside for later debug purposes in event of a crash

111.4

• API server

  • [OM-xxxxx] adding missing licenses

111.3

• API server

  • [OM-37492] disabling new style log system

  • [OM-36781] Used libwebsockets b824504c package version with fixed pong payload issue

  • [OM-xxxxx] api server: meta-upgrade - fixing an issue when paths without a history (such as a channel) weren’t upgraded (110->111)

  • [OM-xxxxx] api server: meta-upgrade - v2->v3 DB upgrade incorrect optional field handling when upgrading mounts

  • [OM-xxxxx] api server: meta-upgrade - fixing an issue while upgrading 109->110, functions to correctly create a key in the DB given

  • [OM-xxxxx] fixing freeze scripts

  • [OM-xxxxx] updating idl (ability to deserialize non-positive floats)

111.2

• API server

  • [OM-32581] In-pipeline vulnerability scans, updating packages

• Connection libraries

  • [OM-34185] Fixing Python connection library total_bytes argument being non-optional in LFTUpload interface

• Meta dumper

  • [OM-36418] Added removing json dump from previous call

  • [OM-31965] Used idl.py@0.7 package version

111.1

• Docker

  • Tweaked path exclusion regexp to correctly exclude strings with \n and \r

111.0

• API server

  • Use OpenSSL 1.1.1i

  • [OM-34044]: fixed an stack overflow issue happenning in the legacy wildcard code

  • [OM-34853]: list2 command supports optional ‘show_hidden’ flag

  • [OM-30574]: register Nucleus in the Discovery

  • [OM-28547]: support remote server shutdown through the service interface

  • [OM-24981]: Server provides special python module for omni.objects tests

  • [OM-24594]: Server ownes omni.objects delta schema and publishes special package ‘omniverse.nucleus-objects.schema’ with this schema and related include files

  • fixing contention issue in authentication related API calls (same user connecting concurrently, say tens of connections at the same time, may see INTERNAL_ERROR on some of them)

  • [OM-24700]: implementing subscribe_read_asset command

  • [OM-27106]: upgrading key space in meta db, optimizing recursive operations

  • [OM-23438]: migrated omniobject business logic into omni.objects plugin

  • [OM-20491]: adding subscribe_list command

  • [OM-20475]: adding create_object command

  • [OM-20476]: adding create_asset command

  • [OM-20477]: adding update_object command

  • [OM-20478]: adding update_asset command

  • [OM-24976]: adding subscribe_read_object command

  • [OM-22912]: [refactoring] porting list command to the new code path

  • [OM-25250]: [refactoring] porting read command to the new code path

  • [OM-24711]: [refactoring] changing list2 & stat command signatures to better support versioning

  • [OM-24492]: [refactoring] porting update command to the new code path

  • [OM-23921]: [refactoring] porting create command to the new code path

  • [OM-22480]: [refactoring] porting delete command to the new code path

  • [OM-23852]: [refactoring] Porting messaging/channels to the new code path

  • [OM-23144]: [refactoring] Porting copy comand to the new code path

  • [OM-22366]: [refactoring] websocketslib to support per-listener and per-connection notification callbacks

  • [OM-23843]: Updated the upload_packages.py script to only include what’s needed (both Windows and Linux)

  • [OM-23844]: Updated premake script to not put static libraries and such into the final build directory

  • [OM-21524]: Improved e2e testsuite execution: include more tests to run in parallel

  • [OM-23846]: Used nopy USD libraries for the delta USD plugin

  • [OM-10390]: Instrument ICore

  • [OM-25101]: Stripped artifact’s binaries for linux platform

  • [OM-25079]: [refactoring] Move LFT interface to omni.core

  • [OM-21815]: Added ability to query locked status by using stat/list2 requests. Added generation of unique omniobject ID by using uuid

  • [OM-22692]: idl.cpp should allow to generate C++ code using std::optional

  • [OM-25981]: [refactoring] omni.core: Added content flushing task

  • [OM-26615]: Added verbose valgrind logging and suppression’s rules to reject unnecessary output for correct memory leaks detecting

  • [OM-26513]: [refactoring] Ported mount/unmount/getMountInfo to omni.core, implemented mount cache in omni.core

  • [OM-20193]: Updated crashreporter for uploading minidumps to internal backtrace io

  • [OM-27832]: Idl.Cpp: Reactor should use poll instead of select to support any values of descriptors

  • [OM-27945]: Libcurl >1024 file descriptor support

  • [OM-22047]: Added the CloudEvent-compliant API logging subsystem

  • [OM-27051]: API server can now launch the LFT server directly

  • [OM-23239]: omni.verifier, omni.meta-upgrade tools report back the directories it’s scrubbing

  • [OM-28590]: Added possibility to measure stress tests duration, fail them if they are longer than a per-test limit

  • [OM-24140]: Stage 2 for omni.objects: support for ext. values with dedup, packed multi-part format & multi-part LFT read

  • [OM-21145]: Improved scanPage function: walking through metadb became faster

  • [OM-21145]: Stop request can interrupt list request execution

  • [OM-29538]: Make e2e server tests wait for the API server to start by trying to establish a connection in lieu of a fixed delay

  • [OM-22366]: Ported DeltaManager to omni.core. The new code is using fibers and accessing the meta db directly with transactions.

  • [OM-31141]: backup_asset() function now returns hash information

  • [OM-31143]: backup_asset() function now returns size

  • [OM-xxxxx]: adding omni_objects2 call to the server features interface

  • [OM-xxxxx]: support meta database repair in the meta-upgrade module

  • [OM-xxxxx]: checkpoints can be obliterated by users with write permission, checkpoints are created automatically with write permission preserved (the checkpoint is still immutable)

  • [OM-21815]: The List/SubscribeList requests have lock info in the response structures

  • [OM-xxxxx]: fixing a bug where old clients would get a COPY event with destination field which they don’t know about. Also avoiding doubling the COPY event by only matching the destination, not source.

  • [OM-26684]: use resolvers which are compatible with HTTPS Nucleus Cache

  • [OM-31658]: Stop request can interrupt list2 request execution

  • [OM-xxxxx]: Default ACLs are configurable

  • [OM-32548]: API acls are configurable

  • [OM-32743]: adding ‘max_full_path_length’ parameter to control maximum full path length

  • [OM-29552]: Changing default ACLs for paths

  • Bugfixes:

    • spec: ping response ‘version’ & ‘ts’ fields are now optional to avoid breaking backwards compatibility with 107 servers

    • [PSA-1687]: [fixed] security review

    • [OM-30714]: fixing a bug in legacy delete implemenation, doing folder children counter fix during meta upgrade to 111 (db v.5)

    • [OM-28226]: make sure super users always see admin/write/read permissions when doing list/list2/stat

    • [OM-23240]: omni.verifier does not work when paths are specified in a relative manner

    • [OM-21118]: PathEvent doesn’t contain locked/unlocked events

    • [OM-21623]: api_server should never send responses on “stop” command

    • [OM-26234]: Resolved Ctrl+C non-clean exit

    • [OM-27863]: Fixed legacy server mount record generation: use encoded ‘path’ in mount record

• Connection libraries

  • Use OpenSSL 1.1.1i

  • [OM-24964]: connection library: add a test for LFT/non-LFT + cache/no-cache headers

  • [OM-25180]: omni1ExtOp commands should be generated

  • [OM-25719]: ConnLib fails if allowed null callback is passed

  • [OM-25722]: idl.cpp connlib protocol should log json that is failed to be deserialized successfully

  • [OM-25772]: ConnLib hangs when omniStop is called from omniAuthorize/omniAuthorizeToken callback

  • [OM-27493]: Crash when omitting omniClientShutdown

  • [OM-26681]: Secure transport

  • [OM-28728]: ConnLib: LFT logic sometimes does not report error

  • [OM-28724]: ConnLib: implement LFT retry logic

  • [OM-28041]: ConnLib: server in OmniAuth*Settings may have path

  • [OM-28235]: Implement support for connecting to host:port/path in C/C++ Clients for Nucleus

  • [OM-28967]: ConnLib: Problems when omni1 objects are removed from their callbacks

  • [OM-28497]: Improve omni and omniIDL interfaces

  • [OM-28981]: connlib: support proxy management in IDL version of the connlib

  • [OM-29054]: ConnLib: Remove python2 support

  • [OM-28982]: connlib: support channels/messaging in the IDL version of the connlib

  • [OM-29444]: ConnLib: Generate C++ wrapper over omni1.h, service.h and omni1ext.h

  • [OM-29661]: ConnLib: Support for https cache (proxy) servers added. A cache server must be trusted, it sees the unencrypted traffic!

  • [OM-xxxxx]: adding omni_objects2 call to the server features interface

  • Using idl.cpp 0.15

  • [OM-29070]: More documentation for IDL functions

• Docker

  • Changed resolver-cache image from squid to nucleus-cache

  • Added enable-test-config option for deploying nucleus api image with test config

  • Number of deployed LFTs is now configurable

  • [OM-31589] now prohibiting path elements that consist of . and ..

  • Added options and hookup to deploy with PBR and register in Discovery

  • Message bus is disabled by default

  • Switched nucleus-api, nucleus-dump-processor, nucleus-lft, nucleus-log-processor images to approved ubuntu image

  • Added repair-meta-db.sh tool into Core container

  • Switched meta-dumper to idl.py==0.6

  • Added PACKAGE-LICENSES to meta-dumper, log-processor and utl-monpx

110.8

• API server

  • Increased timeout for stress tests in testharness.py

110.7

• API server

  • OM-30821: [fixed] deleting a checkpoint shouldn’t generate a list notification

  • OM-30773: [fixed] do not create checkpoint if latest state is already checkpointed

110.6

• API server

  • OM-xxxxx: legacy implementation of copy command didn’t reset rollback path and checkpointed fields

110.5

• API server

  • OM-30714: fixing a bug legacy delete implemenation, where folder children counter may become corrupted after deleting a folder with files

110.4

• Docker Tooling

  • Added Meta Dumper sidecar

  • Exposed Service API

110.3

• API server

  • enabling versioning by default

• Connection library

  • Fixed usage of deleted ConnectionInformation

110.2

• API server

  • [OM-28238]: better support path based routing for lft_address configuration field

  • [OM-25729, OM-29230] * Make the default timeout for delta manager process startup larger

  • Fixed bug in tasking (access to member of potentially destroyed object)

  • Add missing template parameter to ResponseHanlderData (ApiRouter.cpp)

• Connection library

  • Fixing backwards compatibility break with 107 servers in ping command response

  • [OM-29194]: ServerFeatures interface extended with versioning() command to check if versioning is on

• Docker

  • Added nucleus-dump-meta image

  • Log processor: now, reports a note (instead of an error) when the source log isn’t there

110.1

• Docker

  • Updated CI-related tooling to generate proper branch names

110.0

• API server

  • Fixing concurrency issues in the legacy code path, when sometimes orphaned paths could appear

  • Content deduplication (stage 1)

    • Upon first server startup existing content is deduplicated

    • Upon uploading content that server already contains, the content is reused and not duplicated

    • It’s possible to create / update a file by simply providing a hash of content

    • create_asset_with_hash/update_asset_with_hash calls

  • Versioning support (stage 1)

    • [OM-18969]: [added] get_branches call

    • [OM-18970]: [added] get_checkpoints call

    • [OM-18968]: [added] replace_branch

    • [OM-18967]: [added] checkpoint_version call

  • [OM-1776]: [added] rename call

  • [OM-19265]: [added] list2 call

  • [OM-13634]: [added] stat2 call

  • [OM-20538]: [added] delete2 call

  • [OM-20703]: [added] copy2 call

  • [OM-18615]: [added] create_directory call

  • [OM-12059]: [added] get_acl2/set_acl2 calls (now it’s possible to provide multiple paths at once)

  • [OM-18149]: [added] lock, unlock, get_acl, set_acl migrated to the new code path, legacy code is deprecated (will be removed in the next release)

  • [OM-18108]: [added] test tokens support

  • [OM-17756]: using faster regex library internally within API server

  • Removing legacy implementation of all user management API calls, ping and get_transaction_id

  • Implementing new ServerFeatures interface to query server-side configuration parameters

  • Make sure static C++ runtime is used on Windows and Linux

  • Using latest omnitrace v0.5

  • Use carb.launcher v0.2 plugin to kill child process (delta server) automatically and avoid keeping it hanging

  • using carb.tasking plugin that is asan-aware on Linux

  • Disabling API server console input by default

  • [OM-9621]: use latest carb.logging, supporting log append (instead of overwrite)

  • [OM-17326]: better handling for long-running connections (problems with LFT access)

  • [OM-20500]: use breakpad on windows and linux builds

  • [OM-17646]: Handling non-ascii / non-utf8 paths

  • [OM-21126]: respond with NOT_OBJECT instead of DENIED when trying to create a usd/usda/usdc file when omniobject_mode: true

  • Removing omniobject_mode

  • Updating license automation

  • bind to localhost for prometheus metrics by default

  • delta server: bind to localhost by default

  • omni.meta-upgrade: support ‘enabled’ config option

  • [OM-13587] Using idl.cpp 0.14

  • Bugfixes:

    • [OM-9621]: [fixed] server purges its log files upon restart

    • [OM-10230]: [fixed] api server, supporting * as host name for LFT address - bugfix

    • [OM-17001]: [fixed] not able to grant write access

    • [OM-17597]: [fixed] Permissions not inherited from directory

    • [OM-17644]: [fixed] Unable to list folders with special characters in their names.

    • [OM-17807]: [fixed] Admin role not recognized on login

    • [OM-18557]: [fixed] simplify meta db storage for mounts

    • [OM-19061]: [fixed] Nucleus Server returns INVALID_PARAMETERS for content ID without uploaded content

    • [OM-19565]: [fixed] fix path existence check

    • [OM-19679]: [fixed] create(overwrite=true) doesn’t work with LFT enabled

    • [OM-19764]: [fixed] access denied for user on localhost server

    • [OM-19765]: [fixed] no user folder created on localhost server when new user is registered

    • [OM-20471, OM-20787]: [fixed] rename bugfixes

    • [OM-18253]: [fixed] convertation of all timestamps to uint64 numbers (internal meta db upgrade)

    • [OM-18287]: [fixed] revive omnitrace coverage in the API server

    • [OM-19804]: [fixed] delta server sometimes can be terminated without it actually hanging

    • [OM-22103]: [fixed] client capabilities check

    • [OM-22105]: [fixed] created timestamp invalidated by delta manager

    • [OM-22111]: Fixed omni1.idl.ts and core.idl.ts interface/protocol description to provide backward compatibility between API server and Connection library up to 107.6 release.

    • [OM-22425]: Fixing confusing message about failing to delete access log by checking whether the log file exists in the first place

    • [OM-22589]: remove breakpad module and using it from Carbonite

    • [OM-19183]: Fixing default permissions for /Users/ folder to be read-only for users

    • [OM-23913]: [fixed] Mechanism to remove renewed files and directory records from the ignore list did not work correctly.

    • [OM-23703]: [fixed] Fixed signal handler to use only signal-safe functions.

    • [OM-23705]: [fixed] Fixed methods versions in omni1.idl.ts and core.idl.ts

    • [OM-19183] Fixing default permissions for /Users/ folder to be read-only for users

    • [OM-22829] Fixing incorrect behavior when omni object created with “overwrite=true”

    • [OM-23913]: [fixed] Mechanism to remove renewed files and directory records from the ignore list did not work correctly.

    • [OM-23703]: [fixed] Fixed signal handler to use only signal-safe functions.

    • [OM-23705]: [fixed] Fixed methods versions in omni1.idl.ts and core.idl.ts

    • [OM-27321]: [fixed] Fixed removal of needsdm flag in the meta db, when deleting files

• Connection libraries

  • [OM-25719]: ConnLib fails if allowed null callback is passed

• Docker

  • Tweaked Launcher to not require server CL on start

  • Added hookups for data conversion/import process

  • Tweaked release-nucleus.py to use repo tags

  • OmniFlow-compliant naming of artifacts

  • Dockertools supporting BuildKit added

109.9

• Docker / Launcher

  • [OM-25462] Fixed a bug with Launcher’s default configuration preventing creation of paths containing ,

109.8

• API server

  • [OM-24673] Upgrade to omni-config-cpp 2.0.1

• Connection library

  • [OM-24673] Upgrade to omni-config-cpp 2.0.1

• Bugfixes

  • [OM-23462] Pull dependencies through internal instance of Artifactory

  • [OM-23905] Fixing authentication failure when tracing is enabled on the client

• LFT server

  • Cleaning up access logging

  • [OM-23101] Fixing LFT server performance regression

  • [OM-24385] LFT server: try to extract missing headers from the request by using other headers / remote IP address

• Tooling

  • omni.verifier improvements (reporting unserialized meta records, support exit_on_failure options)

  • omni.meta-upgrade changes (upgrading meta db to version 4)

• Connection libraries

  • Bugfixes (OM-17003)

  • Fixing concurrency issue in LFT code

  • Move user callbacks from under the lock to avoid possible deadlocks

  • Removing legacy file-based cache managed by the connection library

  • Adding TOKEN_EXPIRED error code in token-based authentication

  • [OM-18419]: Supporting OMNI_CONN_CACHE env.var to simplify using the new HTTP cache

  • Deprecating omniAuth, OmniAuthSettings, OmniAuthTokenSettings::cacheLocation, OmniAuthTokenSettings::enableCaching

  • Updating IDL internal implementation

  • Supporting separate interface called “ServerFeatures” meant to provide an API to query server features (omniobjects, lft, hash information, etc)

  • Functions to compute hash of content (see create_asset_with_hash/update_asset_with_hash calls)

  • Fixed lock/unlock event names

• Docker Tooling

  • [No Jira Ticket] Modified CI pipeline and tooling to match OmniFlow versioning, and cleaned up ports allocations

  • [No Jira Ticket] Modified Launcher, tooling to have an option to upgrade Meta DB and exit

  • [No Jira Ticket] Switched Delta Server control tool to new Service API

  • [No Jira Ticket] Switched all Python-based tooling to Python 3.8

  • [OM-8332] Added PACKAGE-LICENSES to lft server image config and licenses.sh script for generating licenses file

109.7

• Bugfixes:

  • [PSA-1687]: [fixed] security review

109.6

• API server

  • [OM-19764] Fixed auth command to add new user to ‘users’ group if that user was added to some other group before first login

  • [OM-19765] Fixed auth to create default folder for new user which was added to some group before first login

  • [OM-19804] Improved connection to delta server process: several attempts are performed. Number of attempts and time interval between them can be configured

  • [OM-20195] Fixed copy() command to create missing directories on copy()-destination path

  • [OM-19912] Fixed data corruption due to lack of thread safety during legacy server meta cache update

• Connection library

  • [OM-20244] Fixed LFT failure due to missing LFT pools cleaning on shutdown

109.5

• API server

  • [OM-19787] Fixed data corruption caused by LFT garbage collection

  • [OM-19565] Fixed path existance check in Create command to allow creation of missing folders

  • [OM-19912] Improved transaction usage and legacy server locking in setAclV2 implementation in omni.core.

109.4

• Connection library

  • [OM-17646] Added UTF-8 checks for input strings

  • [OM-16634] Updated idl.cpp library to avoid hang of connection library due to deadlock

109.3

• API server

  • [No Jira Ticket] Disabled input system

  • [No Jira Ticket] Fixed API server hang due to infinite loop

• LFT server

  • [No Jira Ticket] Cleaned up access logging

• Docker

  • [No Jira Ticket] Added squid to stack config as nucleus-resolver-cache, modified proxy healthcheck (check_px) from launcher

  • [No Jira Ticket] Added star and backslash to the list of disallowed path chars by default

  • [No Jira Ticket] Added support for configuring OO mode (on/off)

  • [No Jira Ticket] Fixed bugs in the stack config. It wasn’t properly mount the Resolver Cache data volume

  • [No Jira Ticket] Removed nginx as nucleus-resolver-cache from image and stack configs as well as nginx prometheus exporter

109.2

• API server

  • [No Jira Ticket] Fixed server’s incorrect exit code value. Now server returns proper exit code when it couldn’t startup

  • [No Jira Ticket] Fixed API server threading code to prevent possible crashes and data corruption

109.1

• API server

  • [OM-18366] Fixed incorrect records during meta upgrade

  • [No Jira Ticket] Removed legacy users

109.0

• API server

  • [No Jira Ticket] Added internal documentation in the code

  • [No Jira Ticket] Added allowance for home directory existence before new user creation

  • [OM-8215] Added support for omniobject cache to enable LFT / cache compatibility

  • [OM-5999] Added server notifications

  • [OM-8969] Added new set_path_options API call that allows to change created/modified/created_by/modified_by properties for a path

  • [OM-12059] Added new get_acl, set_acl API calls for omni.core plugin. Now vectorized forms of these calls are supported through the new auto-generated connection library. Old (scalar) calls are converted to the new form (i.e. an array of size 1)

  • [No Jira Ticket] Added carb.launcher 0.2 version to automatically terminate child process (delta server)

  • [OM-13227] Considered logged in user as an admin in case auth token specifies it

  • [No Jira Ticket] Fixed content refcount issue leading to some pieces of content never being removed

  • [No Jira Ticket] Filtered out non essential ‘list’ event for the message bus and list subscriptions

  • [OM-12617] Fixed access logging

  • [OM-12297] Properly stream a data that will be compressed by the delta process instead of sending it in one giant chunk

  • [No Jira Ticket] Memory management bugfixes: new[]/free mismatch, removing unnecessary heap allocations, fixing heap use after free, etc

  • [OM-10588] Replaced C assert with a special OMNI_ASSERT macro which converts assertion into error messages in the log in release build and behaves like assert in debug build

  • [OM-11850] Updated LOCK command. Now it supports etag as an optional argument

  • [OM-10257] Updated CREATE command. Now it supports ‘overwrite’ flag as an optional argument

  • [No Jira Ticket] Updated to omnitrace v0.4

• Misc

  • [OM-15078] Changed all OV_ environment variables prefixes to OMNI_

• LFT server

  • [OM-8215] Added omniobject cache support

  • [OM-14527] Added proper expiration of a content being uploaded through LFT. Fixes an issue leading to pieces of content never being deleted in case file upload was not finalized through the API server

  • [No Jira Ticket] Updated to the latest IDL for interface between API and LFT servers

• Tooling

  • [OM-14890] Added omni.verifier improvements: reporting unserialized meta records, added exit_on_failure options

• Connection libraries

  • [OM-6457] Added raising an exception in case connection is not established in python

  • [OM-9047] Added compression improvements

  • [OM-8969] Added new set_path_options API call that allows to change created/modified/created_by/modified_by properties for a path

  • [No Jira Ticket] Exported reactor functionality through the interface to allow integrating with connection library reactor

  • [No Jira Ticket] Removed init.py in python versions of all libraries

  • [OM-10257] Updated CREATE command. Now it supports ‘overwrite’ flag as an optional argument

  • [No Jira Ticket] Updated to omnitrace v0.4

• Docker

  • [No Jira Ticket] Added ASAN options for special address sanitizing builds

  • [No Jira Ticket] Configured the server to disable terminal control chars in paths

108.1

• API server:

  • Fixing code generation issue (build issue only)

108.0

• API server:

  • Metrics improvements

  • Introducing omni.messaging, the interface to the message bus

    • Publish messages about events hapenning in the Omniverse path space

  • Introducing omni.core, fiber-based server

    • Moving user-group management, get_transaction_id, ping, authorization under omni.core

  • Automatically generating meta data serialization code from spec

  • Automatically generating API serialization code from spec

  • Breaking ApiServer.cpp into a command per file

  • Supporting omniobjects mode (see docs)

  • Numerous bugfixes

  • Asynchronous key-value database interface, introducing transactions, basing all new code in omni.core on transactions. Fiber wrappers for callback interfaces.

  • Introducing forward compatibility feature.

    • From this release onward the server will not decline connections from newer connection libraries. Instead clients are expected to check capabilities reported by the server.

  • Fixing config file collision between omni.server and omni.delta-server

  • Supporting meta database upgrade (both in omni.server.app and with a separate tool omni.meta-upgrade.app)

  • Fixing LIST behavior regression

  • Fixing api log regression

  • Supporting copy with 0 transaction id (it’ll use latest)

  • Split delta process communication into blocks of 128Mb

  • Support ‘allowed_users_legacy’ server setting allowing to limit users from using legacy authentication

  • Fixing UPDATE regression (locked updates do not accept non-empty etags)

  • Pseudo accounts disabled by default

  • omni.websockets: increasing maximum length of HTTP headers

• Misc

  • Env vars renamed to use OMNI_ prefix, and naming generally cleaned up

  • Removed Docker Images’ docs and Kube samples

• Tooling

  • Separate tool omni.meta-upgrade.app to upgrade meta database from previous installation / deployment

  • log_processor: bugfix: archiving logs subprocess could lead to logs’ corruption and log data loss

• Connection library

  • Bugfixes

  • Use portable toolchains to build linux x86-64 and aarch64 versions of the connection library

  • Follow redirects when accessing external links

  • Update to latest curl

  • Support encryption and compression (zlib, brotli)

  • Autogenerate new connection libraries for py/js/C++, the current manually implemented library is still supported but is planned for gradual deprecation

  • Fixing time handling in Python bindings

  • Request compressed encoding when accessing external HTTP(S) resources

  • Use the latest omnitrace API

  • Add some more instrumentation to trace commands end-to-end

  • Allows for update-to-readsubscription (replication) latency measurements without server instrumentation

  • Deprecating legacy authentication

• Docker Hookups

  • Added necessary code to configure Redis for messagebus

  • Added prometheus exporter for Redis messagebus

  • Core API image now exposes an env var for setting a Master password

• Misc

  • Env vars renamed to use OMNI_ prefix, and naming generally cleaned up

  • Removed Docker Images’ docs and Kube samples

107.11

• Connection library

  • Use the latest omnitrace API

  • Add some more instrumentation to trace commands end-to-end

  • Allows for update-to-readsubscription (replication) latency measurements without server instrumentation

107.10

• API server:

  • Cherry-picked “fix delta content process config collision with api server”

107.9

• Connection library:

  • Exclude unreachable proxy servers when a new connection is created

  • Adding omniIsLocalCacheAvailable function

  • Set OmniReadResult::cached to true when the content was delivered through the shared cache (new cache)

107.8

• Hang detection logic

  • Switched configuration parameters from int to float to allow setting smaller values than 1 for the bandwidth estimates.

107.7

• Docker Hookups

  • Configured to allow file mounts from within mnt subdir of the instance’s data dir

107.6

• API server:

  • Fixed data corruption issues related to incorrect refcounts

  • Fixed issues with communicating to Delta Server due to http_proxy env variable being set for S3 resolver to cache files’ metadata

  • Use S3 resolver which support timeout

  • Fixed mount overwrite bug, where it would be possible to remove content with no regard to content refcount from the content storage by copying a file over from a mount.

  • Supporting delta server metrics making it possible to monitor its performance

  • Ability to change master password (for ‘omniverse’ user) from settings

  • Support “allowed user” feature, a list of users allowed to authorize with the server

  • Make connection failure to delta compressing process a hard stop for the API server

  • Detect if a command has taken too long to respond and, if so, remove it

  • Fixing incorrect behavior in LIST

  • Support deep copy of zero-length files from a mount

  • Update delta format plugins to use latest USD

  • Fixing garbage collection of re-created folders

  • Record delta updates in the access log

  • Try to repair LevelDB database before giving up

• LFT server:

  • Raise exception instead of returning it (bugfix)

• Docker Hookups

  • Now using nginx-prometheus-exporter from docker hub rather than nvcr.io

  • Launcher modified to export number of files from Delta Server’s Path Blacklist

  • utl-metricpx renamed utl-monpx and can now show Delta Server’s blacklist at /ds_blacklist URL

  • Added a simple script, dev-deploy.py, to stand up dev instances

• Tooling

  • nvidia.omniverse.server.control: added commands for Delta Server blacklist management

  • Added ds_control.py command line tool

  • Added ws_command.py command line tool to talk to the Server’s Control Channel

• Connection library

  • Fixing LFT route choice (only when LFT is accessible) when uploading content using streaming

107.5

• API server:

  • Support emergency password recovery procedure (add ‘@’ in front of the username to reset password)

  • Fixed bug where path names were being truncated at 512 bytes in length

  • Bugfixes for possible issues initializing metadata

  • config for prometheus - use integers where needed

  • Added a configuration option which lets mount read and list commands run in concurrently. Disabled by default, but to enable it just set “concurrent_mount_commands” to true in the JSON configuration file for the server.

  • Updated delta format plugins to nv-usd and usd_ext 19.11 (newer build, OM-8789)

  • Fixed race condition in omni.eventsystem

  • Load resolver settings correctly - do not use string pairs as this loses JSON typing

  • Optimize requests to resolver (do a single stat if possible and do stat + list if necessary instead of doing stat + list always)

  • Updated Linux build rules - cleanups, preventing from depending on host libstdc++

• Connection library:

  • Lock Python GIL when omniReleaseConnection callback is called

  • Checks for Python reactor on execute

  • Fixed time handling in Python ConnWrapper

• Docker Hookups

  • Resolver-cache added to stack

107.4

• API server:

  • Only use omniobjects if omniobjects library was suscessfully loaded (resolves issue with the index service crashing the server on Centos 7)

• Docker hookups:

  • Lowered default LFT threshold to 256K

  • Added a “simplified” deploy option to exclude extra infrastructure services

107.3

• API server:

  • Adding ‘asset_converter’ to the default user list

107.2

• API server:

  • Fixing config mount issue (OM-7829)

• Connection library:

  • Load redirections even if LFT server is not available (for Cloudfront redirections)

107.1.9

• API server:

  • Updating delta manager plugins

  • Use jemalloc on Linux

  • Do not block during lengthy delete/list operations

  • Updating deployment infrastructure

• Connection library:

  • Updating omni trace library

107.1.8

• Default mounts

  • Changed to use omniverse-content-production bucket and have a root at /NVIDIA

107.1.8

• API server:

  • Fixing a bug with updating meta data for just a folder, it was not preserving the ‘created_by’, ‘modified_by’ etc. fields.

  • Using libcurl with a license (built according to the licensing requirements)

• Connection library

  • Fixing LFT buffer overflow reporting bug

  • Follow redirects when accessing external links

107.1.7

• API server:

  • mounts/allowed_mount_resolver is a strict whitelist now

  • Fix for read notifications delivery delay

  • Fix for possible (not seen previously) issue where some read notifications may be skippped

  • Fix for crash during reading content during client disconnect

  • Enforce mounts read-only

  • Remove mounts coming from config file during startup before adding a new set of mounts specified in the config file

  • Delta manager optimizations

  • Adding ‘admin’ user (same as ‘omniverse’)

• Connection library:

  • Fixing incorrect behavior when LFT content buffer data overflow happens

  • 9.10: support omniGetServer, omniGetToken, omniGetUser to clone a connection

107.1.6

• API server:

  • Support mounts/default_mounts option (specifying this would create mounts on startup automatically)

  • Support mounts/allowed_mount_resolver option (specyfing this would prevent users from mounting using certain resolvers)

  • Support mounts/allowed_file_resolver_root option (specifying this would prevent users from mounting path with certain prefixes)

• Docker hookups:

  • Added support for default, “out of the box” mounts to deployed servers

107.1.5

• API server:

  • Support predefined mounts in the configuration file

  • Removing old unnecessary code

107.1.4

• API server / Delta server:

  • Correctly handle “bye” command and wakeup the server upon receiving SIGINT/SIGTERM

107.1.3

• API server:

  • Updated delta format plugins to nv-usd and usd_ext 19.11

• Connection library:

  • Use newer curl (support windows ssl certificates)

• Infrastructure:

  • Updating core dump processor

107.1.2

• API server:

  • Fixes to better handle filters during list subscriptions

• Connection library:

  • Follow redirections when accessing external links (like cloudfront)

107.1.1

• API server:

  • Fixes for “support list subscriptions to nonexistent paths”

• Docker hookups:

  • Added nucleus-dumps-processor service to push coredumps to Loki

107.1

• Connection library:

  • Support list subscriptions to nonexistent paths

  • OMNI_CONN_REDIRECTION_DICT now expects API server port/address

  • v9.9 - More utility functions (omniGetServerStatus, omniRegisterConnectionDestructionCallback)

  • v9.10 - Connection is refcountable. For each omniGetConnectionWithId a corresponding omniReleaseConnection call is expected.

• LFT server changes:

  • Better handle Content-Range header to be compatible nginx

• Monitoring:

  • Improved logging in metricpx

  • Improved core dump automation

• API server:

  • Using gcc 8.2.0 toolchain

  • Support list subscriptions to nonexistent paths

  • Mount fixes

  • statically link C++ runtime in all modules

  • use ld script for all modules to reduce export surface

107.0

• Repository changes:

  • removed copy_server

  • removed api_server

    • kept docs and tests and move them into api_server_cpp

    • kept scripts and move them into server_tools

  • removed delta_server

  • removed api_server_cpp/ext

  • removed common/templates/python/python_exe_with_cmdargs.py

  • removed tools

  • moved ov-tools into separate repo, omnitools

  • moved ov-dash into omniops repo, tools/ dir

  • renamed api_server_cpp to api_server

  • updated .clang_format for api_server and api_client to be inline with Carbonite

  • made .editorconfig common for both api_server and api_client

• API server changes:

  • [?] deprecated OmniverseClient. Instead use connection library.

  • moved sources around to adhere to a typical Carbonite project structure.

    • created omni.server module, containing all of the existing API server code

    • renamed OmniverseServer -> omni.server

    • renamed DeltaContentServer -> omni.delta-server

    • moved api_server/plugins to api_server/src/plugins/non-carb.plugins to separate them from real Carbonite plugins.

      • later these plugins will become real Carbonite plugins and we will move them top api_server/plugins/omni.*

    • moved api_server/source into api_server/src/plugins/omni.server

    • moved and organize tests. Those tests which measure performance are under api_server/tests/performance/

    • moved existing server tests to api_server/tests/e2e - end-to-end tests

    • moved private server headers from include/ into src/plugins/omni.server along with the sources

      • include/ now is only supposed to contain public headers, i.e. headers used by other applications or modules

  • Supporting mounts feature

• LFT server changes:

  • Supporting mounts feature

• Connection library

  • v9.8 - Deprecated explicit ‘statusDescription’ from each result, added ‘omniGetStatusDescription’ funciton

  • v9.7 - Supporting OmniAuthResult::isSuperuser property

  • v9.6 - Python bindings use logTarget instead of logCallback in init settings.

  • v9.5 - added kOmniErrorTypeLftCouldntConnect, kOmniErrorTypeLftProxyCouldntConnect

  • v9.4 - mount does not use server_redirect_url (options are used instead).

  • v9.3 - list command supports showHidden option (false by default).

  • v9.2 - adds support for omni objects

• Docker hookups

  • Reworked to support the above

  • Removed special promtail image - Promtail now comes standard as a part of Docker tooling

  • loki-url deploy option renamed to loki

  • omni-instance deploy option renamed to instance

  • Renamed all artifacts to have new Nucleus notation

• Docker docs

  • kube.md now out of date, renamed to _kube.md to indicate that

  • images.md updated to reflect reality (mainly, naming updates)

106.4

• Connection library

  • Support subscription to nonexistent paths

  • 9.1b: support omniGetServer, omniGetToken, omniGetUser to clone a connection

• API server

  • Support subscription to nonexistent paths

  • Fixed verifyPermissions crash

106.3

• Docker hookups

  • Added promtail log collector

  • Added option for loki-url to stack-config

106.2

• Connection library:

  • Fixing LFT <-> reactor interaction that leads to hanging LFT downloads

106.1

• Core Server

  • Handle empty uri in LIST

106.0

• Core Server

  • User/group management

  • Libwebsockets transport enabled by default

  • Delta compaction using nv-usd 19.07

• Connection library:

  • Version 9.1:

    • PEP8 compliance

    • Reorganization of the folder structure of the library

    • User/group management

    • Fixing omniStop not being able to stop omniAuthorize

105.5 hotfix

• Docker hookups

  • When using Stack Config for deployment, newly deployed servers now get a Marker file showing which server and version this is

• Core Server

  • Backported LIST / show_hidden option

• Connection library

  • Backported LIST / show_hidden option and bumped the version to 8.8

105.4 hotfix

• Docker hookups

  • Ditched most of the legacy cpp notations and moved files around to match

  • New service names:

    • core-api remained

    • core-lft remained

    • log-processor renamed core-log-processor

    • metricpx renamed utl-metricpx

  • Renamed image configs to match service names

  • Added a lot of ENV vars for config options for sidecar images (log processor, metricpx, et al)

  • Docs for images and how to get them in docker/doc

105.3 hotfix

• Core Server

  • Fixed Access.log to not include INFO as Command

  • Instead, INFOs include -- as their “direction”.

• Log Processor / Metrics Generator

  • Updated to handle the above

105.2 hotfix

• LFT server

  • Fixing HEAD handler (should fix omnicache)

  • Fixing logging for HEAD request

• Connection library:

  • Fixing py2/py3 wrapper for delete - changing stop condition to prevent client hangs

• API server:

  • Internal code changes that shouldn’t affect he API - reorganizing code that gets server time for all responses.

~2019.5~ 105.1

• Versioning

  • Versioning changed to YYY.ZZ where YYY is sequential version number, starting with 100, and ZZ is patch.

  • This version is assigned version 105.1 to maintain continuity. Next version will be 106.0

• Architecture

  • LFT added to the Server

• Connection Lib

  • 07/11/2019 : OM-4110 : Fix a stale char ptr : ssantoleri/ishutov

  • 07/20/2019 : Fixing connection library request id ordering

• API server changes

  • 06/26/2019 : OM-3100 : Refactored the code base to use RapidJSON for all JSON processing : jratcliff

  • 06/26/2019 : OM-3256 : Fixed bug if commands were issued for files that don’t exist : jratcliff

  • 06/26/2019 : OM-3190 : Added a debug string in ‘MessagePacket’ indicating which record it refers to : jratcliff

  • 06/26/2019 : OM-3232 : Defer deleting the UserAccountInstance until the next pump cycle : jratcliff

  • 06/26/2019 : OM-2856 : You can disable password checking by passing: “-disablepasswords true” : jratcliff

  • 06/27/2019 : OM-3084 : Implemented timestamps for update/read subscription responses : jratcliff

  • 06/27/2019 : OM-3167 : existsRecord method was too slow, sped it up. : jratcliff

  • 06/27/2019 : OM-3575 : Calling ‘flush’ after ever write to a record : jratcliff

  • 07/04/2019 : OM-3113 : Store delta manager repro cases and black list in data dir. Prettyprint json. : sduthaler

  • 07/04/2019 : OM-3114 : ApplyDelta() is run in a separate process to handle crashes and hangs. : sduthaler

  • 07/09/2019 : OM-3535 : Added transaction id to message result for channel messages : brianh

  • 07/11/2019 : OM-3096 : CPP server now using an ‘event based’ API for websockets : jratcliff

  • 07/11/2019 : OM-3096 : Command line argument: -websockets specifies which websockets plugin : jratcliff

  • 07/11/2019 : OM-3096 : Integrated libwebsockets dependencies into the libwebsockets plugin : jratcliff

  • 07/11/2019 : OM-4142 : Implemented the ‘createrandom’ command which runs the create benchmark but in a randomized folder location : jratcliff

  • 07/12/2019 : OM-4142 : Added the ‘-contentflushtime’ command line option, specifies time in seconds before flushing content records. Default value is 10 seconds. : jratcliff

  • 07/12/2019 : OM-4173 : Due to this bug, had to use standard OS file find to fix delete records on startup issue : jratcliff

  • 07/12/2019 : OM-4174 : Created ‘deltatest’ and ‘deltatestlocked’ benchmark commands to diagnose delta server bugs : jratcliff

  • 07/15/2019 : OM-4175 : Hash computation no longer asynchronous

  • 07/19/2019 : OM-0000 : Restored the framed message send with flood control code : jratcliff

• Docker and Devops Tooling

  • Service name for the API server changed from ov-cpp to core-api

  • Stack name changed to ov-core

  • Image config added and stack config updated to support LFT

  • Fixed RAM footprint control of API server to work again

  • Log processor and stats generator updated to support a couple more stats for API Server, and LFT server support added to it.

  • Core API server now launched via the launcher

  • Now, metrics are exposed by containers individually and aggregated via a new service, ‘metricpx’

2019.4

NOTE: this release was backed out from the release candidate server, so the next release will contain the same changes and some more.

• Connection Lib

  • 07/11/2019 : OM-4110 : Fix a stale char ptr : ssantoleri/ishutov

• API server changes

  • 06/26/2019 : OM-3100 : Refactored the code base to use RapidJSON for all JSON processing : jratcliff

  • 06/26/2019 : OM-3256 : Fixed bug if commands were issued for files that don’t exist : jratcliff

  • 06/26/2019 : OM-3190 : Added a debug string in ‘MessagePacket’ indicating which record it refers to : jratcliff

  • 06/26/2019 : OM-3232 : Defer deleting the UserAccountInstance until the next pump cycle : jratcliff

  • 06/26/2019 : OM-2856 : You can disable password checking by passing: “-disablepasswords true” : jratcliff

  • 06/27/2019 : OM-3084 : Implemented timestamps for update/read subscription responses : jratcliff

  • 06/27/2019 : OM-3167 : existsRecord method was too slow, sped it up. : jratcliff

  • 06/27/2019 : OM-3575 : Calling ‘flush’ after ever write to a record : jratcliff

  • 07/04/2019 : OM-3113 : Store delta manager repro cases and black list in data dir. Prettyprint json. : sduthaler

  • 07/04/2019 : OM-3114 : ApplyDelta() is run in a separate process to handle crashes and hangs. : sduthaler

  • 07/09/2019 : OM-3535 : Added transaction id to message result for channel messages : brianh

2019.3

• Deploy tooling

  • Added hookups to ensure that release versions are returned by deployed stacks

• OV CLT Tools

  • sync_tree.py - added ability to use e-tags to prevent repeat downloads

• API server changes

  • 06/07/2019 : OM-0000 : Integrated in-memory keyvalue database plugin : brianh

  • 06/07/2019 : OM-2857 : Fixed bug when downloading mount files over ov-web : jratcliff

  • 06/08/2019 : OM-2867 : Fixed ACL permissions bug after omni-web rename : jratcliff

  • 06/10/2019 : OM-2659 : Moved the DeltaContent operation into a separate process : jratcliff

  • 06/10/2019 : OM-2885 : Integrated the Carbonite process ‘Launcher’ plugin : jratcliff

  • 06/10/2019 : OM-2891 : Partial integration of shared memory IPC library into a plugin : jratcliff

  • 06/11/2019 : OM-2855 : Reduced the CPU consumption of the server : jratcliff

  • 06/12/2019 : OM-2905 : List doesn’t report ‘admin’ folders using an exclusion list : jratcliff

  • 06/13/2019 : OM-0000 : Disabled launching DeltaContentServer as a separate process temporarily : jratcliff

  • 06/13/2019 : OM-2949 : Fixed bug when a copy command referenced a non-existent source diretory : jratcliff

  • 06/13/2019 : OM-2949 : Added a new debug log channel ‘metadirectory’ for virtual file system table : jratcliff

  • 06/13/2019 : OM-2949 : Fixed bug where ‘-logdir’ wasn’t being honored at startup : jratcliff

  • 06/13/2019 : OM-2949 : Fixed a bug with the OmniResolver list command, because the callback can happen immediately : jratcliff

  • 06/13/2019 : OM-2942 : Added a command line option: “-sequentialcommands” to diagnose this bug : jratcliff

  • 06/14/2019 : OM-2964 : Renamed CopyServer to CopyManager and DeltaServer to DeltaManager : jratcliff

  • 06/18/2019 : OM-2995 : Put mount specifications in a separate config file ‘mounts.json’ : jratcliff

  • 06/18/2019 : OM-0000 : Fixed a bug with the configuration code not properly handling -datadir : jratcliff

  • 06/18/2019 : OM-0000 : Fixed a bug in the delta manager, not properly detecting locked records : jratcliff

  • 06/18/2019 : OM-2999 : Implemented a content validation system; to make sure all content matches meta: jratcliff

  • 06/19/2019 : OM-3041 : Added: -contentkeyvaluedatabase <true/false> Option to use leveldb for content: jratcliff

  • 06/20/2019 : OM-3070 : Treat user ‘omniverse’ as a super-user : jratcliff

  • 06/20/2019 : OM-3030 : Normalized record names in the code base (internal cleanup) : jratcliff

  • 06/20/2019 : OM-3077 : Refactored KeyValueDatabase to support instancing, also database logging : jratcliff

  • 06/21/2019 : OM-3085 : Treat all users in the ‘gm’ group as super-users : jratcliff

  • 06/21/2019 : OM-3084 : Implemented timestamps for pings (not update or read events yet) : jratcliff

2019.2:

Deployment tools

• Added a quick tool to easily build and deploy server (wrapper around build.py/deploy.py)

• Embedded YYYY.## style versioning into stack config

2019.1:

Note: this is the first ever release notes we check in; and I had to reconstruct a lot of it from commit messages. Please don’t consider this 100% accurate or full, but an early exercise – Fedor ‘Fidot’ Fomichev

Server

• Added symbols to the build

• Removed obsolete data and meta backends

• Lots of cleanup, refactor, housekeeping

  • OM-2685 (rename KeyValueDatabaseAsync -> KeyValueDatabase)

  • OM-2683 (remove content cache)

  • Memory tracking disabled by default

• Bugfixes

  • OM-2735 (LevelDB crashes on some CPUs because ‘Snappy’ is using a privileged insruction)

  • List subscription bug

• Delta Service

  • Enabled storing repro cases in case delta server plugins crash

Client (Conn Lib)

• Connection lib

  • Logging fixes

  • Crash fixes (OM-2706)

  • “Fixed hangup in fetch”

  • nvidia.omniverse.client (Python wrapper)

    • added get_acl()

Installer

• Now using commint installer template

Tools

• Cache tool packaging fixes

• Sync tools (sync_tree.py, upload_tree.py), chmod.py, ls.py added