Administration

General Notes

Almost all aspects of DeepSearch stack are documented via its settings:

  • in the case of helm deployment - .yaml file included in the stack helm chart (see Helm configuration for more details).

  • in the case of docker-compose deployment - (.env) file included in the stack tarball we provide (see Docker compose configuration for more details).

We try to keep our documentation on settings and options as close to the “code” as possible here.

That file should be self explanatory, with settings and comments talking about what they do.

This document should be considered an addendum to information in the values.yaml and .env files respectively.

Monitoring

Monitoring your instances is imperative to understanding the general health of the system and if more resources are necessary. At a minimum, one should monitor:

  • Disk space

  • CPU and LA

  • Memory

Additionally, NGSearch stack itself exposes quite a few metrics about its load characteristics (such as number of assets that awaits processing, number of processed assets, memory and CPU load, etc).

We recommend to take advantage of these metrics. We expose them to be consumable by Prometheus. As usual, the ports for scraping metrics can be found in the Stack`s .env file. We also summarize them here for clarity:

Service name

Prometheus metrics port

NGSearch Search service

8008

NGSearch Storage service

8007

NGSearch Nucleus indexing

8001

NGSearch Tags Crawling

8002

Cache service

8010

Farm Worker service

8011

Non Farm Worker service

8012

Writer service

8013

Monitor service

8014

Model Updater service

8015

Embedding service

8016

For more information about individual services please refer to Components.