Skip to content

Latest commit

 

History

History

stack

README.md

Observability stack inside FreeUnit

Builds of Prometheus (metrics), Jaeger (traces), VictoriaLogs (logs) and Perses (dashboards) to run as FreeUnit external applications, with the FreeUnit Go module (unit.ListenAndServe). Under FreeUnit (NXT_UNIT_INIT set) each one serves its HTTP API through FreeUnit, otherwise it behaves as upstream. All of them are Apache-2.0; the first two are CNCF graduated projects, Perses is a CNCF project too, and VictoriaLogs is the only one outside the CNCF (the CNCF has no project that stores logs).

Users don't build them: CI builds and releases them and the CLI installs them.

snunit stack install           # downloads them into ~/.cache/snunit/stack (checksums verified)
snunit run --stack App.scala   # run the app with the stack, dashboards on http://localhost:3000

or in the code: Observability("my-service", stack = Some(Stack())), see snunit.config.Stack.

Role (archive and directory) Built from Where
metrics Prometheus http://localhost:9090
traces Jaeger http://localhost:16686 (OTLP/HTTP on 4318, /v1/traces)
logs VictoriaLogs http://localhost:9428/select/vmui
dashboards Perses http://localhost:3000

Ports: the four above and 4318, the OTLP/HTTP receiver of Jaeger, are served through FreeUnit on bindAddress (127.0.0.1 by default). Jaeger gets both through the one handler FreeUnit gives to a program: unitmux/ routes /v1/* to the receiver (a patched copy of the OpenTelemetry collector's, patch-jaeger-otlp.sh) and the rest to the query server. The programs also bind ports of their own on 127.0.0.1 behind FreeUnit (dashboards, traces HTTP and gRPC, logs): the launcher asks for any free port (0), so they can't clash. All of them are fields of snunit.config.Stack; the application keeps port 8080.

The archives, the directories and the executables are named after the role (snunit-stack-metrics-*, snunit-metrics...), not after the project: the Linux Foundation asks not to use its marks in the name of a product, and the same is safer for the others.

Layout

File
versions.env upstream versions of the programs
patches/ the change to Prometheus and Jaeger (its query server)
patch-jaeger-otlp.sh, unitmux/ the change to the OTLP/HTTP receiver Jaeger uses (a copy of the OpenTelemetry collector's) and the Go package that shares the one FreeUnit handler between it and the query server
patch-victorialogs.sh, patch-perses.sh the change to VictoriaLogs (vendored lib/httpserver) and Perses (a copy of perses/common)
build.sh clone, patch, build (including the web UIs), collect the licenses, check them and package one program
collect-licenses.sh, collect-npm-licenses.sh, collect-npm-licenses.js license files of the Go modules and the npm packages in a binary
check-licenses.sh fails on strong copyleft licenses (AGPL, GPL, SSPL, BUSL...)
SOURCE.md.in SNUNIT-SOURCE.md of every archive
RELEASE_NOTES.md text of the GitHub release
LICENSES.md, licenses/ what is distributed under which license and what we do about it

Releasing

  1. Change versions.env and the patches (build.sh <program> builds one locally, see its header for the environment).
  2. Pick a new tag stack-vN, put it in .github/stack-release and in StackRelease of snunit-build-core/SNUnitBuild.scala (CI checks they agree).
  3. Push the tag: .github/workflows/stack-binaries.yml builds every program for Linux x86_64, Linux aarch64, macOS aarch64 and macOS x86_64 and publishes the archives and SHA256SUMS in the release.
  4. Release the CLI (cli-v* tag) to point users to it.

Limits

FreeUnit speaks HTTP/1.1 only, so gRPC doesn't go through it. Each program runs with a single process. The patches have to be redone for new upstream versions. None of the programs is configured with a login: they listen on the loopback interface.