[hermes] Add log-router for data plane audit events by notque · Pull Request #11330 · sapcc/helm-charts

notque · 2026-04-08T15:19:40Z

Summary

Adds log-router as a new opt-in component (logRouter.enabled: false by default)
to the hermes chart for routing data plane audit events to customer Ceph/S3 buckets
Consumes from a dedicated dataplane.audit RabbitMQ queue (separate from
the existing notifications.info control plane queue)
PostgreSQL (postgresql-ng) dependency for dynamic per-tenant routing configuration
New audit_admin Keystone role
Phase 1: QA/testing only — validates end-to-end event flow

New Files

templates/log-router-statefulset.yaml — StatefulSet with WAL PVC
templates/log-router-configmap.yaml — Non-secret config + policy.json
templates/log-router-secret.yaml — RabbitMQ, S3, Keystone credentials
templates/log-router-service.yaml — ClusterIP + headless service
templates/log-router-pdb.yaml — PodDisruptionBudget
templates/_log-router-utils.tpl — Image helper + common env vars

Modified Files

Chart.yaml — postgresql-ng dependency, version bump to 0.2.0
values.yaml — logRouter + log-router-postgresql sections
keystone-seed.yaml — audit_admin role (conditional on logRouter.enabled)
ci/test-values.yaml — log-router test overrides

Test plan

helm template renders cleanly with logRouter.enabled: false (no log-router resources)
helm template renders cleanly with logRouter.enabled: true and test values
Verify postgresql-ng secret/service naming matches _utils.tpl references
Deploy to QA and validate end-to-end event flow
Confirm existing hermes components are unaffected

Adds log-router as an opt-in component (disabled by default) for routing data plane audit events from RabbitMQ to customer Ceph/S3 buckets. - StatefulSet with WAL persistence for crash recovery - PostgreSQL via postgresql-ng dependency for tenant configuration - Keystone audit_admin role for config API RBAC - Prometheus metrics scraping on port 8080 - PodDisruptionBudget and headless service for StatefulSet - CI test values with minimal resources

joluc

LGTM

bbobrov

New audit_admin Keystone role scoped to ccadmin/cloud_admin project

Sorry, but this makes no sense. A role is not scoped to anything - it exists by itself. It can be assigned to a user and project, and then the user can get a bearer token on the project with this role, and go to hermes with it.

What you do is probably a rule that utilizes this combination... but to that i have comments inline.

sapcc-bot · 2026-04-14T18:31:07Z

Failed to validate the helm chart. Details. Readme.

The original policy.json restricted all config operations to ccadmin/cloud_admin scope, preventing customers from managing their own data plane event routing. Add project-scoped and domain-scoped rules matching the existing hermes audit_viewer pattern so customers with audit_admin can manage their own config. Remove the cloud_admin project block and all role assignments from the seed — cloud admins get access through cluster policy rules, and the service user doesn't need audit_admin.

notque · 2026-04-14T18:42:26Z

@bbobrov I've attempted to change this to what I've actually intended as we discussed on the call. Tomorrow if you have time, I'd appreciate your consideration on it, and again thank you for giving me a proper check.

bbobrov

from authorization pov this looks now better, thanks

# Conflicts: # openstack/hermes/Chart.lock

sapcc-bot · 2026-05-15T13:50:09Z

Failed to validate the helm chart. Details. Readme.

# Conflicts: # openstack/hermes/Chart.yaml

sapcc-bot · 2026-06-16T21:38:34Z

Failed to validate the helm chart. Details. Readme.

viennaa

Thanks for providing the log-router PR. Couple of remarks before it can go live.

viennaa · 2026-06-17T13:11:07Z

+    log_router: {}
+  users:
+    log_router: {}


according to postgresql-ng they shall not include _. will fail

Suggested change

log_router: {}

users:

log_router: {}

log-router: {}

users:

log-router: {}

viennaa · 2026-06-17T13:16:34Z

+      name: '{{ $.Release.Name }}-pguser-log-router'
+      key: postgres-password
+- name: LOG_ROUTER_DB_URL
+  value: "postgres://log-router:$(LOG_ROUTER_DB_PASSWORD)@{{ $.Release.Name }}-postgresql.{{ $.Release.Namespace }}.svc:5432/log-router?sslmode=disable"


ideally the log-router occurrences of these strings go into values.yaml. if log-router-postgresql values change, this fails silently.

values.yaml: logRouter: db: user: log-router name: log-router

viennaa · 2026-06-17T13:18:03Z

+          emptyDir: {}
+      initContainers:
+        - name: fix-permissions
+          image: {{ required ".Values.init_image is missing" .Values.logRouter.init_image }}


Different values are checked here, is this on purpose?

LeoZhangZXZ · 2026-06-19T11:19:12Z

@viennaa thanks for the review, good catch

sapcc-bot · 2026-06-19T11:20:22Z

Failed to validate the helm chart. Details. Readme.

notque requested review from Kuckkuck, ashifnihalb, ibakshay, joluc, notandy, olandr, richardtief, timojohlo, trouaux and viennaa as code owners April 8, 2026 15:19

joluc previously approved these changes Apr 9, 2026

View reviewed changes

bbobrov requested changes Apr 14, 2026

View reviewed changes

Comment thread openstack/hermes/templates/keystone-seed.yaml Outdated

Comment thread openstack/hermes/templates/keystone-seed.yaml

Comment thread openstack/hermes/templates/keystone-seed.yaml Outdated

bbobrov reviewed Apr 14, 2026

View reviewed changes

Comment thread openstack/hermes/templates/keystone-seed.yaml Outdated

Comment thread openstack/hermes/templates/log-router-configmap.yaml Outdated

notque dismissed joluc’s stale review via acf130d April 14, 2026 18:30

notque force-pushed the hermes_log-router-integration branch from acf130d to 5bc3899 Compare April 14, 2026 18:34

bbobrov previously approved these changes Apr 17, 2026

View reviewed changes

update the log router image

c487cba

LeoZhangZXZ dismissed bbobrov’s stale review via c487cba May 12, 2026 09:24

LeoZhangZXZ self-requested a review as a code owner May 12, 2026 09:24

LeoZhangZXZ added 2 commits May 12, 2026 11:35

update the user name in postgres

79a6049

mount log-router-secrets

fc5d471

LeoZhangZXZ force-pushed the hermes_log-router-integration branch from 81c4a0f to fc5d471 Compare May 12, 2026 11:21

LeoZhangZXZ added 3 commits May 12, 2026 13:26

fix the wal database access issue

de51e98

update the test init container image

50aea94

correct the openstack parameters

575a584

LeoZhangZXZ force-pushed the hermes_log-router-integration branch from 2d60324 to 575a584 Compare May 15, 2026 07:35

LeoZhangZXZ added 2 commits May 15, 2026 15:45

update storage config

fdc5ded

Merge branch 'master' into hermes_log-router-integration

39c6c96

# Conflicts: # openstack/hermes/Chart.lock

update the helm dependieces

3ffb488

LeoZhangZXZ force-pushed the hermes_log-router-integration branch 2 times, most recently from 89fedec to 3ffb488 Compare May 18, 2026 07:20

LeoZhangZXZ added 2 commits May 18, 2026 19:17

remove the test postgrepsql version

4f04aca

logrouter init image

e59eb6e

LeoZhangZXZ force-pushed the hermes_log-router-integration branch 2 times, most recently from ea450ab to e59eb6e Compare May 19, 2026 12:07

LeoZhangZXZ added 3 commits May 19, 2026 14:16

update the postgresql version

de30ea4

remove keystone auth related parameters

35d2915

Merge branch 'master' into hermes_log-router-integration

333a0a4

# Conflicts: # openstack/hermes/Chart.yaml

LeoZhangZXZ previously approved these changes Jun 16, 2026

View reviewed changes

Merge branch 'master' into hermes_log-router-integration

3029c93

notque dismissed LeoZhangZXZ’s stale review via 3029c93 June 16, 2026 21:37

[hermes] helm dep up

fb6cafb

LeoZhangZXZ previously approved these changes Jun 17, 2026

View reviewed changes

LeoZhangZXZ added 2 commits June 17, 2026 10:01

Merge branch 'master' into hermes_log-router-integration

9687763

update the postgrepsql-ng

d3c7b3f

LeoZhangZXZ dismissed their stale review via d3c7b3f June 17, 2026 08:29

LeoZhangZXZ previously approved these changes Jun 17, 2026

View reviewed changes

viennaa reviewed Jun 17, 2026

View reviewed changes

update regarding to the comments

a74331d

LeoZhangZXZ dismissed their stale review via a74331d June 19, 2026 11:10

Merge branch 'master' into hermes_log-router-integration

afa8d03

update test values logrouter image

9e5713c

Conversation

notque commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

New Files

Modified Files

Test plan

Uh oh!

joluc left a comment

Choose a reason for hiding this comment

Uh oh!

bbobrov left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sapcc-bot commented Apr 14, 2026

Uh oh!

notque commented Apr 14, 2026

Uh oh!

bbobrov left a comment

Choose a reason for hiding this comment

Uh oh!

sapcc-bot commented May 15, 2026

Uh oh!

sapcc-bot commented Jun 16, 2026

Uh oh!

viennaa left a comment

Choose a reason for hiding this comment

Uh oh!

viennaa Jun 17, 2026

Choose a reason for hiding this comment

Uh oh!

viennaa Jun 17, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

viennaa Jun 17, 2026

Choose a reason for hiding this comment

Uh oh!

LeoZhangZXZ Jun 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

LeoZhangZXZ commented Jun 19, 2026

Uh oh!

sapcc-bot commented Jun 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

notque commented Apr 8, 2026 •

edited

Loading

bbobrov left a comment •

edited

Loading