Shared authentication mechanism across pyobs web projects (robotic-backend, archive, ...)

[thusser]

Problem

Each pyobs web service currently rolls its own auth, uncoordinated:

• pyobs-archive (pyobs_archive/authentication/backends.py) already delegates to an external OAuth2 provider ("odin"): OAuth2Backend does a password-grant token exchange against settings.OAUTH_CLIENT['TOKEN_URL'], and BearerAuthentication validates bearer tokens against a PROFILE_URL. Successful auth mints/updates a local Django User + Profile (storing the odin access/refresh token).
• pyobs-robotic-backend (settings.py DEFAULT_AUTHENTICATION_CLASSES) uses plain DRF TokenAuthentication + SessionAuthentication against its own local Django user database — no connection to odin or to archive's users at all.

No shared identity: a user (or a service) authenticated against one pyobs web project has no way to be recognized by another. This will only get worse as more web services join the ecosystem.

Question

What should replace this? Some options to weigh:

1. Standardize on the existing odin OAuth2 provider — reuse archive's pattern (OAuth2Backend/BearerAuthentication) in robotic-backend and future services, so there's one identity provider instead of N local user tables. Lowest-effort given odin already exists and is already proven in archive.
2. A dedicated new OAuth/OIDC provider for the pyobs ecosystem — more standard (OIDC discovery, refresh tokens, scopes) but is new infrastructure to stand up and operate.
3. Something else — shared JWT-signing secret/service tokens for service-to-service calls (e.g. Mastermind -> robotic-backend), separate from end-user SSO? These may be two different problems (user login SSO vs. inter-service auth) that shouldn't necessarily share one mechanism.

Scope questions to resolve before implementing

• Is this about user-facing SSO (one login across archive/robotic-backend/future dashboards), service-to-service auth (e.g. pyobs-core modules calling robotic-backend's API), or both?
• Does odin (referenced in archive's code, no design doc found for it) still exist/is still the org's intended identity provider, or was it specific to a past deployment?
• Where should a shared auth client library live — a new pyobs-auth package, or duplicated per-service (current state, what we're trying to get away from)?

[thusser]

Correction on "odin": it's not a bespoke internal identity provider, it's LCO's observation-portal — the same third-party scheduling system pyobs-core's `lco` `ObservationArchive`/`TaskArchive` backends (`pyobs/robotic/storage/lco/`) already integrate with. archive's `OAuth2Backend`/`BearerAuthentication` are authenticating against LCO's portal OAuth2 endpoint, not an org-internal server.

This changes option 1's tradeoff: "standardize on the existing provider" would mean making pyobs's own cross-service auth depend on an external, third-party system we don't control — reasonable only for facilities that already use LCO's portal for scheduling, not a general answer for pyobs sites that don't. Strengthens the case for option 2 (a dedicated pyobs-owned identity provider) as the more broadly applicable direction, with LCO-portal auth staying as one optional backend for sites that want it, similar to how `lco` is already just one of several `ObservationArchive` implementations rather than the default.

[thusser]

Follow-up on option 2 (dedicated pyobs-owned identity provider): without the LCO-compatibility constraint, OAuth2 password-grant + bearer-token + profile-endpoint (LCO's current shape) is not what I'd pick for a new design:

• Password-grant is discouraged for new systems (RFC 9700 / OAuth 2.0 Security BCP) — it hands the raw user password to the client, which defeats delegated auth (no MFA at the IdP, no consent step). It only survives as a legacy option for old first-party clients like LCO's portal.
• Prefer OIDC (Authorization Code + PKCE) for user login across the web frontends — the current standard for SSO, gives an ID token instead of a bespoke profile-endpoint call.
• Prefer JWT access tokens over opaque bearer tokens — each service verifies the signature locally instead of round-tripping to a profile endpoint on every request (what archive's `BearerAuthentication` does today against LCO). Less coupling, less latency, no single point of failure on every API call.
• Service-to-service auth should be a separate mechanism (client-credentials grant or API keys) from human login (e.g. Mastermind calling robotic-backend's API) — different trust model, shouldn't be forced through the same flow as a browser login.
• Lean toward an existing open-source IdP (Keycloak, Authentik, Ory Hydra) rather than hand-writing an OAuth2/OIDC server — token issuance/refresh/revocation/MFA is a lot of security-sensitive code to own; pyobs services would just be OIDC clients instead.

Archive's LCO connection stays as its own separate, opaque-bearer backend either way (per the earlier correction) — this is about what a new, pyobs-owned provider should look like, not a replacement for that.

[thusser]

Which existing IdP: Authentik is the best fit — it's Python/Django itself (matches the rest of the pyobs stack, so it's debuggable/extensible by the same team), deploys as a single self-contained Docker Compose stack (matches how every pyobs repo already deploys), and includes login UI + user management out of the box with solid OIDC/PKCE support.

For contrast:

• Keycloak: more battle-tested, better if SAML/LDAP federation is ever needed, but JVM-based and operationally heavier for a small team.
• Ory Hydra: deliberately headless — only the OAuth2/OIDC protocol layer, no login UI or user store. Would also need Ory Kratos (identity) plus a custom login app. More flexible, much more integration work than "drop it in and get SSO."

Worth checking before building anything, though: this is a university institute, and `pyobs-iagvt` already points at `gitlab.gwdg.de` — does GWDG (Göttingen's central IT/computing service) already run an institutional SSO/OIDC broker to federate against, instead of standing up and operating a new identity provider from scratch?

[thusser]

Two more resolved points from discussion:

Federating to GWDG's IdP doesn't mean automatic access. Authentication (proving identity) and authorization (granting access) are separate concerns:

• If GWDG's IdP exposes group/role claims (LDAP groups / SAML/OIDC group attributes, common for institutional SSO), gate pyobs access on membership in a specific group.
• Otherwise (or as a second layer): first successful login creates a local user with no permissions by default (same pattern archive already uses on first auth — `get_or_create`), and a pyobs admin explicitly grants access after. Login via SSO ≠ automatic access either way.
• Open question: does GWDG's IdP actually expose group/role claims we could use? Determines which of the two is the real mechanism vs. a fallback.

External collaborators won't have GWDG accounts, so GWDG-only federation isn't sufficient on its own. Standard fix: run our own IdP (Authentik) as a broker in front of GWDG rather than relying on GWDG alone — federate upstream to GWDG SSO for internal accounts, and maintain local accounts in Authentik for everyone else (invite-based or self-registration). pyobs services only ever talk to one Authentik OIDC endpoint and get a consistent identity back regardless of which path a user came through; group-based authorization applies uniformly on top since groups live in Authentik, not inherited from GWDG.

This confirms the earlier direction rather than changing it: we want our own IdP either way, with GWDG as one upstream login source feeding into it, not a full replacement for it.