An organization-neutral template for operating a hybrid k3s platform with Flux, Kustomize, Helm, and SOPS/age.
It provides host-bootstrap examples, cluster services, reusable application bases, and tenant/project overlays. It is a starting point, not a turnkey production installer: private inventory, credentials, hardware configuration, domains, provider state, and production data belong outside this repository.
Important
Existing installations must follow the
mandatory migration preflight
before changing their Git source or ref. The checked-in suspend and prune
settings cannot make that live transition atomic by themselves.
The template separates three ownership layers:
hosts and private network
│
▼
k3s
│
▼
Flux + Kustomize/Helm + SOPS/age
│
├── cluster services and operators
├── tenant-owned services
└── project workloads
- Host: provider/node interfaces plus generic Ansible and NixOS examples establish machines, k3s membership, private reachability, storage, and node roles.
- Cluster: Flux reconciles networking, certificates, ingress, operators, storage, backup, observability, and other selected platform capabilities.
- Workload: reusable application bases are configured by tenant and project overlays that own domains, persistence, policies, and secret references.
The intended core is k3s, Flux reconciliation, encrypted configuration, a working network, and a selected storage strategy. Most platform modules and all example workloads are optional. See Architecture for the ownership model and Stack parity for the capability inventory.
| Path | Purpose |
|---|---|
infrastructure/, templates/nodes/ |
Provider and node interfaces, including a plan-only Hetzner adapter |
ansible/, nixos-config/ |
Generic host and k3s bootstrap examples |
manifests/clusters/my-cluster/ |
Flux entrypoint and reconciliation dependency graph |
manifests/system/ |
Cluster substrate, operators, and optional platform modules |
manifests/applications/ |
Reusable application bases |
manifests/tenants/my-tenant/ |
Example tenant, company services, storage, runners, and projects |
manifests/shared/templates/ |
Reusable tenant and project building blocks |
docs/ |
Architecture, deployment, secrets, recovery, and component guidance |
my-tenant, example-client, TEMPLATE_*, and CHANGE_ME_* are scaffolding.
Rename or replace them before deployment.
-
Review Architecture and choose the provider, networking, storage, ingress, identity, backup, and workload capabilities the target actually needs. Remove unused modules before enabling them.
-
Use
.env.exampleas a non-secret planning reference. Replace tenant, project, domain, storage, repository, image, load-balancer, DNS, and archive placeholders in deployment-owned configuration. -
Run the read-only marker check:
./reset-to-template.sh --check
-
Generate the deployment's age key, replace the placeholder recipient in
.sops.yamlwith its public recipient, then create each required encrypted Secret from the adjacent*.secret.template.yaml. Never deploy the template or commit plaintext values. Follow Secrets. -
Follow Deployment to prepare hosts, bootstrap Flux, and reconcile only the selected system and tenant components.
A successful local render proves composition, not production readiness. Verify provider access, DNS, storage, backups, restores, and application behavior in the target environment.
This revision is not an unattended in-place upgrade. Before advancing Git, suspend the owning root and tenant Flux Kustomizations, disable pruning on the live tenant object, and verify that live state. Keep tenant reconciliation suspended while retained Secrets and PVCs are inventoried, encrypted replacements are installed, stateful data is migrated, and recovery behavior is proven. Explicitly retire legacy objects only after their data is no longer required.
The complete procedure—including the Matomo filesystem and legacy Hermes data boundaries—is in the deployment preflight.
Never commit credentials, tokens, private keys, provider state, backups, or exported application data. For deployment:
- replace the placeholder age recipient in
.sops.yamlwith the deployment's public recipient; - copy a required
*.secret.template.yamlto*.secret.enc.yaml; - replace every placeholder locally;
- encrypt
dataandstringDatawith SOPS using that public recipient; and - add only the encrypted file to its owning Kustomization.
Keep templates out of deployable resources: lists. Store the age private key
and recovery material outside Git. See Secrets for the full
contract.
Hermes is an opt-in external integration. The separately versioned
org-as-code project owns the operator, CRDs, chart, controller, and runtime
release; this repository owns only signed, immutable consumer wiring and
generic workload declarations.
The operator and example Flux Kustomizations are suspended by default, actuation is disabled, and the example workload remains inactive at zero replicas. Enabling Hermes requires encrypted registry credentials, a review of the published CRDs and managed namespaces, verified release identities, and an explicit decision for every actuation gate. Runtime credentials use typed same-namespace Secret references. See Hermes packaging and rollout.
Run the local baseline before opening a change:
bun run typecheck
bun run lint
bun run format:check
bun run build
bun run test
scripts/ci/render-kustomizations.sh
scripts/ci/check-plaintext-secrets.sh
scripts/ci/check-template-literals.sh
kubectl kustomize manifests/clusters/my-clusterCI additionally validates all YAML, negative secret fixtures, the Hetzner adapter, NixOS configurations, Ansible playbooks, and workflow syntax. A change to one of those areas should run its focused checks locally as well.
- Architecture — layers, ownership, and exclusions.
- Deployment — fresh installs and existing-install migration.
- Secrets — SOPS/age workflow and key custody.
- Stack parity — included and conditional capabilities.
- Hermes — external operator boundary and safe rollout.
- Manifest layout — manifest ownership conventions.
- Contributing — repository contribution guidance.