feat: port entity schema to gen-schema#563
Draft
sini wants to merge 37 commits into
Draft
Conversation
sini
force-pushed
the
feat/entity-gen-schema-port
branch
from
6574fac to
a370d30
Compare
sini
force-pushed
the
feat/entity-gen-schema-port
branch
4 times, most recently
from
d14dc0d to
dabc9d5
Compare
Nested aspects from freeform traversal (e.g., den.aspects.disk.zfs-disk-single) have __provider set by aspectContentType.merge but lack name/meta. Use __provider to derive the path key, matching how the pathSet stores these entries. (cherry picked from commit 5a3bf90)
- has-aspect.nix: accept refs with __provider (set by aspectContentType) - types.nix: annotate nested attrset children in content merger with __provider so deeply nested aspects carry provenance - Only annotate unregistered keys (skip class/pipe/structural keys) - Tests: nested present/absent, provenance distinct, deeply nested (3 levels) (cherry picked from commit aa11ce8)
aspectContentType's multi-def branch forwarded sub-keys with a shallow
`//`, so when several files each contribute a different child under the
same deeply-nested namespace, all but the last were dropped from
navigation (e.g. services/network/cilium/{cilium,hubble-ui,
cilium-bgp-resources}.nix all defining children of network.cilium).
Deep-merge instead: colliding attrsets recurse and colliding lists
concatenate (matching den's own merge semantics); scalars keep
last-def-wins, with __contentValues remaining the canonical source for
emit/forward collection. Adds a deadbugs regression test.
Full suite 860/861; the one failure (issue-583) is pre-existing
nixpkgs drift (nixpkgs now declares programs.atuin.flags, colliding with
the test's mock module) and fails identically at HEAD.
(cherry picked from commit ca4569a)
wrapChild only injected identity from __provider for content wrappers carrying __contentValues; a single-def navigated nested aspect carries __provider (its full path) but no __contentValues, so it fell through nameless and children.nix renamed it to <parent>/<anon>:<idx>. That gave the same nested aspect a different identity depending on the inclusion path — apps.gaming.steam reached via roles.gaming (host scope) vs. a per-user entity-named aspect's includes (applied by a policy at user scope) — defeating cross-scope dedup, so its nixos content (programs.steam.package) was defined twice. Derive name + meta.provider from __provider whenever a navigated child has no name. Adds a cross-scope dedup regression test. Full suite 861/862 (only the pre-existing nixpkgs issue-583). (cherry picked from commit beb2b49)
…tuin.flags The issue-583 forwarding test mocked options.programs.atuin.flags. nixpkgs gained that option in rev 64c08a7 (CI lock bumped in 4701e77), so when the denful#583 fix landed on this branch the mock redeclared an option nixpkgs already owns -> "option programs.atuin.flags is already declared" -> the test failed (it passed in the PR's original, older-nixpkgs context). Forward into a custom `forwardTarget` option instead, so the mock can't collide with nixpkgs. Behaviour and intent unchanged; full suite now 862/862. (cherry picked from commit a2cb870)
The host-aspects battery re-resolved the host aspect tree for a user's
classes (homeManager) in an isolated sub-pipeline seeded with only
{ host, user }, dropping the ancestor context the host scope actually
carries (e.g. a parent `environment` entity). A parametric host quirk
emit `{ environment, host, ... }: ...` was then stranded as a raw
function at the {host,user} projection scope, crashing any homeManager
consumer that read the pipe ("expected a set but found a function").
from-host now fires as a policy (receiving the full resolveCtx) and
threads the ambient entity-kind chain bindings into the re-resolution,
so re-fired parametric host aspects bind the same args they would at the
host scope. The same threading is applied to home-env's userForward
extraction path.
Adds deadbugs/host-aspects-chain-ctx regression test.
(cherry picked from commit eea3d6b)
sini
force-pushed
the
feat/entity-gen-schema-port
branch
from
eea3d6b to
407bd03
Compare
Replaces hand-rolled schemaEntryType with gen-schema mkSchemaOption. Sidecars: includes, excludes. Computed: isEntity (structural content only). Extracts resolvedCtxModule (id_hash, resolved, collisionPolicy) to _types.nix for entity type reuse. collisionPolicy flows through deferred module merge to entity instances (not a sidecar) preserving existing ctx.host.collisionPolicy resolution path.
den.hosts now accepts both forms:
- Legacy: den.hosts.x86_64-linux.igloo = { ... }
- Flat: den.hosts.igloo = { system = "x86_64-linux"; ... }
The outer option type uses a permissive submodule with deepMergeAttrs
freeformType (lib.recursiveUpdate-based merge that avoids the infinite
recursion lib.types.anything causes with cross-option references).
The apply function preprocesses flat entries into two-level form and
re-evaluates through the original attrsOf systemType, so all 6
consumers see the canonical { system.name = hostConfig } shape.
Same pattern as den.hosts: deepMergeAttrs + preprocessHosts + apply. Cross-entity host lookup and osConfig injection preserved.
Covers: id_hash, freeform, topology, meta introspection, isEntity computed, schema includes sidecar.
Update flake inputs and references to match the renamed repo at github:sini/gen-schema.
gen-schema flattened _meta into _-prefixed options and renamed sidecars → collections. nix-effects changed bindAttrs so true is a literal param, not an optionality marker — translate __args values to fx.bind.optionalArg before bind.fn.
sini
force-pushed
the
feat/entity-gen-schema-port
branch
from
407bd03 to
56bbc59
Compare
…tting-node A general primitive — `spawnNode` — materializes a child resolution node from any parent scope, threaded with the parent pipeline's resolved scope-tree state (parent + siblings), so the node's own assemblePipes re-derives inherited/ collected pipe values with full fleet visibility. Paired with resolve-at-emitting- node: a pipeline-parametric pipe emit resolves to concrete data at its emitting node on every crossing (local/collected/exposed), never as a function; config-dependent emits stay deferred (__configThunk). Home extraction is the first and driving consumer: it replaces three isolated sub-pipelines (host-aspects resolveImports; makeHomeEnv/hm-host resolveEntity) with spawnNode, fixing the originating bug where a host-aspects-projected ssh app saw only the local host, never the fleet peers. The mechanism is general (den-hoag `spawn` with one read-only inherited edge) and applies to any parent->child entity relationship; home is just where it is currently exercised. - assemble-pipes: resolve-at-emitting-node on the collected and exposed crossings. - spawn-node: spawnNode primitive (re-walk for one class, merge parent state, own assemblePipes, class isolation). - policy.spawn effect + register-spawn handler + drain augmentation: a deferred node spawn resolved post-walk. - route: forward source resolves via spawnNode (from = parent scope); drop the chainCtx workaround in home-env.nix. - Includes the scopeParent-walk pipe inheritance keeper. - Tests: all-peers, resolved-users, in-tree==threaded equivalency, server-host membership. Full CI 870/870.
The late-sibling dispatch re-fired every policy registered at the parent OR any sibling scope at every sibling (entity-kind filter only). So a policy a user registered via its own includes — opting into the host-aspects battery, a per-user `to-users` policy, etc. — fanned to every other user on the host, regardless of opt-in. Make eligibility ancestor-or-self: at each sibling, only policies registered at the parent (the host, whose subtree spans every user) plus the sibling's own fire. A user's runtime includes stay in its own subtree; host-registered provides still fan to all users (the legacy mutual-provider pattern). Fixes host-aspects projecting a host's homeManager onto users who never included the battery (a pre-existing leak, not from the spawnNode work). host-aspects.nix is unchanged — this is a dispatch fix, not a per-battery guard. Tests: host-aspects-sibling-leak (regression guard); user-host-mutual-config re-patterns the user->siblings case to host-level registration and adds test-user-include-stays-in-subtree.
Expose entity.aspects alongside entity.hasAspect: the flat list of all resolved aspect nodes (every depth), each the resolved node augmented with .identity (base FQN, ctx-stripped), .identityKey (full unique key incl {ctxId}), and .isNamed. Excludes the entity root and tombstoned/excluded aspects; anonymous nodes are included so callers can surface them.
One fxFullResolve per class now yields both the membership pathSet (hasAspect) and a parallel resolvedNodes state map, so the accessor adds no extra resolution beyond what hasAspect already pays. Nodes are stored behind the existing state thunk, preserving deepSeq-safety (class-content bodies are never forced by state deepSeq; reading .aspects forces only name/meta/identity).
isNamed inspects the full identity, not just the node name: isMeaningfulName misses nested anonymous instances like roles/dev/<anon>:3 (whose name slips past the exact-<anon> check), so consumers (e.g. colmena tags) can filter cleanly on .isNamed.
Adds Group K has-aspect tests: flat coverage across depths, exclude-aware, entity-root excluded, anonymous exposure, nested identity distinctness, and the named-have-clean-identity invariant.
A policy's destructured args double as a dispatch guard: `{ <kind>, ... }:`
fans the policy across every entity of that kind. There was no way to say
"fire once at my own scope, don't fan" — and naming the scope's own kind
doesn't work when that kind isn't bound in the scope's ctx. A flake-scope
resolution policy is the motivating case: `{ flake, ... }:` fails
resolveArgsSatisfied (flake isn't in its own ctx), so the policy silently
never fires — which is exactly how a stale `{ flake-system, ... }:` on
to-fleet broke the fleet→env→cluster cascade.
Add `self`: always bound in the dispatch ctx (to the scope's own context), so
`{ self, ... }:` is satisfiable and callable at any scope, and excluded from
the late-sibling fan so it fires only at its registration scope.
- dispatch.nix: inject `self` at the single dispatch chokepoint, used for both
resolveArgsSatisfied and the policy call. Backwards compatible — existing
policies use `...` and ignore the extra key.
- policy/schema.nix: drop `self`-guarded policies from the late-dispatch
fan-out so they fire once, at their own scope.
`self` is the right guard for resolution policies (to-fleet, fleet-to-envs, …)
that create children and must fire once; `{ <kind>, ... }:` stays the tool for
genuine fan-out.
Tests (self-guard): self fires at its registration scope (host OS config set);
a host-registered self policy does not fan to user children, whereas
`{ user, ... }:` fans to each.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
schemaEntryTypeinoptions.nixwith gen-schema'smkSchemaOption(sidecars:includes,excludes; computed:isEntity)_topologyand_metaintrospection from gen-schemaresolvedCtxModuleto shared_types.nixfor reuse across entity typesDetails
Schema port:
den.schemanow uses gen-schema'smkSchemaEntryTypewhich provides sidecar extraction, computed fields, and__functorwrapping. TheresolvedCtxModule(id_hash, resolved, collisionPolicy) is extracted to_types.nixand injected into entity submodule imports.Flat form: Both
den.hostsandden.homesnow accept flat declarations:A
deepMergeAttrscustom type accepts both forms, andapplypreprocesses flat entries into the canonical two-level shape viapreprocessHosts. All 6 consumers see the unchanged{ system.name = entity }shape.Tests: 18 new tests (843 total, up from 825) covering flat hosts, flat homes, id_hash, freeform attrs, topology, meta introspection, isEntity computed, and schema sidecars.
Test plan
nix develop -c just ci)default,minimal,example) unaffected