Skip to content

Backfill CRD Established=True for 6 operator-owning health checks #1246

Closed

Description

@mchmarny

Summary

Follow-up to PR #1245 (closed #1222, closed epic #660). Add
Established=True assertions for the CRDs owned by 6 operator
components that PR #1245 deferred because each needs chart-version
verification.

Background

PR #1245 added CRD Established=True assertions for 4 operator-owning
components where the CRD names were verifiable from in-tree sources
(tests/chainsaw/ai-conformance/cluster/assert-crds.yaml and the Go
verifyClusterPolicyReady check):

  • gpu-operator → clusterpolicies.nvidia.com
  • dynamo-platform → dynamocomponentdeployments.nvidia.com
  • nodewright-operator → skyhooks.skyhook.nvidia.com
  • cert-manager → certificates.cert-manager.io,
    issuers.cert-manager.io, clusterissuers.cert-manager.io

Six operators that own CRDs were left out because their CRD lists
aren't enumerated in-tree at the pinned chart version. Bundling them
blindly would risk introducing typo'd CRD names that fail on live
clusters — better to verify each chart's actual output first.

Components to enhance

Component Pinned chart Expected CRD group
gatekeeper (see recipes/registry.yaml) gatekeeper.sh, templates.gatekeeper.sh, config.gatekeeper.sh, expansion.gatekeeper.sh, mutations.gatekeeper.sh, status.gatekeeper.sh
kueue (see recipes/registry.yaml) kueue.x-k8s.io (ClusterQueue / LocalQueue / ResourceFlavor / Workload / ProvisioningRequestConfig / AdmissionCheck / Topology / Cohort / MultiKueueCluster / MultiKueueConfig / WorkloadPriorityClass)
kubeflow-trainer (see recipes/registry.yaml) trainer.kubeflow.org (TrainJob / TrainingRuntime / ClusterTrainingRuntime / JobSet)
network-operator (see recipes/registry.yaml) mellanox.com / nvidia.com (NicClusterPolicy / MacvlanNetwork / IPoIBNetwork / HostDeviceNetwork)
k8s-nim-operator (see recipes/registry.yaml) apps.nvidia.com (NIMService / NIMCache / NIMPipeline)
slinky-slurm-operator (see recipes/registry.yaml) sibling slinky-slurm-operator-crds already covers slinky.slurm.net CRDs — confirm the operator component itself doesn't ship additional CRDs

(CRD lists above are best-effort guesses; the implementation PR
should re-verify against each pinned chart via helm template or by
inspecting the actual chart's crds/ directory.)

Approach

For each component:

  1. Pull the chart at its pinned version (helm pull or helm template).
  2. Enumerate every CRD the chart installs (typically under crds/,
    sometimes under templates/ for older charts).
  3. Add (conditions[?type == 'Established']).status: True asserts
    for each, in a new validate-<comp>-crds-established step at the
    top of the component's recipes/checks/<comp>/health-check.yaml.
  4. Live-cluster validation per component (the load-bearing step —
    typo'd CRD names fail at runtime).

Can land as one PR per component (granular, easier review) or one
PR per logical pair (e.g., slinky-slurm-operator together with its
slinky-slurm-operator-crds sibling). User preference.

Acceptance Criteria

  • Every component listed above has at least one CRD asserted
    Established=True (where the chart actually installs CRDs — if
    any of these turn out not to ship CRDs in their pinned version,
    document the negative finding in the PR body).
  • Allowlist sweep (TestValidateTestReadOnly_RegistryContent) +
    registry lint guard (TestComponentRegistry_RequiresHealthCheck)
    remain green.
  • Live-cluster validation per touched component documented in the
    PR description.

Related

Activity

  1. mohityadav8 commented on Jul 24, 2026

    @mohityadav8
    Contributor

    /assign

  2. added a commit that references this issue on Jul 28, 2026
  3. yuanchen8911 commented on Jul 31, 2026

    @yuanchen8911
    Contributor

    Triaged: confirmed at P2 / In progress. PR #1919 is in review; priority stays P2 for now.

  4. added 2 commits that reference this issue on Jul 31, 2026
    5aefb94
    68c378a
  5. mohityadav8 commented on Aug 2, 2026

    @mohityadav8
    Contributor

    @mchmarny completed this issue via #1919

  6. mchmarny commented on Aug 4, 2026

    @mchmarny
    MemberAuthor

    Closing: completed by #1919. The CRD Established=True backfill merged on 2026-07-31 and the assignee confirmed delivery.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions