panic: close of closed channel when staticManifestPath is set — SetupSignalHandler called twice

[fastly-ffej]

Description

When staticManifestPath is configured (i.e. staticManifestMode is not "off"), main() calls both staticRun() and run() sequentially. Both functions call ctrl.SetupSignalHandler(). The controller-runtime signal handler uses a package-level sync.Once and panics on the second call with panic: close of closed channel.

Reproduction

Configure spire-controller-manager with staticManifestPath set (any non-empty value). After the static reconcilers complete, the manager starts briefly then panics:

panic: close of closed channel

goroutine 1 [running]:
sigs.k8s.io/controller-runtime/pkg/manager/signals.SetupSignalHandler()
        .../controller-runtime@v0.22.4/pkg/manager/signals/signal.go:31 +0x1e
main.run(...)
        /workspace/cmd/main.go:274 +0xa5
main.main()
        /workspace/cmd/main.go:107 +0xfd

Root Cause

In cmd/main.go:

// line 101
if mainConfig.ctrlConfig.StaticManifestPath != nil {
    if err := staticRun(mainConfig); err != nil {  // calls SetupSignalHandler() at line 495
        os.Exit(1)
    }
}
if err := run(mainConfig); err != nil {            // calls SetupSignalHandler() again at line 274
    os.Exit(1)
}

staticRun completes its full reconciliation and returns, then run is called and hits SetupSignalHandler a second time.

Potential Fix

Since staticRun and run appear to be mutually exclusive lifecycle paths, main() could exit after staticRun returns rather than falling through to run():

if mainConfig.ctrlConfig.StaticManifestPath != nil {
    if err := staticRun(mainConfig); err != nil {
        os.Exit(1)
    }
    os.Exit(0)
}

Happy to submit a PR if this approach looks reasonable to maintainers.

Observed Behaviour

• Controller-manager reconciles all static entries successfully
• Manager starts, briefly binds metrics (:8082) and health (:8083) ports
• Immediately panics, process exits with code 2
• Pod enters a crash loop (every ~3 minutes in our environment with ~10k entries)

Version

v0.6.4 (latest). Same code present in main branch.

[fastly-ffej]

Update: confirmed fix + secondary bug discovered

We've been running this in production with a patched image. Two findings:

Fix 1: os.Exit(0) after staticRun() ✅

The proposed fix works — adding os.Exit(0) after staticRun() returns prevents fall-through into run() and eliminates the panic entirely. Our patched image has been stable.

Fix 2: health/metrics ports never bind during reconciliation

After applying Fix 1, we hit a secondary bug: in staticRun(), mgr.Start() is called after wg.Wait(). Since wg.Wait() blocks until all static reconcilers complete, the manager never actually starts during reconciliation — meaning ports 8082 (metrics) and 8083 (healthz) are never bound while the work is happening.

In a Kubernetes deployment with liveness probes, this causes a kill loop:

1. Pod starts, staticRun() begins reconciling (~10k entries in our case, takes 60–90s)
2. Liveness probe fires after initialDelaySeconds, finds port 8083 unbound, sends SIGTERM
3. staticRun() exits via os.Exit(0) (Fix 1) → pod restarts
4. Repeat indefinitely

The fix is to start the manager concurrently before wg.Wait():

mgrErrCh := make(chan error, 1)
go func() {
    setupLog.Info("starting manager")
    mgrErrCh <- mgr.Start(ctx)
}()

wg.Wait()
if err := <-mgrErrCh; err != nil {
    setupLog.Error(err, "problem running manager")
    return err
}

This binds the health/metrics ports immediately so probes pass during the full reconciliation window.

Note: the "Observed Behaviour" in the issue description ("briefly binds metrics/health ports before panicking") reflects a best-case timing scenario. In practice with a large entry set and liveness probes enabled, the ports are never bound before the probe fires.

Happy to submit a PR with both fixes if that would be helpful.