Skip to content

design.md

Latest commit

 

History

History
48 lines (33 loc) · 2.39 KB

design.md

File metadata and controls

48 lines (33 loc) · 2.39 KB
Raw

QueueCTL Design

Goal

QueueCTL is designed as a small durable job queue that can be driven entirely from a CLI. The implementation favors correctness and reviewability over extra features: persistent jobs, atomic claiming, retry with backoff, DLQ support, and worker recovery are the core concerns.

Components

  • cmd: Cobra command definitions and CLI validation.
  • internal/db: SQLite schema and all persistence operations.
  • internal/engine: Worker loop, command execution, retry scheduling, graceful shutdown, and stale-job recovery.
  • internal/models: Shared job and configuration structs.
  • constants: Job states and supported config keys.

Data Model

jobs stores every job and its lifecycle state:

  • pending: ready to run.
  • processing: claimed by a worker.
  • completed: command exited with code 0.
  • failed: command failed but will be retried after run_at.
  • dead: retry budget exhausted and visible through the DLQ.

config stores persisted settings such as max-retries and backoff-base.

workers stores active foreground worker processes and lets worker stop request graceful shutdown from another terminal.

Worker Flow

  1. A worker process registers itself in workers.
  2. Worker goroutines repeatedly call AcquireNextJob.
  3. AcquireNextJob atomically changes one eligible job to processing and returns it.
  4. The worker executes the command through the OS shell.
  5. Exit code 0 marks the job completed.
  6. Non-zero exit increments attempts and either schedules a retry or moves the job to dead.
  7. On Ctrl+C, SIGTERM, or worker stop, workers finish the current job before exiting.

Recovery Flow

Crash recovery is timeout-based with a worker-heartbeat guard. If a process dies while a job is processing, no cleanup handler can run. The next active worker periodically sweeps old processing jobs, but it only moves them back to pending when the owning worker heartbeat is also stale. This prevents permanent processing jobs without rescuing a job that is still actively being heartbeated by a live worker.

Operational Notes

  • SQLite WAL mode and a busy timeout reduce lock contention across worker processes.
  • A single DB connection per process avoids avoidable intra-process SQLite writer contention.
  • JSON list output is clean stdout so scripts can parse it.
  • Human-readable worker logs go through standard logging and do not affect JSON list output.