refactor(tasks): schedule with RQ instead of rq-scheduler

[sdornan]

Description

Supersedes #4272 and #4273, both closed in favour of this. The scan-guard fixes #4272 made are all carried here, and the two workarounds it needed are deleted rather than kept.

Fixes #4186

Drops rq-scheduler and schedules with RQ itself. The library has one release since 2023 (v0.14, Oct 2024), 109 open issues, and RomM ran a fork of it for Redis username/SSL options upstream never took. Everything it was here for now ships with RQ.

Both of its failure modes are why #4272 needed workarounds:

This is also what fixes #4186. On an affected instance the scheduler is stalled on a job it cannot read, so the watcher's rescans pile up in its registry, and the scan guard added in 99e214a counted those as scans waiting to start and refused every manual scan. Here the scan guard consults only what is queued on a worker, and startup clears the legacy registry outright, so both halves of that are gone.

• It stalls on a job it cannot read. Scheduler.enqueue_job logs the job's function name — which unpickles the payload — before removing it from the registry, so one bad job crashes it on every poll forever. That is the outage in [Bug] Scan already in progress, ignoring request #4186. RQ's scheduler removes every id it looked at whether or not the job could be read, so this cannot happen (verified below).
• Cancelling a delayed job took two calls. Scheduler.cancel left the status alone, Job.cancel left the registry entry, and the scheduler then queued the "cancelled" job anyway. Job.cancel alone now clears the scheduled registry.

What changes:

• Delayed jobs (the watcher's deferred rescan) go on the queue with Queue.enqueue_in and are released by the worker running --with-scheduler. One fewer process.
• Periodic tasks are declared in tasks/cron_config.py and loaded by an rq cron process replacing rqscheduler. A task is registered only when it is enabled and has a cron string, so the schedule-then-unschedule dance around the ENABLE_SCHEDULED_* toggles is gone, and with it PeriodicTask.init/schedule/unschedule and the unschedule() call each task made against itself.
• Jobs are enqueued by task name through run_task_by_name, resolved against a catalog in tasks/registry.py. Nothing pickles a task instance into Redis any more — which is what made a payload unreadable across an upgrade in the first place. Manual runs go through the same path.
• Scan discovery keys off the type a job carries rather than the scheduled rescan's function name, since every task now shares an entry point. That also drops SCAN_LIBRARY_TASK_FUNC, which existed only to name a function across an import cycle.
• The watcher's dedupe is fixed. It filtered on job.args[0] while scans are enqueued with keyword arguments only, so it never matched and every filesystem change scheduled another scan. It now summarises what the pending scans cover — by platform id, by folder slug, and by rom — and matches a changed folder against the slugs as well as the ids, so a queued slug-scoped or rom-scoped scan no longer reads as a full-library rescan and suppresses everything. A scan whose payload cannot be read is left out: its scope is unknowable, and a duplicate scan costs less than a rescan that never happens. It also deliberately ignores a scan already running, which may have walked past the folder that just changed.
• Startup clears the old scheduler's keys once, deleting only job hashes no queue still references.
• The two startup backfills use unique=True instead of asking whether their job id exists and then enqueueing it, which two instances starting together can both get past. RQ does the check, the save and the push in one Lua call and raises DuplicateJobError. They are enqueued by name too, so no payload in the codebase carries a pickled task any more.

Net −496 lines.

The rq bump, folded in from #4273

uv.lock sat on rq 2.9.0 while pyproject.toml allowed anything under 3.0, so the floor now names 2.11 — the version this was written and tested against.

To be precise about why: the migration runs on 2.9, which is what was already locked. Every API it uses exists there. 2.11 is what it benefits from — RQScheduler acquires and refreshes its lock before enqueueing, which matters once a worker rather than a separate process releases delayed jobs, and each CronJob gained a name plus the ids of the jobs it created, which is the introspection that moving cron out of Redis would otherwise cost. 2.10 added a stable scheduler identity, and 2.9.1 covers redis-py >= 8.

The constraint stays under 3.0 deliberately: that release moves get_current_job() to contextvars, reworks dependencies behind a ReadyJobRegistry, and changes the Worker.handle_job_success() signature, which RomMWorker subclasses around.

2.11.0 was published on 2026-08-17, so it is inside the rolling 7-day exclude-newer window until 2026-08-24 and needs a per-package exclusion to resolve before then. That entry is dated and can be deleted from tomorrow onward — say the word and I will strip it rather than leave it for someone to notice.

Behaviour worth reviewing

1. A cron schedule now lives in the rq cron process rather than in Redis, so a missed run during downtime is skipped rather than fired late, and cron_job.get_job_ids() (new in 2.11) is where run history lives.
2. tasks/registry.py holds every scheduled task, including the four that were never surfaced by the API (cleanup_netplay, cleanup_upload_tmp, sync_retroachievements_progress, sync_push_pull). GET /api/tasks still lists the same six as before, via VISIBLE_SCHEDULED_TASKS — say the word if they should all be visible now.
3. startup.py imports drop_stale_scheduled_scans from endpoints.sockets.scan, which has the layering backwards. It is not a new coupling — that module is already in startup's import graph via tasks.scheduled.scan_library — and fixing it properly means moving scan job discovery, or scan_platforms itself, out of the endpoints layer. Left as follow-up rather than growing this diff.
4. PeriodicTask.func is no longer used at runtime, only asserted on by test_task_func_paths.py. Left in place to keep this diff focused; it can go in a follow-up.

Checklist

• I've tested the changes locally
• I've updated relevant comments
• I've assigned reviewers for this PR
• I've added unit tests that cover the changes

Review findings addressed

• REDIS_SSL was read in the shell as redis${REDIS_SSL:+s}, which only asks whether the variable is set, while env.template ships REDIS_SSL=false. Following the documented value pointed the RQ worker at rediss:// where the app itself parsed the same variable correctly and used redis://. That predates this PR (since dd6669e), but adding the cron process put a second process behind it, so both entrypoints now parse the value against the truthy set the app accepts.
• The legacy-key cleanup returned early when the old registry was empty, skipping rq:scheduler_lock and rq:scheduler_instance:*. Only the orphan-job work is guarded now.
• greptile, P1: a request body could set name and redirect a manual run to a task the route had refused (one not surfaced by the API, or with manual_run off). The caller's arguments are now nested under task_kwargs rather than spread, so the collision is impossible rather than order-dependent, and a task stays free to take an argument called name. Covered by a test at the runner and at the endpoint.

Testing

Full backend suite green (3052 passed, 2 skipped), trunk fmt && trunk check clean. watcher.py gets the test file it never had (19 tests over scope summarising and the dedupe; four of them fail against the previous scope check), as do the job accessors that tolerate an unreadable payload. New tests also cover the cron config (what it registers, and that it matches the real catalog), the registry, run_task_by_name, and the rewritten scan discovery; the tests for the removed rq-scheduler lifecycle are gone.

Also driven against a real Redis with the real CronScheduler, since none of that is exercised by mocks:

• rq cron loads tasks.cron_config and registers exactly the enabled tasks with a cron string, every one as run_task_by_name with no pickled task in the payload.
• A delayed watcher scan lands in ScheduledJobRegistry, reads as scheduled but not queued (so it cannot refuse a manual scan), is visible to the watcher as pending, and its platform_ids are readable for the dedupe.
• job.cancel() alone clears the registry entry.
• RQScheduler.enqueue_scheduled_jobs() releases a due job onto the queue and out of the registry.
• A job with a deliberately corrupted payload leaves the registry anyway — the [Bug] Scan already in progress, ignoring request #4186 failure mode is structurally impossible now.
• The stale-backlog sweep from fix(scan): stop refusing scans a stalled scheduler left pending #4272 still drops long-overdue scans and keeps one still waiting out its delay.
• unique=True refuses a second enqueue of the same job id with DuplicateJobError and leaves the queue at depth 1, where the same pair without it enqueues twice.

Not exercised locally: the container entrypoints. docker/init_scripts/init and entrypoint.sh were rewritten to start rq cron (with a pid file for the watchdog, since rq cron has no --pid) and to pass --with-scheduler to the worker; both need a Docker run before merge.

AI assistance disclosure

This change was written with AI assistance (Claude Opus 5 via Claude Code). The design, the code, the tests, and this description were AI-generated, then reviewed and verified by me: I read RQ's scheduler, cron, registry, and job sources to confirm the semantics claimed here, ran the suite and linters, and ran the real-Redis verification above.

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	pypi/​rq@​2.9.0 ⏵ 2.11.0

View full report

[greptile-apps]

Greptile Summary

The PR replaces rq-scheduler with RQ 2.11's native cron and delayed-job scheduling, centralizes task lookup in a registry, and updates scan discovery, watcher deduplication, startup cleanup, process supervision, tests, and documentation.

• Adds rq cron registration for enabled periodic tasks and worker-side delayed-job scheduling.
• Routes scheduled and manual jobs through a stable task-name registry.
• Reworks scan discovery, cancellation, stale-backlog cleanup, and watcher deduplication.
• Removes rq-scheduler and updates container and development entrypoints.

Confidence Score: 3/5

This PR should not merge until the task-name override is fixed because it bypasses the endpoint's restrictions on which registered tasks may be run manually.

The endpoint validates the route-selected task but enqueues a user-overridable name that the worker resolves against the full registry, allowing hidden or non-manual tasks to execute.

Files Needing Attention: backend/endpoints/tasks.py, backend/tasks/tasks.py, backend/tasks/registry.py

Security Review

The manual-task endpoint allows request kwargs to overwrite the validated registry name, enabling callers with task-run permission to enqueue hidden or non-manually-runnable tasks. How this was verified: The request body is expanded after the trusted name, and the worker resolves that overwritten value against the unrestricted task registry without repeating the endpoint check.

Important Files Changed

Filename	Overview
backend/endpoints/tasks.py	Replaces local task lists with the shared registry and name-based dispatch, but permits body kwargs to override the task name validated by the endpoint.
backend/tasks/tasks.py	Adds the common name-based task runner and removes rq-scheduler lifecycle methods.
backend/tasks/registry.py	Defines stable scheduled and manual task catalogs shared by cron and API execution.
backend/tasks/cron_config.py	Registers enabled tasks with cron strings through RQ's native cron configuration.
backend/endpoints/sockets/scan.py	Reworks scan discovery across workers, queues, and the native scheduled-job registry and adds stale delayed-scan cleanup.
backend/watcher.py	Moves delayed scans to Queue.enqueue_in and deduplicates them using keyword payload scopes.
backend/startup.py	Removes periodic-task initialization and cleans stale native scans plus legacy rq-scheduler state.
docker/init_scripts/init	Replaces the rq-scheduler process with rq cron and enables delayed-job scheduling in the worker.
entrypoint.sh	Updates the development process topology to run rq cron and a scheduler-enabled worker.
pyproject.toml	Removes the rq-scheduler dependency and raises RQ to the native-cron-capable 2.11 series.
uv.lock	Resolves RQ 2.11 and removes rq-scheduler and its obsolete transitive dependency.

Prompt To Fix All With AI

### Issue 1
backend/endpoints/tasks.py:325
**Task name validation is bypassed**

When an authorized caller includes `name` in the request body, the expansion overwrites the route-selected task name, so `run_task_by_name` can execute hidden or non-manually-runnable tasks without their endpoint restrictions. **How this was verified:** The body kwargs are expanded after the trusted name, and the worker resolves the resulting value against the unrestricted task registry.

```suggestion
        kwargs={**(task_kwargs or {}), "name": task_name},
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

_{Reviews (1): Last reviewed commit: "refactor(tasks): schedule with RQ instea..." | Re-trigger Greptile}

[greptile-apps]

Task name validation is bypassed

When an authorized caller includes name in the request body, the expansion overwrites the route-selected task name, so run_task_by_name can execute hidden or non-manually-runnable tasks without their endpoint restrictions. How this was verified: The body kwargs are expanded after the trusted name, and the worker resolves the resulting value against the unrestricted task registry.

Suggested change

	kwargs={"name": task_name, **(task_kwargs or {})},
	kwargs={**(task_kwargs or {}), "name": task_name},

Knowledge Base Used: Tasks and Scheduler

Prompt To Fix With AI

This is a comment left during a code review.
Path: backend/endpoints/tasks.py
Line: 325

Comment:
**Task name validation is bypassed**

When an authorized caller includes `name` in the request body, the expansion overwrites the route-selected task name, so `run_task_by_name` can execute hidden or non-manually-runnable tasks without their endpoint restrictions. **How this was verified:** The body kwargs are expanded after the trusted name, and the worker resolves the resulting value against the unrestricted task registry.

```suggestion
        kwargs={**(task_kwargs or {}), "name": task_name},
```

**Knowledge Base Used:** [Tasks and Scheduler](https://app.greptile.com/romm/-/custom-context/knowledge-base/rommapp/romm/-/docs/tasks-and-scheduler.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[sdornan]

Fixed in bce4a3c — valid, and the impact was as described: the endpoint validated the path name, then the worker resolved whatever name the payload ended up holding, so {"name": "sync_push_pull"} in the body reached a task that is neither surfaced by the API nor manually runnable.

I nested the caller's arguments instead of reordering the spread:

kwargs={"name": task_name, "task_kwargs": task_kwargs or {}},

{**(task_kwargs or {}), "name": task_name} also closes it, but the collision then depends on key order surviving future edits, and it silently drops a legitimate name argument to a task's own run(). Nesting makes it structurally impossible. run_task_by_name takes task_kwargs: dict | None and forwards it.

Covered at both levels: test_forwarded_arguments_cannot_name_another_task at the runner, and test_body_cannot_override_the_task_name asserting the enqueued payload keeps the route's name.

[Copilot]

Pull request overview

Refactors background scheduling from rq-scheduler to RQ 2.11 using rq cron, worker-managed delayed jobs, and name-based task dispatch.

Changes:

• Replaces the scheduler dependency and updates startup/container wiring.
• Adds a centralized task registry and cron configuration.
• Reworks watcher scans, deduplication, cancellation, legacy cleanup, documentation, and tests.

Reviewed changes

Copilot reviewed 28 out of 29 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Review summary
uv.lock	Updates RQ dependencies and removes rq-scheduler.
pyproject.toml	Replaces the scheduler dependency with RQ 2.11.
entrypoint.sh	Starts RQ cron and scheduler-enabled workers. Critical (1 vote): ${REDIS_SSL:+s} treats false as TLS, causing cron to use rediss:// for plaintext Redis.
docs/BACKEND_ARCHITECTURE.md	Documents the new scheduling architecture.
docker/init_scripts/init	Updates production process lifecycle management. Critical (1 vote): ${REDIS_SSL:+s} treats false as TLS, causing cron to use rediss:// for plaintext Redis.
backend/watcher.py	Uses delayed RQ jobs and payload-based scan deduplication.
backend/tests/tasks/test_tasks.py	Updates task lifecycle and runner tests.
backend/tests/tasks/test_scan_library.py	Updates scan task tests.
backend/tests/tasks/test_registry.py	Tests task registry resolution and uniqueness.
backend/tests/tasks/test_prevent_requeue.py	Removes obsolete scheduler tests.
backend/tests/tasks/test_cron_config.py	Tests cron registration behavior.
backend/tests/tasks/test_cleanup_zip_cache.py	Updates cleanup task tests.
backend/tests/tasks/test_cleanup_orphaned_resources.py	Removes obsolete scheduling lifecycle tests.
backend/tests/endpoints/test_tasks.py	Updates task endpoint tests for registry-based dispatch.
backend/tests/endpoints/sockets/test_scan.py	Tests revised scan discovery and cancellation.
backend/tasks/tasks.py	Adds name-based task execution.
backend/tasks/scheduled/scan_library.py	Removes scheduler lifecycle handling.
backend/tasks/scheduled/cleanup_zip_cache.py	Removes self-unscheduling behavior.
backend/tasks/scheduled/cleanup_upload_tmp.py	Removes self-unscheduling behavior.
backend/tasks/scheduled/cleanup_orphaned_resources.py	Removes custom scheduling initialization.
backend/tasks/scheduled/cleanup_netplay.py	Removes self-unscheduling behavior.
backend/tasks/registry.py	Defines scheduled and manual task catalogs.
backend/tasks/cron_config.py	Registers enabled cron tasks.
backend/startup.py	Cleans legacy scheduler state and stale scans. Moderate (4 votes): an empty old sorted set causes an early return that skips deleting legacy scheduler keys.
backend/handler/redis_handler.py	Adds safe job status and payload helpers.
backend/exceptions/task_exceptions.py	Adds unknown-task error handling.
backend/endpoints/tasks.py	Uses the registry and stable task-name payloads. Moderate (4 votes): merging the request body after name allows overriding a visible task with hidden cleanup_netplay, bypassing visibility and manual-run checks.
backend/endpoints/sockets/scan.py	Reworks scan discovery and delayed-job handling.
backend/endpoints/responses/__init__.py	Uses RQ job status types.

Suppressed comments (18)

backend/endpoints/sockets/scan.py:198

• ScheduledJobRegistry.get_scheduled_time() can return None if the registry entry is released or cancelled between _get_scheduled_scan_jobs() and this lookup. Comparing that value with cutoff aborts startup's stale-scan sweep, leaving the remaining backlog untouched. Treat a missing timestamp as a raced-away job and continue.

        if scheduled_at > cutoff:

backend/endpoints/tasks.py:205

• This catch only protects the worker-pointer branch. Queue and finished/failed entries still flow through _build_task_status_response, which calls job.get_status() directly; an expired job hash raises InvalidJobOperation, so /api/tasks/status can still return 500. Use get_job_status in the common response and skip or represent jobs whose status is None.

        try:
            current_job = worker.get_current_job()
        except NoSuchJobError:
            continue

backend/handler/redis_handler.py:106

• A queue entry can disappear between get_jobs() and this property access. RQ raises InvalidJobOperation/NoSuchJobError for that case, not DeserializationError, so the watcher can abort filesystem-change processing instead of skipping the stale job. Treat missing jobs like other unreadable payloads.

    except DeserializationError:

backend/handler/redis_handler.py:92

• This wrapper is not used by _build_task_status_response in endpoints/tasks.py, which still calls job.get_status() directly. If a queue or finished job expires between enumeration and response construction, GET /api/tasks/status can still raise InvalidJobOperation and return 500, so the expired-job handling does not cover the status endpoint. Use this helper in the response path and skip jobs whose status is gone.

    try:
        return job.get_status()
    except InvalidJobOperation:
        return None

backend/startup.py:187

• The new payload invariant does not cover bootstrap jobs: _enqueue_convert_images_to_webp() still enqueues the bound convert_images_to_webp_task.run method, and the recompute helper above does the same, so RQ still pickles task instances. A deployment can leave one of these jobs in Redis and hit the upgrade/deserialization failure this refactor is intended to remove. Route these bootstrap jobs through run_task_by_name with their registry keys too.

            _enqueue_convert_images_to_webp()

backend/startup.py:154

• The orphan check only consults queue lists. A periodic rq-scheduler job is re-added to rq:scheduler:scheduled_jobs for its next run, but once a worker starts it, its ID is removed from the queue list. During a rolling restart this code can therefore classify that active job as an orphan and delete its hash while it is executing. Include worker or StartedJobRegistry IDs before deleting legacy hashes.

        queued = set()
        for queue in (high_prio_queue, default_queue, low_prio_queue):
            queued.update(queue.get_job_ids())

        orphans = legacy_job_ids - queued

backend/tasks/tasks.py:19

• This claim is not true for startup backfills: startup.py still enqueues recompute_save_content_hashes_task.run and convert_images_to_webp_task.run as bound methods. Those payloads pickle task instances, so these jobs can still hit the unreadable-payload upgrade failure that this runner is meant to eliminate. Route both startup enqueues through run_task_by_name with their registry names, and update the startup tests accordingly.

    Every scheduled and manually triggered task is enqueued through here, so a
    job payload holds a name rather than a pickled task, and nothing in Redis
    depends on where the code that runs it lives.

backend/watcher.py:160

• The new watcher deduplication path is not covered by tests: there is no test for a pending delayed job whose platform_ids are stored in kwargs, nor for suppressing a second change for the same platform. A regression to the old job.args[0] lookup would therefore pass the current suite even though it would recreate duplicate rescans. Add focused tests for full-library and per-platform pending scopes.

        pending_scopes = [
            kwargs.get("platform_ids") or []
            for job in get_pending_scan_jobs()
            if (kwargs := get_job_kwargs(job)) is not None
        ]

docker/init_scripts/init:249

• rq cron treats positional arguments as queue names, not Python modules to import. With tasks.cron_config in this position, the cron process never executes the registration module, so no periodic tasks are registered. Pass the module through the CLI's config option instead.

		tasks.cron_config &

docker/init_scripts/init:228

• The URL embeds raw Redis credentials, so a password containing @, :, /, or % is parsed as URI syntax instead of as the credential. The new cron process then connects with the wrong host or password. Percent-encode the username and password before constructing the URL, in both entrypoints.

		echo "redis${REDIS_SSL:+s}://${REDIS_USERNAME-}:${REDIS_PASSWORD}@${REDIS_HOST:-127.0.0.1}:${REDIS_PORT:-6379}/${REDIS_DB:-0}"

docker/init_scripts/init:248

• rq cron uses the --logging_level option spelling, so this new command exits with an unknown-option error before loading tasks.cron_config. The watchdog then repeatedly restarts it without ever running periodic tasks.

		--logging-level "${LOGLEVEL}" \

docker/init_scripts/init:249

• The schedule is process-local, so this production startup path also launches one independent cron scheduler per RomM container. With multiple replicas, each periodic task executes once per replica instead of once per schedule; cron needs singleton deployment or distributed ownership.

		rq cron \
		--path /backend \
		--logging-level "${LOGLEVEL}" \
		tasks.cron_config &

docker/init_scripts/init:274

• The stale sweep runs only from startup.py, but this worker is independently restarted by watchdog_process_pid. If it is down for more than STALE_SCHEDULED_SCAN_AGE while the watcher keeps enqueueing, RQ promotes every overdue low-queue job as soon as this worker starts, bypassing the sweep and recreating the backlog storm. Run drop_stale_scheduled_scans before each scheduler-enabled worker start, or move the purge into the worker startup path.

		--with-scheduler \

entrypoint.sh:67

• rq cron treats positional arguments as queue names, not Python modules to import. With tasks.cron_config in this position, the cron process never executes the registration module, so no periodic tasks are registered. Pass the module through the CLI's config option instead.

	tasks.cron_config &

entrypoint.sh:63

• The URL passed to the new cron process embeds raw Redis credentials, so passwords containing @, :, /, or % are parsed as URI delimiters and authentication fails. Percent-encode the username and password before constructing REDIS_URL, consistently with the other entrypoint.

	RQ_REDIS_URL="${REDIS_URL}" \

entrypoint.sh:66

• rq cron uses the --logging_level option spelling (the worker invocation below already uses that form). With --logging-level, the CLI exits with an unknown-option error, so this process never starts and no periodic tasks are registered.

	--logging-level "${LOGLEVEL:-INFO}" \

entrypoint.sh:67

• The schedule is process-local, so starting rq cron unconditionally here makes every horizontally scaled RomM container register and execute the same periodic tasks. Run cron as a singleton or add explicit distributed ownership, otherwise schedules such as cleanup and sync will run once per replica.

	rq cron \
	--path /app/backend \
	--logging-level "${LOGLEVEL:-INFO}" \
	tasks.cron_config &

entrypoint.sh:83

• In this entrypoint, main.py is launched in the background before this scheduler-enabled worker starts. On a restart with overdue RQ delayed scans, the worker can release them before startup.py reaches drop_stale_scheduled_scans(), making the stale-backlog protection timing-dependent. Start the worker after startup cleanup has completed in this entrypoint, as the production init script already does.

	--with-scheduler \

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[sdornan]

Fixed in bce4a3c. Same finding as the review above, and your cleanup_netplay example is exactly right — it is in the registry for cron but absent from VISIBLE_SCHEDULED_TASKS, so it was reachable that way.

I took the "reject the reserved key" half of your suggestion rather than "keep the route name last": the caller's arguments now sit under a task_kwargs key, so a body cannot collide with the task name at all, instead of relying on ordering.

[sdornan]

Valid, fixed. The early return did skip the key deletion, so an old scheduler that registered itself without ever holding a delayed job left rq:scheduler_lock and rq:scheduler_instance:* behind.

Now only the orphan-job calculation is guarded, and the keys go unconditionally:

if legacy_job_ids:
    ...  # delete only hashes no queue still references
redis_client.delete(*LEGACY_SCHEDULER_KEYS)
for key in redis_client.scan_iter("rq:scheduler_instance:*"):
    redis_client.delete(key)

For the record on severity: both of those keys carry a TTL of the scheduler's poll interval plus ten seconds, so they expire on their own about a minute after the old process stops, and rq:scheduler appears never to be written by rq-scheduler at all — its instances key off rq:scheduler_instance:<name>. So this was untidy rather than harmful. Three tests added, including one asserting the keys are removed when the registry is empty.

[sdornan]

Valid, and worse than "the new cron path" — it predates this PR. ${REDIS_SSL:+s} has been choosing the scheme for the RQ worker since dd6669e, and env.template ships REDIS_SSL=false, so anyone following the documented value has had the worker pointed at rediss:// while the app itself parsed the same variable correctly through safe_str_to_bool and used redis://. Adding cron just put a second process behind the same bug.

Fixed for both processes by parsing the value, matching Python's truthy set (1, true, yes, on, case-insensitive):

ssl="$(printf '%s' "${REDIS_SSL-}" | tr '[:upper:]' '[:lower:]')"
case "${ssl}" in
1 | true | yes | on) echo "rediss" ;;
*) echo "redis" ;;
esac

Verified across false, 0, true, 1, TRUE, yes, empty and unset. tr rather than ${ssl,,} so it does not depend on bash 4.

[sdornan]

Valid, and worse than "the new cron path" — it predates this PR. ${REDIS_SSL:+s} has been choosing the scheme for the RQ worker since dd6669e, and env.template ships REDIS_SSL=false, so anyone following the documented value has had the worker pointed at rediss:// while the app itself parsed the same variable correctly through safe_str_to_bool and used redis://. Adding cron just put a second process behind the same bug.

Fixed for both processes by parsing the value, matching Python's truthy set (1, true, yes, on, case-insensitive):

ssl="$(printf '%s' "${REDIS_SSL-}" | tr '[:upper:]' '[:lower:]')"
case "${ssl}" in
1 | true | yes | on) echo "rediss" ;;
*) echo "redis" ;;
esac

Verified across false, 0, true, 1, TRUE, yes, empty and unset. tr rather than ${ssl,,} so it does not depend on bash 4.

In this file the scheme is resolved once into REDIS_SCHEME before the URL branches, so the worker and the cron process both get it.