feat: add GitHub issue queues with manual and opt-in automatic dispatch - #1254
filipenevola wants to merge 3 commits into
Conversation
|
CI follow-up: fc432d2 regenerates the bundle budget using the same Bun 1.3.14 runtime as CI. The official measurement now matches CI exactly (JS gzip 200141 bytes); no application code or guard thresholds were changed by hand. All 12 local gates pass again. The first docker-build failure happened while downloading moby/buildkit:buildx-stable-1 (HTTP 502), before the project image build; the new commit triggers a fresh run. |
|
Added an equivalent generic startup-readiness fix after staging reproduced an ingress race: automatic pickup now verifies the read-only Warren queue path before reserving budget or attempting the dispatch mutation. Readiness failures safely retry; post-mutation uncertainty remains fail-closed and durable. Includes regression tests for unavailable/unsupported/malformed queues and exactly-once pickup after recovery. Full source gates passed (12/12), and the extension has 24 passing tests. |
luantaraschi
left a comment
There was a problem hiding this comment.
I ran this rather than only reading it, since it is the biggest PR in the queue and Jaymin is out until the 14th. On a clean oven/bun:1.3.14 container at a36726e, bun run check:all passes 12 of 12. The contract change is additive and every new field is optional (supportsIssueListing, ready, url, repositoryUrl, resolveForProject), so the jira and ado extensions need no edits.
I also pointed the adapter at one of my repos with GITHUB_LABELS set. It returned the same 19 ready issues the GitHub API returns for that label, 24 issues in /issue-statuses with the pull requests left out (a PR number comes back 404), 404 for an issue outside the configured scope, 403 on close without GITHUB_ALLOW_CLOSE, and 401 without the bearer. With AUTO_DISPATCH_ENABLED unset, /dispatches reports automatic false and no scheduler starts, which is what the description promises.
Four things in the line comments, none of them blocking in my view, and one question: readiness never looks at assignees, so an issue a person has claimed stays in the queue and the automatic path can pick it up. That may be intended for a label-driven queue, but it is the kind of surprise an operator hits once. Whether a feature this size lands is Jaymin's call, so this is a comment review rather than an approval.
| "raw": { | ||
| "js": 690019, | ||
| "css": 54756 | ||
| "js": 694824, |
There was a problem hiding this comment.
Every earlier raise in this file carries a $comment entry saying what grew and by how much, and the script header says the ratchet only goes down. This one raises all four numbers without a note. A one-line entry for the new project-issues page would keep the file readable the way the previous five raises did.
| if (!repository) throw new TrackerFailure("invalid_scope", "Repository scope is required", 422); | ||
| const result = new Map<number, IssueRecord>(); | ||
| for (let page = 1; page <= this.config.maxPages; page++) { | ||
| const url = `${this.config.apiUrl}/repos/${repository}/issues?state=all&sort=created&direction=asc&per_page=100&page=${page}`; |
There was a problem hiding this comment.
This lists state=all on every poll, and /issue-statuses lists it again. A closed issue can never be ready, so the ready path could ask for state=open and leave state=all to /issue-statuses. On a repo with a few thousand issues the difference is tens of REST calls per tick at the default 60 second interval.
| maxPages: integer(env, "GITHUB_MAX_PAGES", 100), | ||
| timeoutMs: integer(env, "GITHUB_TIMEOUT_MS", 15000, 120000), | ||
| port: integer(env, "TRACKER_PORT", 8080, 65535), | ||
| bearer: env.TRACKER_BEARER, |
There was a problem hiding this comment.
With TRACKER_BEARER unset and GITHUB_ALLOW_CLOSE=true, POST /issues/{id}/close is reachable by anyone who can reach the port, using the extension's GitHub token. The jira adapter has the same optional bearer, so this matches the house pattern, but refusing that pair at load time would close the one combination that mutates GitHub without auth.
| for (const row of this.store.list()) { | ||
| if (row.state !== "running" || row.runId === null) continue; | ||
| // A failed poll isn't evidence that a run ended. Keep the reservation and stop this tick. | ||
| const run = await this.warren.getRun(row.runId); |
There was a problem hiding this comment.
If a run gets deleted on the Warren side, getRun throws on the 404 and this tick stops before pickup. Nothing settles the row, so the next tick throws at the same place, and pickup never runs again until someone edits the sqlite file. Failing closed is the right default here, but a 404 could settle the row as uncertain, which already parks the controller and shows up on /dispatches with a message instead of a console line per tick.
|
Thanks for putting this together. GitHub Issues/Projects support is a useful integration for Warren, and I appreciate the care around explicit dispatch, duplicate prevention, and keeping GitHub credentials in the extension. I’m closing this PR because its scope goes beyond the tracker integration: it adds an issue-queue UI, a new dispatch flow, automatic pickup with its own SQLite-backed Could you open an issue so we can agree on the scope of a smaller GitHub tracker integration before reworking this? We can discuss the UI and automatic pickup separately. Thanks again for contributing and using Warren! |
Summary
Add a standalone GitHub Issues / Projects v2 tracker with an operator queue and optional automatic pickup. Teams can keep their existing GitHub backlog instead of maintaining a second Seeds queue. Manual dispatch is the default; connecting a tracker alone never starts paid work.
This is built directly on current upstream
main(524b702a), without downstream fork changes or organization-specific configuration.Changes
GitHub adapter (
extensions/tracker-github)warren-tracker/v1adapter supports repository-only issues and user- or organization-owned Projects v2, including cross-repository Projects.allorany). Status/label names are configuration, not hard-coded workflow assumptions.Shared Warren integration
trackerconfiguration through a resolver, retaining Seeds as the fallback. Repository-controlled endpoints and credential variable names require operator allowlists; redirects are refused.Opt-in automatic pickup
Requires an explicit repository-to-Warren-project map, credentials, agent, persistent SQLite journal, per-run cap, and daily budget. Concurrency is bounded. Unknown dispatch results remain journaled and block further automatic pickup rather than being blindly retried. Failed/cancelled runs are not automatically retried. No merge or deployment is requested.
The core changes are needed to make the adapter usable through the existing UI and runtime; the GitHub transport and unattended controller remain entirely out-of-process. The extension README documents setup, permissions, failure recovery, and protocol additions.
Test plan
bun run check:all— 12/12 gates passed, including lint, typecheck, tests/coverage, generated docs, dependency/layer checks, and bundle budget.bun run build:ui.Boundaries / limitations
Submitted for maintainer review; no auto-merge requested.