Skip to content

feat(dashboard): drop the post-connect test run, explain failed runs on HQ - #1160

Merged
aaronjmars merged 6 commits into
mainfrom
feat/connect-no-test
Oct 3, 2026
Merged

aaronjmars merged 6 commits into
mainfrom
feat/connect-no-test

Conversation

@aaronjmars

Copy link
Copy Markdown
Collaborator

What changed

  • No test run after connecting a model. The owner decided the connect-check run is not needed: the first real run already proves the credential. After saving, the Connect modal says what was saved and that the next run uses it (and which harness it switched to, for login captures). No "Test now" / "Test again", no /api/connect-check, no polling hook.
  • Setup checklist: "Model connected" is done once a credential for the selected harness is saved.
  • ./aeon init: step 7 "Test connection" and --no-test are gone; Telegram is step 7 again. init-sandbox.sh now checks that init never dispatches a workflow run.
  • Skill removed: skills/connect-check and its aeon.yml entry. catalog/skills.json, catalog/packs.json, skill icons (+ lib/skill-icons.data.ts), eyebrowlock.json, docs and skill counts (86 -> 85) updated.
  • Failed runs explain themselves. lib/connect-check.ts is now lib/run-diagnosis.ts: for a failed run it reads the Run output and error lines and returns one plain reason, a next step, and whether the credential is the problem (for example "The provider rejected the credential. Next step: Paste a fresh key or log in again."). New GET /api/runs/[id]/diagnosis reads the log with gh and caches the result. HQ Recent activity and the run detail panel show it under failed runs only, loaded lazily and cached, with a Connect button when the credential failed.

Verification

  • apps/dashboard: npm ci, npm run typecheck, npm run lint (0 errors), npm test (297 pass), npm run build
  • apps/cli: npm run typecheck, npm run lint, bash apps/cli/test/init-sandbox.sh (PASS)
  • bash scripts/tests/test_credential_manifest.sh (266 checks pass)
  • skills.json / packs.json regen diff (same normalisation as CI), bin/generate-skill-icons --check, validate-readme-catalog, validate-skill-packs, check-aeon-skill-sync, check-skill-categories, validate-config (+ its tests): all OK
  • eyebrow verify not run locally (not installed); the lock entry was removed together with the skill

…on HQ

The owner decided the connect-check test run is not needed: the first real
run already proves the credential. So:

- No test after saving a credential. The Connect modal now says what was
  saved and that the next run uses it (plus the harness it switched to).
  No Test now / Test again, no /api/connect-check, no page polling hook.
- The HQ Setup checklist marks "Model connected" done once a credential
  for the selected harness is saved.
- ./aeon init drops step 7 "Test connection" and --no-test; Telegram is
  step 7 again. The sandbox test now asserts init never dispatches a run.
- skills/connect-check and its aeon.yml entry are gone; catalog/skills.json,
  packs.json, skill-icons (+ dashboard icon map), eyebrowlock.json and the
  skill counts (86 -> 85) are back in step.

The log reader is kept and repurposed: lib/connect-check.ts is now
lib/run-diagnosis.ts. For a failed run it slices the Run output and error
annotations and returns one plain reason, a next step, and whether the
credential is the problem (and which harness ran, from the run banner).
GET /api/runs/[id]/diagnosis reads the log with gh and caches the verdict.
HQ Recent activity and the run detail panel show it under failed runs only,
fetched lazily and cached, with a Connect button when the credential failed.
…out runs

Match the hosted port (aeon-connect#60):
- HQ Recent activity no longer reads the log of every failed row. Each
  failed or timed-out row has a "Why?" toggle; the log is read only when it
  is opened, and the per-run client cache is kept. The run detail panel
  still loads it right away, since that is one run the operator opened.
- Runs that timed out are diagnosed too (server and client).
Match the hosted port: token expired / invalid_grant / revoked refresh
token / expired session now read "The saved login has expired." with the
next step "Log in again and paste the new login." (credential, so HQ offers
Connect). A plain 401 or invalid key stays "The provider rejected the
credential."
Copy the SIGNATURES list from aeon-connect#60 so canon and hosted say the
same thing: "The saved login expired." / "Log in again and connect the new
login." (also matches "refresh token expired"), and the hosted hints for
rate limit, model not available and no usable credential.
A real Codex run whose ChatGPT login was revoked (aeon-oneshot-test run
37131474562) read as "The provider rejected the credential.": codex died
before the usage notice, so only the ##[error] annotation ("unauthorized
(401)") was read, never the harness output that says token_revoked.

- extractRunOutput: with no usage notice, anchor the slice on the first
  ##[error] line, so the failing step's own output is read.
- Expired-login signature also matches token_revoked and "invalidated
  oauth token".
- Test built from the trimmed real log (gh shape, request ids and cf-ray
  dropped): expired login, credential, harness codex.
…harness

Same as hosted aeon-connect#60: when the "Using harness: X" banner is
missing, the harness comes from the "effective model for X:" notice (outside
script blocks). Banner wins when both are printed. Test copied from hosted.
@aaronjmars
aaronjmars merged commit 6c7ee98 into main Oct 3, 2026
12 checks passed
@aaronjmars aaronjmars mentioned this pull request Oct 3, 2026
5 tasks done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant