Repository navigation
feat(*): raise the attachment upload ceiling to 100 MB - #549
Conversation
A 26 MB deck dropped on the composer was refused. The upload ceiling stood at 25 MB, a number it never chose: it was the file viewer's, adopted when fs.upload landed beside it. The viewer picked 25 MB because no renderer in the page does anything useful past that size, which is not a reason that applies to an upload. An upload answers a different question. It only has to land a path in the workspace -- a non-image attachment is named to the model and never read into the message -- so the bytes exist to reach the disk, not a renderer. MAX_VIEW_BYTES therefore stays where it is, which means a file between the two ceilings can be attached and not previewed; the constant now says so instead of implying the two move together. The transport follows on its own: frame_ceiling_for_upload derives the WebSocket frame ceiling from this constant, so the base64 expansion that once capped attachments at roughly 3 MB cannot reappear from raising the number. The page's mirrored copy moves with it. The ceiling that remains is memory, not policy. One upload at this size costs the gateway the frame, the JSON string parsed out of it and the decoded bytes at once, and the parse holds the event loop while it runs. A much larger limit wants the streamed HTTP upload path, not a bigger number here. Four comments that stated 25 MB as the current ceiling now name the constant instead, so the next change cannot leave them lying. The three that describe the old 3 MB frame bug keep their number: they report what was true then. Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
gloryfromca
left a comment
There was a problem hiding this comment.
No blockers; this can merge as far as I am concerned.
Reviewed github/main...HEAD at 702ca543e00a. I found no issue worth raising.
Coverage included the repository rules in AGENTS.md / CLAUDE.md and CONTEXT-MAP.md; the complete diff; the server upload enforcement and WebSocket frame sizing; the browser-side mirrored limit and the knowledge, composer, and subagent upload callers; the relevant history; backward compatibility of retaining the 25 MB viewer ceiling; the UI-to-runtime RPC boundary; and whether tests were weakened. The test edits only remove stale literal wording, while the existing arithmetic transport contract still derives from MAX_UPLOAD_BYTES. The known encode-before-refusal behavior in two callers predates this change and is not made incorrect by the new ceiling.
Independent verification:
uv run pytest tests/test_rpc_transport.py tests/test_rpc_files.py tests/test_rpc_console.py -q: 172 passednpm test --prefix ui-web: 108 files, 1837 tests passednpm run type-check --prefix ui-web: passed- the
check-source-languagecommand from the Makefile, invoked directly becausemakeis unavailable here: passed git diff --check github/main...HEAD: passed
At posting time all completed GitHub checks pass; only coverage gates is still pending.
The same change as #549, carried onto this branch so the two trunks do not disagree about the ceiling while work continues here. 25 MB was the file viewer's number, adopted when fs.upload landed beside it, and the viewer picked it because no renderer in the page does anything useful past that size. An upload answers a different question: it only has to land a path in the workspace, since a non-image attachment is named to the model and never read into the message. MAX_VIEW_BYTES therefore stays where it is, and a file between the two ceilings can be attached and not previewed; the constant now says so. frame_ceiling_for_upload derives the WebSocket frame ceiling from this constant, so the base64 expansion that once capped attachments at roughly 3 MB cannot reappear from raising the number. Not a cherry-pick: two of the six files in #549 do not exist here. The page's mirrored copy moved from ui-web/src/live/020-rpc.js into ui-web/src/lib/upload.ts, and the two knowledge comments that change swept went with the files this branch removed. raven/rpc/files.py is byte-identical between the branches, so that hunk is the same one. Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
## Summary Second catch-up of `refactor/ui_web_architecture` with `main`, this time to `6ebc2baa` (#592): the 31 commits `main` gained since #529 took it to `dbe259f6`. Synced main to `6ebc2baa`; the next round starts from that SHA. #529 landed as a squash, so git still saw the two branches diverge at `5a29ea22` and re-raised every conflict #529 had already settled (68 paths instead of 33). The first commit here merges `dbe259f6` with `-s ours` -- tree unchanged, ancestry recorded -- so the second commit merges only what is new; a third takes #528 and a fifth takes #592 (READMEs only, no conflict), both of which landed on `main` while the earlier commits were being verified. The same steps work for every later round. Thirty-three paths conflicted in the second commit, one in the third; the fourth commit is what an independent read of the merge turned up (the sheet's Connect, `capabilities_wanted`'s fourth case, the dead styles); the sixth merges this branch's own tip (#584, #591, #586), where only the fixture-shape gate's UNSENT list met (both sides kept). How each conflict was settled: - 17 legacy ui-web files (`live/`, `demo/`, `features/xa`, `features/knowledge`, `scripts/boot-order.test.mjs`, `composer/open-conversation.test.ts`), edited on `main` for #542, #549 and #554: deleted on this branch, and the deletions stand. `main`'s `shell/workdir.ts` and its test, which git relocated into `lib/`, are dropped for the same reason -- see the follow-up below. - `main.tsx`, `page.html`, `state/session/registry.test.ts`: this branch's version. `main`'s edits there were the legacy chip wiring; the registry conflict was a rename git guessed from `scripts/page-switch-live.test.mjs`. - `raven/agent/subagent/probe.py`: both imports; Test records a snapshot for a preset too (`main`, #554) through `record_capabilities` (this branch, #559); a kept record carries `needs_auth`; the boot backfill re-measures an outdated model menu (this branch) and a recorded credential refusal (`main`). - `raven/rpc/methods/subagents.py`: `needs_auth` reads the snapshot this branch already loads per row. - `raven/agent/subagent/manager.py` (#528, third commit): both sides kept at both hunks -- `main`'s two shutdown refusals beside `SPAWN_REFUSED_PREFIX` with this branch's `_row_pin` after them, and the spawn announce that types its mark and carries `node_id` (this branch) then returns early when `_inject` was refused (`main`). - `tests/test_subagent_probe.py`, `test_subagent_acp.py`, `test_rpc_subagents.py`, `test_rpc_console.py`, `test_config_update.py`: both sides' new tests, in order. This branch's two backfill tests pin `_unconfigured_acp_preset_rows` the way `main`'s do, and the `_agent_home` helper both sides defined is one function passing a `Path`, as the config property does. - `README.md`, `README.zh-CN.md`: `main`'s figure and showcase section. - `ui-web/src/rpc/generated.ts`, `ui-tui/src/rpc/generated.ts`, `ui-tui/src/i18n/messages.generated.ts`: regenerated from the merged schema and catalogue. Carried onto the rebuilt tree, since the backend halves merged cleanly: - #554: the agent hub's rows, its sheet and the wizard step name a refused agent `Unauthorized` (disabled) instead of offering Connect; the sheet offers Test beside it, the press that re-measures the handshake and lets the row return to Connect after a sign-in. `capabilities_wanted` treats a recorded refusal as wanting a re-measure, the same fourth case the boot backfill gained from `main`. Four tests. - #542: rail rows carry the session's `workdir`, so the folder group `RailPage.tsx` gained in the merge can fill. The tag's class moves under the rail namespace (`rail-wdt`) with its rule in the new `features/rail/styles.css`, as check-class-namespace requires. - #549: the page-side upload ceiling follows `raven/rpc/files.py` to 100 MB. - The offline fixtures send `needs_auth`; `session.list.sessions[].workdir` is pinned unsent in fixture-shape until the chip lands. The folder chip's thirty lines of popover rules that `main` added to `styles/page.css` are dropped with the chip; the follow-up brings its own. Follow-up, not in this PR: the folder chip itself (#542's `shell/workdir.ts`: the composer chip, the `fs.dirs` browser, `workdir` on `session.create`) has to be rebuilt on this tree's chip pattern (`state/perm.ts` + `chrome/PermChip.tsx`). The backend, the i18n keys and the rail half are already here. ## Type - [ ] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [x] Other (catch-up merge) ## Verification Run in the worktree on the merged tree: - `uv run pytest tests/test_subagent_probe.py tests/test_subagent_acp.py tests/test_rpc_subagents.py tests/test_rpc_console.py tests/test_config_update.py tests/test_rpc_session.py tests/test_rpc_contract_shapes.py tests/test_rpc_schema_match.py tests/test_rpc_registration.py tests/test_i18n_boundary.py tests/test_subagent_manager.py tests/test_subagent_dag_runner.py tests/test_rpc_bootstrap.py tests/test_cli_tui_commands.py tests/test_cli_gateway_commands.py tests/test_rpc_transport.py tests/test_cli_serve_commands.py tests/test_importer_orchestrator.py tests/test_cli_gateway_page.py -q` on the final head -- 1893 passed (the same files, run per commit as the branch grew: 1141, 590, 501). - `make lint-python` -- ruff check clean; ruff format clean after formatting the three spliced test files. - `npm test` in `ui-web` on the final head -- 189 files, 2468 tests passed (before the sync merge: 193 files, 2609; the first run had failed two gates, check-class-namespace and fixture-shape, on the merged `.wdt` class and the three new optional wire fields, both fixed as described above). - `npm run type-check`, `npm run lint` (0 errors, 5 pre-existing warnings), `npm run gen:check` (193 methods) in `ui-web`. - `npm run lint:i18n`, `npm run lint:rpc`, `npm run type-check`, `npm test` in `ui-tui` -- all clean. - `make build-ui` on the final head -- page built, `boot-snapshot: OK` on both snapshots (235 nodes, the golden #591 set). - `scripts/check_source_language.py origin/refactor/ui_web_architecture..HEAD` and `scripts/check_large_files.py origin/refactor/ui_web_architecture..HEAD` -- both exit 0; `git merge-tree --write-tree origin/main HEAD` -- clean. The full Python suite is left to CI. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed ## Risk No source change of its own beyond the resolutions and the three carries above. On the page: a refused external agent now shows a disabled `Unauthorized` button where it showed Connect; conversations pinned to a folder group under their own rail heading with the folder's name on the row; uploads up to 100 MB are sent instead of refused at 25 MB. All three match what `main` ships today. This branch is squash-only, so the merge base stays at `5a29ea22` after this lands; the next round repeats the `-s ours 6ebc2ba` step first. Rollback is reverting the squash commit; the branch is back at `99c37e74`. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues N/A. Unblocks #475. --------- Co-authored-by: xfng-sd <xufang@shanda.com> Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com> Co-authored-by: Handsome-wzw <68996445+Handsome-wzw@users.noreply.github.com> Co-authored-by: Dizhan Xue <dizhan.xue@evermind.ai> Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com> Co-authored-by: admin <admin@SH-HuKai.local> Co-authored-by: 江国庆/Forrest <mr.jianggq@163.com> Co-authored-by: 江国庆 <guoqingjiang@deepglint.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: ypflll <ypflll@163.com> Co-authored-by: yao pengfei <yaopengfei@shanda.com> Co-authored-by: zhanghui <huizhang1995@gmail.com> Co-authored-by: Zuyi Zhou <144661423+ZuyiZhou@users.noreply.github.com> Co-authored-by: Tong Li <litong02@shanda.com> Co-authored-by: litong <238663200+TongLi31@users.noreply.github.com> Co-authored-by: Tchen-data <176354753+Tchen-data@users.noreply.github.com> Co-authored-by: gloryfromca <23442919+gloryfromca@users.noreply.github.com> Co-authored-by: Dizhan Xue <xuedizhan17@mails.ucas.ac.cn> Co-authored-by: silverLXT <25195409+silverLXT@users.noreply.github.com> Co-authored-by: xiaotian.luo <xiaotian.luo@thetahealth.ai> Co-authored-by: userName20260323 <zhao.wang@evermind.ai> Co-authored-by: zhao.wang <270284818+userName20260323@users.noreply.github.com> Co-authored-by: arelchan <1239372199@qq.com> Co-authored-by: arelchan <204152633+arelchan@users.noreply.github.com>
Summary
A 26 MB deck dropped on the composer was refused. The upload ceiling stood at 25 MB, a number it never chose: it was the file viewer's, adopted when
fs.uploadlanded beside it. The viewer picked 25 MB because no renderer in the page does anything useful past that size, which is not a reason that applies to an upload.An upload answers a different question. It only has to land a path in the workspace -- a non-image attachment is named to the model and never read into the message -- so the bytes exist to reach the disk, not a renderer. This raises
MAX_UPLOAD_BYTESto 100 MB and leavesMAX_VIEW_BYTESat 25 MB. A file between the two can therefore be attached and not previewed; the constant's comment now states that instead of implying the two move together.frame_ceiling_for_uploadderives the WebSocket frame ceiling from this constant, so the base64 expansion that once capped attachments at roughly 3 MB cannot come back from raising the number. The page's mirrored copy moves with it.The ceiling that remains is memory, not policy: one upload at this size costs the gateway the frame, the JSON string parsed out of it and the decoded bytes at once, and the parse holds the event loop while it runs. A much larger limit wants the streamed HTTP upload path, not a bigger number here. That is left for its own change.
This also sweeps four comments that stated 25 MB as the current ceiling; they name the constant now. The three describing the old 3 MB frame bug keep their number, because they report what was true then.
Deliberately out of scope: the composer and the subagent page measure an attachment after base64-encoding it rather than before, so an oversized file is read whole and thrown away while the tab blocks.
uploadRefusalBySizealready exists for the pre-encode check and the knowledge page uses it; wiring the other two callers crosses the composer source contract, so it belongs in its own change.Type
Verification
Server side, end to end at the size that was refused (26.2 MB), through the real aiohttp WebSocket and the real method:
fs_uploadaccepts it and the bytes land inuploads/at the right sizeMAX_UPLOAD_BYTES + 1is still refused, with100 MBin the messageThose three ran as a throwaway probe and are not added to the suite. The committed guard is
test_the_frame_ceiling_can_carry_the_largest_upload_the_method_accepts, which reads the constant rather than a literal and so keeps the two limits coupled at any value.Gates run locally, after rebasing onto github/main at 82a51d8:
Page side, in a real browser against a
raven servebuilt from this branch, on an isolated RAVEN_HOME:<workspace>/uploads/at exactly 27,400,000 bytes with its non-ASCII name intactThe drop was a synthesized DragEvent: it exercises the page's own drop handler and the whole upload path, but not the operating system's part of the drag.
No user-facing docs name this ceiling. The one doc mention of 25 MB is the viewer's render cap in
docs/specs/2026-08-26-desk-deliverables-shelf.md, which this change does not touch.Risk
Behaviour change: an attachment up to 100 MB is now accepted where 25 MB was the wall. Per upload the gateway holds the frame, the parsed JSON string and the decoded bytes at once -- roughly 370 MB peak for a maximal one -- and the JSON parse blocks the event loop for a second or more. Single-user desktop use absorbs that; a shared gateway taking concurrent uploads would feel it.
Security: the route is unchanged.
fs.uploadis reachable only over an authorized socket, so the larger allocation is available to the signed-in local reader and to nobody new. What grew is how much memory that reader can ask for in one call.A file between 25 MB and 100 MB can be attached and not opened in the viewer, which still refuses past
MAX_VIEW_BYTES.Rollback: revert the two constants. Nothing persists, no schema or contract moved, and
frame_ceiling_for_uploadrecomputes from whatever the constant says.Related Issues
N/A