Skip to content

feat(*): raise the attachment upload ceiling to 100 MB - #549

Merged
0xKT merged 1 commit into
mainfrom
feat/upload_limit_100mb
Sep 20, 2026
Merged

0xKT merged 1 commit into
mainfrom
feat/upload_limit_100mb

Conversation

@arelchan

Copy link
Copy Markdown
Contributor

Summary

A 26 MB deck dropped on the composer was refused. The upload ceiling stood at 25 MB, a number it never chose: it was the file viewer's, adopted when fs.upload landed beside it. The viewer picked 25 MB because no renderer in the page does anything useful past that size, which is not a reason that applies to an upload.

An upload answers a different question. It only has to land a path in the workspace -- a non-image attachment is named to the model and never read into the message -- so the bytes exist to reach the disk, not a renderer. This raises MAX_UPLOAD_BYTES to 100 MB and leaves MAX_VIEW_BYTES at 25 MB. A file between the two can therefore be attached and not previewed; the constant's comment now states that instead of implying the two move together.

frame_ceiling_for_upload derives the WebSocket frame ceiling from this constant, so the base64 expansion that once capped attachments at roughly 3 MB cannot come back from raising the number. The page's mirrored copy moves with it.

The ceiling that remains is memory, not policy: one upload at this size costs the gateway the frame, the JSON string parsed out of it and the decoded bytes at once, and the parse holds the event loop while it runs. A much larger limit wants the streamed HTTP upload path, not a bigger number here. That is left for its own change.

This also sweeps four comments that stated 25 MB as the current ceiling; they name the constant now. The three describing the old 3 MB frame bug keep their number, because they report what was true then.

Deliberately out of scope: the composer and the subagent page measure an attachment after base64-encoding it rather than before, so an oversized file is read whole and thrown away while the tab blocks. uploadRefusalBySize already exists for the pre-encode check and the knowledge page uses it; wiring the other two callers crosses the composer source contract, so it belongs in its own change.

Type

  • Fix
  • Feature
  • Docs
  • CI / tooling
  • Refactor
  • Other

Verification

Server side, end to end at the size that was refused (26.2 MB), through the real aiohttp WebSocket and the real method:

  • a frame carrying that file as base64 comes back answered, not disconnected
  • fs_upload accepts it and the bytes land in uploads/ at the right size
  • MAX_UPLOAD_BYTES + 1 is still refused, with 100 MB in the message

Those three ran as a throwaway probe and are not added to the suite. The committed guard is test_the_frame_ceiling_can_carry_the_largest_upload_the_method_accepts, which reads the constant rather than a literal and so keeps the two limits coupled at any value.

Gates run locally, after rebasing onto github/main at 82a51d8:

uv run pytest tests/test_rpc_transport.py tests/test_rpc_files.py tests/test_rpc_console.py -q
  172 passed
make lint-python     All checks passed / 1625 files already formatted
make lint-imports    clean
make lint-deps       Success! No dependency issues found.
make lint-types      All checks passed!
npm run gen:check --prefix ui-web    generated.ts matches the contract (178 methods)
npm run type-check --prefix ui-web   clean
npm test --prefix ui-web             108 files, 1837 tests passed
node ui-web/scripts/check-page.mjs             OK
node ui-web/scripts/count-shared-globals.mjs   OK
node ui-web/scripts/check-css.mjs              OK
node ui-web/scripts/check-class-namespace.mjs  OK

Page side, in a real browser against a raven serve built from this branch, on an isolated RAVEN_HOME:

  • a 26.1 MB .pptx dropped on the composer attaches: the chip goes solid at "26.1 MB" with no failure note, and the file lands in <workspace>/uploads/ at exactly 27,400,000 bytes with its non-ASCII name intact
  • a 105 MB file is refused, and the note names the new ceiling ("105.0 MB", "100.0 MB"), so the limit was raised rather than removed and the message follows the constant

The drop was a synthesized DragEvent: it exercises the page's own drop handler and the whole upload path, but not the operating system's part of the drag.

No user-facing docs name this ceiling. The one doc mention of 25 MB is the viewer's render cap in docs/specs/2026-08-26-desk-deliverables-shelf.md, which this change does not touch.

  • Relevant tests pass locally
  • Relevant lint / type checks pass locally
  • User-facing docs or screenshots are updated when needed

Risk

Behaviour change: an attachment up to 100 MB is now accepted where 25 MB was the wall. Per upload the gateway holds the frame, the parsed JSON string and the decoded bytes at once -- roughly 370 MB peak for a maximal one -- and the JSON parse blocks the event loop for a second or more. Single-user desktop use absorbs that; a shared gateway taking concurrent uploads would feel it.

Security: the route is unchanged. fs.upload is reachable only over an authorized socket, so the larger allocation is available to the signed-in local reader and to nobody new. What grew is how much memory that reader can ask for in one call.

A file between 25 MB and 100 MB can be attached and not opened in the viewer, which still refuses past MAX_VIEW_BYTES.

Rollback: revert the two constants. Nothing persists, no schema or contract moved, and frame_ceiling_for_upload recomputes from whatever the constant says.

  • Security impact considered
  • Backward compatibility considered
  • Rollback path is clear for risky changes

Related Issues

N/A

A 26 MB deck dropped on the composer was refused. The upload ceiling
stood at 25 MB, a number it never chose: it was the file viewer's,
adopted when fs.upload landed beside it. The viewer picked 25 MB because
no renderer in the page does anything useful past that size, which is
not a reason that applies to an upload.

An upload answers a different question. It only has to land a path in
the workspace -- a non-image attachment is named to the model and never
read into the message -- so the bytes exist to reach the disk, not a
renderer. MAX_VIEW_BYTES therefore stays where it is, which means a file
between the two ceilings can be attached and not previewed; the constant
now says so instead of implying the two move together.

The transport follows on its own: frame_ceiling_for_upload derives the
WebSocket frame ceiling from this constant, so the base64 expansion that
once capped attachments at roughly 3 MB cannot reappear from raising the
number. The page's mirrored copy moves with it.

The ceiling that remains is memory, not policy. One upload at this size
costs the gateway the frame, the JSON string parsed out of it and the
decoded bytes at once, and the parse holds the event loop while it runs.
A much larger limit wants the streamed HTTP upload path, not a bigger
number here.

Four comments that stated 25 MB as the current ceiling now name the
constant instead, so the next change cannot leave them lying. The three
that describe the old 3 MB frame bug keep their number: they report what
was true then.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

Reviewed github/main...HEAD at 702ca543e00a. I found no issue worth raising.

Coverage included the repository rules in AGENTS.md / CLAUDE.md and CONTEXT-MAP.md; the complete diff; the server upload enforcement and WebSocket frame sizing; the browser-side mirrored limit and the knowledge, composer, and subagent upload callers; the relevant history; backward compatibility of retaining the 25 MB viewer ceiling; the UI-to-runtime RPC boundary; and whether tests were weakened. The test edits only remove stale literal wording, while the existing arithmetic transport contract still derives from MAX_UPLOAD_BYTES. The known encode-before-refusal behavior in two callers predates this change and is not made incorrect by the new ceiling.

Independent verification:

  • uv run pytest tests/test_rpc_transport.py tests/test_rpc_files.py tests/test_rpc_console.py -q: 172 passed
  • npm test --prefix ui-web: 108 files, 1837 tests passed
  • npm run type-check --prefix ui-web: passed
  • the check-source-language command from the Makefile, invoked directly because make is unavailable here: passed
  • git diff --check github/main...HEAD: passed

At posting time all completed GitHub checks pass; only coverage gates is still pending.

@0xKT
0xKT merged commit d0ff0ba into main Sep 20, 2026
21 checks passed
@0xKT
0xKT deleted the feat/upload_limit_100mb branch September 20, 2026 12:32
arelchan added a commit that referenced this pull request Sep 21, 2026
The same change as #549, carried onto this branch so the two trunks do
not disagree about the ceiling while work continues here.

25 MB was the file viewer's number, adopted when fs.upload landed beside
it, and the viewer picked it because no renderer in the page does
anything useful past that size. An upload answers a different question:
it only has to land a path in the workspace, since a non-image
attachment is named to the model and never read into the message.
MAX_VIEW_BYTES therefore stays where it is, and a file between the two
ceilings can be attached and not previewed; the constant now says so.

frame_ceiling_for_upload derives the WebSocket frame ceiling from this
constant, so the base64 expansion that once capped attachments at
roughly 3 MB cannot reappear from raising the number.

Not a cherry-pick: two of the six files in #549 do not exist here. The
page's mirrored copy moved from ui-web/src/live/020-rpc.js into
ui-web/src/lib/upload.ts, and the two knowledge comments that change
swept went with the files this branch removed. raven/rpc/files.py is
byte-identical between the branches, so that hunk is the same one.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
gloryfromca added a commit that referenced this pull request Sep 21, 2026
## Summary

Second catch-up of `refactor/ui_web_architecture` with `main`, this time
to
`6ebc2baa` (#592): the 31 commits `main` gained since #529 took it to
`dbe259f6`. Synced main to `6ebc2baa`; the next round starts from that
SHA.

#529 landed as a squash, so git still saw the two branches diverge at
`5a29ea22` and re-raised every conflict #529 had already settled (68
paths
instead of 33). The first commit here merges `dbe259f6` with `-s ours`
--
tree unchanged, ancestry recorded -- so the second commit merges only
what
is new; a third takes #528 and a fifth takes #592 (READMEs only, no
conflict), both of which landed on `main` while the earlier commits were
being verified. The same steps work for every later round.

Thirty-three paths conflicted in the second commit, one in the third;
the
fourth commit is what an independent read of the merge turned up (the
sheet's Connect, `capabilities_wanted`'s fourth case, the dead styles);
the
sixth merges this branch's own tip (#584, #591, #586), where only the
fixture-shape gate's UNSENT list met (both sides kept). How each
conflict
was settled:

- 17 legacy ui-web files (`live/`, `demo/`, `features/xa`,
  `features/knowledge`, `scripts/boot-order.test.mjs`,
  `composer/open-conversation.test.ts`), edited on `main` for #542, #549
  and #554: deleted on this branch, and the deletions stand. `main`'s
  `shell/workdir.ts` and its test, which git relocated into `lib/`, are
  dropped for the same reason -- see the follow-up below.
- `main.tsx`, `page.html`, `state/session/registry.test.ts`: this
branch's
version. `main`'s edits there were the legacy chip wiring; the registry
conflict was a rename git guessed from
`scripts/page-switch-live.test.mjs`.
- `raven/agent/subagent/probe.py`: both imports; Test records a snapshot
  for a preset too (`main`, #554) through `record_capabilities` (this
  branch, #559); a kept record carries `needs_auth`; the boot backfill
  re-measures an outdated model menu (this branch) and a recorded
  credential refusal (`main`).
- `raven/rpc/methods/subagents.py`: `needs_auth` reads the snapshot this
  branch already loads per row.
- `raven/agent/subagent/manager.py` (#528, third commit): both sides
kept
at both hunks -- `main`'s two shutdown refusals beside
`SPAWN_REFUSED_PREFIX`
  with this branch's `_row_pin` after them, and the spawn announce that
types its mark and carries `node_id` (this branch) then returns early
when
  `_inject` was refused (`main`).
- `tests/test_subagent_probe.py`, `test_subagent_acp.py`,
`test_rpc_subagents.py`, `test_rpc_console.py`, `test_config_update.py`:
  both sides' new tests, in order. This branch's two backfill tests pin
  `_unconfigured_acp_preset_rows` the way `main`'s do, and the
`_agent_home` helper both sides defined is one function passing a
`Path`,
  as the config property does.
- `README.md`, `README.zh-CN.md`: `main`'s figure and showcase section.
- `ui-web/src/rpc/generated.ts`, `ui-tui/src/rpc/generated.ts`,
  `ui-tui/src/i18n/messages.generated.ts`: regenerated from the merged
  schema and catalogue.

Carried onto the rebuilt tree, since the backend halves merged cleanly:

- #554: the agent hub's rows, its sheet and the wizard step name a
refused
  agent `Unauthorized` (disabled) instead of offering Connect; the sheet
offers Test beside it, the press that re-measures the handshake and lets
the row return to Connect after a sign-in. `capabilities_wanted` treats
a
recorded refusal as wanting a re-measure, the same fourth case the boot
  backfill gained from `main`. Four tests.
- #542: rail rows carry the session's `workdir`, so the folder group
`RailPage.tsx` gained in the merge can fill. The tag's class moves under
  the rail namespace (`rail-wdt`) with its rule in the new
  `features/rail/styles.css`, as check-class-namespace requires.
- #549: the page-side upload ceiling follows `raven/rpc/files.py` to 100
MB.
- The offline fixtures send `needs_auth`;
`session.list.sessions[].workdir`
is pinned unsent in fixture-shape until the chip lands. The folder
chip's
thirty lines of popover rules that `main` added to `styles/page.css` are
  dropped with the chip; the follow-up brings its own.

Follow-up, not in this PR: the folder chip itself (#542's
`shell/workdir.ts`: the composer chip, the `fs.dirs` browser, `workdir`
on
`session.create`) has to be rebuilt on this tree's chip pattern
(`state/perm.ts` + `chrome/PermChip.tsx`). The backend, the i18n keys
and
the rail half are already here.

## Type

- [ ] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [x] Other (catch-up merge)

## Verification

Run in the worktree on the merged tree:

- `uv run pytest tests/test_subagent_probe.py tests/test_subagent_acp.py
tests/test_rpc_subagents.py tests/test_rpc_console.py
tests/test_config_update.py tests/test_rpc_session.py
tests/test_rpc_contract_shapes.py tests/test_rpc_schema_match.py
tests/test_rpc_registration.py tests/test_i18n_boundary.py
tests/test_subagent_manager.py tests/test_subagent_dag_runner.py
tests/test_rpc_bootstrap.py tests/test_cli_tui_commands.py
tests/test_cli_gateway_commands.py tests/test_rpc_transport.py
tests/test_cli_serve_commands.py tests/test_importer_orchestrator.py
tests/test_cli_gateway_page.py -q` on the final head -- 1893 passed (the
same files, run per commit as the branch grew: 1141, 590, 501).
- `make lint-python` -- ruff check clean; ruff format clean after
formatting the three spliced test files.
- `npm test` in `ui-web` on the final head -- 189 files, 2468 tests
passed (before the sync merge: 193 files, 2609; the first run had failed
two gates, check-class-namespace and fixture-shape, on the merged `.wdt`
class and the three new optional wire fields, both fixed as described
above).
- `npm run type-check`, `npm run lint` (0 errors, 5 pre-existing
warnings), `npm run gen:check` (193 methods) in `ui-web`.
- `npm run lint:i18n`, `npm run lint:rpc`, `npm run type-check`, `npm
test` in `ui-tui` -- all clean.
- `make build-ui` on the final head -- page built, `boot-snapshot: OK`
on both snapshots (235 nodes, the golden #591 set).
- `scripts/check_source_language.py
origin/refactor/ui_web_architecture..HEAD` and
`scripts/check_large_files.py origin/refactor/ui_web_architecture..HEAD`
-- both exit 0; `git merge-tree --write-tree origin/main HEAD` -- clean.

The full Python suite is left to CI.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

No source change of its own beyond the resolutions and the three carries
above. On the page: a refused external agent now shows a disabled
`Unauthorized` button where it showed Connect; conversations pinned to a
folder group under their own rail heading with the folder's name on the
row; uploads up to 100 MB are sent instead of refused at 25 MB. All
three
match what `main` ships today.

This branch is squash-only, so the merge base stays at `5a29ea22` after
this lands; the next round repeats the `-s ours 6ebc2ba` step first.
Rollback is reverting the squash commit; the branch is back at
`99c37e74`.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A. Unblocks #475.

---------

Co-authored-by: xfng-sd <xufang@shanda.com>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
Co-authored-by: Handsome-wzw <68996445+Handsome-wzw@users.noreply.github.com>
Co-authored-by: Dizhan Xue <dizhan.xue@evermind.ai>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
Co-authored-by: admin <admin@SH-HuKai.local>
Co-authored-by: 江国庆/Forrest <mr.jianggq@163.com>
Co-authored-by: 江国庆 <guoqingjiang@deepglint.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: ypflll <ypflll@163.com>
Co-authored-by: yao pengfei <yaopengfei@shanda.com>
Co-authored-by: zhanghui <huizhang1995@gmail.com>
Co-authored-by: Zuyi Zhou <144661423+ZuyiZhou@users.noreply.github.com>
Co-authored-by: Tong Li <litong02@shanda.com>
Co-authored-by: litong <238663200+TongLi31@users.noreply.github.com>
Co-authored-by: Tchen-data <176354753+Tchen-data@users.noreply.github.com>
Co-authored-by: gloryfromca <23442919+gloryfromca@users.noreply.github.com>
Co-authored-by: Dizhan Xue <xuedizhan17@mails.ucas.ac.cn>
Co-authored-by: silverLXT <25195409+silverLXT@users.noreply.github.com>
Co-authored-by: xiaotian.luo <xiaotian.luo@thetahealth.ai>
Co-authored-by: userName20260323 <zhao.wang@evermind.ai>
Co-authored-by: zhao.wang <270284818+userName20260323@users.noreply.github.com>
Co-authored-by: arelchan <1239372199@qq.com>
Co-authored-by: arelchan <204152633+arelchan@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants