Skip to content

feat: backend comparison benchmarks - #2299

Open
vados-cosmonic wants to merge 6 commits into
bytecodealliance:mainfrom
vados-cosmonic:feat/perry-backend-comparison
Open

vados-cosmonic wants to merge 6 commits into
bytecodealliance:mainfrom
vados-cosmonic:feat/perry-backend-comparison

Conversation

@vados-cosmonic

Copy link
Copy Markdown
Collaborator

No description provided.

@vados-cosmonic
vados-cosmonic force-pushed the feat/perry-backend-comparison branch from 4aef636 to feaa7d2 Compare October 22, 2026 16:15
| -------------------------------- | ------------: | -------------: |
| Perry | 11.42 MiB | 7.40 MiB |
| componentize-js (StarlingMonkey) | 15.99 MiB | 15.42 MiB |
| componentize-qjs (QuickJS) | 6.79 MiB | 6.66 MiB |

@andreiltd andreiltd Oct 22, 2026 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Size of Perry and QuickJS components is a bit surprising here. For example componentize-qjs@0.5.0 hello world is producing component that is ~1.4M, and just under 1M when you use --opt-size. I would expect Perry is even less than that? Is it the size of script or is it aot compiled with wasmtime?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I was incredibly confused by this as well which is why this hasn't merged -- trying to figure out what is causing I expected something in the 100s of kbs.

My guess is that Perry here is including all the nodejs compat by default, but I need to figure out if there's an easy minimal build.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK so yeah it looks like the main reason is primarily the included JS runtime that ends up being pulled in, and you can get down to 3.xMB... But how to consistently get it smaller might require more work.

Being able to do this dynamically for components (and get a small build) is going to be somewhat difficult, and those changes would also probably need to go up to perry-wit anyway if we pick that up).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting! Just for reference, here are componentize-qjs hello world sizes:

$ componentize-qjs --wit examples/hello.wit --js examples/hello.js -o hello.wasm
componentize-qjs
  WIT:    examples/hello.wit
  JS:     examples/hello.js
  Output: hello.wasm
Component written to hello.wasm
  Size: 1450762 bytes

$ componentize-qjs --wit examples/hello.wit --js examples/hello.js -o hello.wasm --opt-size
componentize-qjs
  WIT:    examples/hello.wit
  JS:     examples/hello.js
  Output: hello.wasm
Component written to hello.wasm
  Size: 966512 bytes

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When it comes to optimization, qjs comes with prebuilt runtime components which have already been optimized with wasm-opt for speed or size. --opt-size switches runtime component.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah it's a bit unfortunate but I think Perry isn't at a point where it can produce the size of binaries that match what we expect just yet.

I'm starting to think that actually what we want if we're going to AOT in the first place is not to bring a runtime at all and an approach like https://github.com/drogus/jawsm is what we want for truly small binaries.

Clearly there needs to be more work done in perry to make it more minimal!

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if there is an easy win here opt-ing all the prebuilt runtime components that perry uses as well.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about quickjs? Why is it 4x bigger than expected?

@vados-cosmonic vados-cosmonic Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah so it looks like it's the node adapter code-- note that the code in question uses the node SDK as if it's real: https://github.com/vados-cosmonic/jco/blob/5d95cb6f3d60327a5b260d1a6a2ca1c20bf810e9/packages/jco/test/bench/fixtures/filesystem/guest.ts

Perry has built in NodeJS SDK support, but componentize-qjs doesn't so we use the jco-provided NodeJS SDK (at least only the parts pulled in by fliesystem stuff should be added).

I think the file system adapter may not be as tree-shakeable as I thought, and once you have that code initialized in the quickJS wasm snapshot it gets much bigger.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to start a new branch and put some 🤖 s on whether jco-std can be refactored to tree shake better and reduce the size.

@vados-cosmonic
vados-cosmonic force-pushed the feat/perry-backend-comparison branch from 5d95cb6 to 44a574b Compare October 23, 2026 09:47
@vados-cosmonic
vados-cosmonic force-pushed the feat/perry-backend-comparison branch from 5237dcf to 3ebc5f7 Compare October 26, 2026 03:19

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants