Running in Deno

[lonnietc]

Hello,

I am experimenting with Web-Based LLM's to take advantage of the WebGPU type acceleration. I am interested in seeing if it is possible to run the BrowserAI inside a Deno v8 runtime engine (https://deno.com/) and putting an API or Web Server in the front so that the user just needs to connect to the Deno engine.

Still working out some flow ideas here, but the idea is to basically start up Deno with the BrowserAI LLM and then users can connect to it via the web server pages or possible as a headless OpenAI compatible API from other systems.

Any thoughts would be greatly appreciated.

[sirenkovladd]

I think this project is more of a convenient wrapper for the browser.
If you need to do this on the server side (like Deno or NodeJS), it's better to do it through transformers.js.

See the example
https://huggingface.co/onnx-community/Llama-3.2-1B-Instruct