This library provides a simple way to interact with the DeepInfra API.
Check out our docs here.
npm install deepinfraCreate an isolated Linux microVM, run bash/python inside it, move files in and out, and tear it down — in a few lines:
import { Sandbox } from "deepinfra";
const sb = await Sandbox.create({ plan: "medium", timeout: "10m" }); // resolves once running
const r = await sb.exec(
"bash",
"-c",
"pip install --break-system-packages pandas && python3 -c 'import pandas; print(pandas.__version__)'",
);
console.log(r.stdout, r.stderr, r.returncode);
const out = await sb.runPython("print(21 * 2)");
out.check(); // throws CommandFailedError on a non-zero exit code
await sb.fs.write("/workspace/in.csv", "a,b\n1,2\n");
const data = await sb.fs.read("/workspace/in.csv"); // Buffer
await sb.stop(); // frees compute, keeps disk; resolves once stopped
await sb.start(); // resumes on the same disk; resolves once running
await sb.terminate(); // deletes the sandbox (stays fetchable by id as "deleted" briefly)/workspace is the only path fs.write/fs.read accept, and it's the only
part of the filesystem that survives stop()/start() — everything else
(e.g. packages installed with pip/apt outside it) resets to the base
image on restart, and the sandbox's idle timeout has the same effect as an
explicit stop().
Auto-terminate with try/finally:
const sb = await Sandbox.create({ plan: "small" });
try {
await sb.runPython("open('/workspace/out.txt', 'w').write('hi')");
console.log((await sb.fs.read("/workspace/out.txt")).toString());
} finally {
await sb.terminate();
}Other lookups:
import fs from "node:fs";
// Find existing sandboxes
const sb = await Sandbox.fromId("sb_...");
const etlBoxes = await Sandbox.list({ tags: { job: "etl-42" } });
// List available plans (id, vcpu, ram_gb, disk_gb, price_per_hour)
for (const plan of await Sandbox.catalog()) {
console.log(plan.id, plan.vcpu, plan.ram_gb, plan.price_per_hour);
}
// Large scripts: upload, then run
await sb.fs.write(
"/workspace/script.py",
await fs.promises.readFile("script.py", "utf8"),
);
await sb.exec("python3", "/workspace/script.py", { timeout: "30m" });Errors are typed: AuthenticationError (401), NotFoundError (404),
ConflictError (409, e.g. exec on a stopped sandbox), RateLimitError
(429; for sandboxes that's the per-account cap — TooManySandboxesError is
an alias), CapacityError (503), plus SDK-side SandboxTimeoutError /
SandboxFailedError / CommandFailedError. If Sandbox.create() fails
while waiting for it to come up, the raised error carries .sandboxId so you
can inspect or terminate the sandbox it created.
See examples/sandbox-quickstart.ts for a runnable end-to-end example.
The Mixtral mixture of expert model, developed by Mistral AI, is an innovative experimental machine learning model that leverages a mixture of 8 experts (MoE) within 7b models. Its release was facilitated via a torrent, and the model's implementation remains in the experimental phase._
import {TextGeneration} from "deepinfra";
const modelName = "mistralai/Mixtral-8x22B-Instruct-v0.1";
const apiKey = "YOUR_DEEPINFRA_API_KEY";
const main = async () => {
const mixtral = new TextGeneration(modelName, apiKey);
const body = {
input: "What is the capital of France?",
};
const output = await mixtral.generate(body);
const text = output.results[0].generated_text;
console.log(text);
};
main();Gte Base is an text embedding model that generates embeddings for the input text. The model is trained by Alibaba DAMO Academy.
import { GteBase } from "deepinfra";
const apiKey = "YOUR_DEEPINFRA_API_KEY";
const modelName = "thenlper/gte-base";
const main = async () => {
const gteBase = new Embeddings(modelName, apiKey);
const body = {
inputs: [
"What is the capital of France?",
"What is the capital of Germany?",
"What is the capital of Italy?",
],
};
const output = await gteBase.generate(body);
const embeddings = output.embeddings[0];
console.log(embeddings);
};
main();Use SDXL to generate images
SDXL requires unique parameters, therefore it requires different initialization.
import { Sdxl } from "deepinfra";
import axios from "axios";
import fs from "fs";
const apiKey = "YOUR_DEEPINFRA_API_KEY";
const main = async () => {
const model = new Sdxl(apiKey);
const input = {
prompt: "The quick brown fox jumps over the lazy dog with",
};
const response = await model.generate({ input });
const { output } = response;
const image = output[0];
await axios.get(image, { responseType: "arraybuffer" }).then((response) => {
fs.writeFileSync("image.png", response.data);
});
};
main();import { TextToImage } from "deepinfra";
import axios from "axios";
import fs from "fs";
const apiKey = "YOUR_DEEPINFRA_API_KEY";
const modelName = "stabilityai/stable-diffusion-2-1";
const main = async () => {
const model = new TextToImage(modelName, apiKey);
const input = {
prompt: "The quick brown fox jumps over the lazy dog with",
};
const response = await model.generate(input);
const { output } = response;
const image = output[0];
await axios.get(image, { responseType: "arraybuffer" }).then((response) => {
fs.writeFileSync("image.png", response.data);
});
};
main();import { AutomaticSpeechRecognition } from "deepinfra";
const apiKey = "YOUR_DEEPINFRA_API_KEY";
const modelName = "openai/whisper-base";
const main = async () => {
const model = new AutomaticSpeechRecognition(modelName, apiKey);
const input = {
audio: path.join(__dirname, "audio.mp3"),
};
const response = await model.generate(input);
const { text } = response;
console.log(text);
};
main();import { ObjectDetection } from "deepinfra";
const apiKey = "YOUR_DEEPINFRA_API_KEY";
const modelName = "hustvl/yolos-tiny";
const main = async () => {
const model = new ObjectDetection(modelName, apiKey);
const input = {
image: path.join(__dirname, "image.jpg"),
};
const response = await model.generate(input);
const { results } = response;
console.log(results);
};import { TokenClassification } from "deepinfra";
const apiKey = "YOUR_DEEPINFRA_API_KEY";
const modelName = "Davlan/bert-base-multilingual-cased-ner-hrl";
const main = async () => {
const model = new TokenClassification(modelName, apiKey);
const input = {
text: "The quick brown fox jumps over the lazy dog",
};
const response = await model.generate(input);
const { results } = response;
console.log(results);
};Use fill mask models
import { FillMask } from "deepinfra";
const apiKey = "YOUR_DEEPINFRA_API_KEY";
const modelName = "GroNLP/bert-base-dutch-cased";
const main = async () => {
const model = new FillMask(modelName, apiKey);
const body = {
input: "Ik heb een [MASK] gekocht.",
};
const { results } = await model.generate(body);
console.log(results);
};import { ImageClassification } from "deepinfra";
const apiKey = "YOUR_DEEPINFRA_API_KEY";
const modelName = "google/vit-base-patch16-224";
const main = async () => {
const model = new ImageClassification(modelName, apiKey);
const body = {
image: path.join(__dirname, "image.jpg"),
};
const { results } = await model.generate(body);
console.log(results);
};import { ZeroShotImageClassification } from "deepinfra";
const apiKey = "YOUR_DEEPINFRA_API_KEY";
const modelName = "openai/clip-vit-base-patch32";
const main = async () => {
const model = new ZeroShotImageClassification(modelName, apiKey);
const body = {
image: path.join(__dirname, "image.jpg"),
candidate_labels: ["dog", "cat", "car"],
};
const { results } = await model.generate(body);
console.log(results);
};import { TextClassification } from "deepinfra";
const apiKey = "YOUR_DEEPINFRA_API_KEY";
const modelName = "ProsusAI/finbert";
const misc = async () => {
const model = new TextClassification(apiKey);
const body = {
input:
"DeepInfra emerges from stealth with $8M to make running AI inferences more affordable",
};
const { results } = await model.generate(body);
console.log(results);
};Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.
This project is licensed under the MIT License - see the LICENSE file for details.