Inference Bridge

Official reference implementation of the Inference Provider API (IPA).

Inference Bridge is a Manifest V3 Chrome extension that injects window.inference, prompts for per-origin permission, and routes chat requests to a user-chosen provider (OpenAI, Anthropic, OpenRouter, local Ollama, or user-configured OpenAI-compatible servers). API keys stay in the extension. Page scripts never see them.

The specification defines the API contract. This repository implements that contract and may also ship experimental capabilities that are not part of the standard yet. Experimental features will be clearly labeled; they do not silently expand the core API.

Features

window.inference.request() for streaming text chat
window.inference.getFeatures() (toolCalling: false until tools graduate to stable request)
Per-origin Allow / Deny / Remember permission flow
User-controlled provider and model selection
OpenAI (BYOK), Anthropic (BYOK), OpenRouter (BYOK), and local Ollama support
Named OpenAI-compatible endpoints (LM Studio, llama.cpp, vLLM, etc.)
Experimental function tools via window.inference.experimental (page-executed relay, optional runTools loop)
Origin/Referer stripping for local Ollama and other loopback OpenAI-compatible servers (no OLLAMA_ORIGINS required in the common case)
Secure-context injection only (https: or loopback http:)

Installation

Chrome Web Store (recommended)

Install from the Chrome Web Store.

For local development or unreleased builds, use the load-unpacked steps below.

Development (Load unpacked)

Clone this repository
Open chrome://extensions
Enable Developer mode
Click Load unpacked
Select this repository root
Open the extension Options page (or click the toolbar icon)
Choose a default provider:
- OpenAI — paste your API key and choose a default model
- Anthropic — paste your Anthropic API key and choose a Claude model
- OpenRouter — paste your OpenRouter API key; models load from the public catalog (searchable)
- Ollama — no API key; models are listed from your local Ollama install
- OpenAI-compatible — add named servers under OpenAI-compatible servers (Chrome prompts for that host only on save)
Click Save

Supported Providers

Provider	Auth	Notes
OpenAI	API key in Options	Curated chat model list in the UI
Anthropic	API key in Options	Curated Claude model list in the UI; Messages API (not Chat Completions)
OpenRouter	API key in Options	Live catalog from `GET /api/v1/models`; searchable autosuggest
Ollama	None	Fixed at `http://localhost:11434`; models from `GET /api/tags`
OpenAI-compatible	Optional API key	User-named endpoints; `<select>` for small `GET /v1/models` catalogs, searchable autosuggest when large, free-text fallback when empty; chat via `/v1/chat/completions`

To add another built-in provider: implement the same shape as src/providers/openai.js / src/providers/anthropic.js / src/providers/ollama.js / src/providers/openrouter.js (shared OpenAI-compatible streaming lives in src/providers/openai-compat-stream.js; Anthropic uses a dedicated Messages API adapter). Models use the ModelInfo contract in src/providers/types.js. Register the provider in src/providers/registry.js, and extend the options UI if it needs extra credentials. For most local/self-hosted OpenAI-compatible servers, use the named-endpoint UI instead.

Try it

OpenAI

Set Default provider to OpenAI and save an API key
On any top-level HTTPS page (or http://localhost), open DevTools and run:

for await (const chunk of window.inference.request({
  method: "chat",
  messages: [{ role: "user", content: "Say hello in one short sentence." }],
})) {
  if (chunk.type === "accepted") {
    console.log("accepted");
  } else if (chunk.type === "reasoning_delta") {
    console.log("reasoning", chunk.content);
  } else if (chunk.type === "delta") {
    console.log("delta", chunk.content);
  } else if (chunk.type === "done") {
    console.log("done", chunk.model, chunk.message, chunk.usage);
  }
}

Anthropic

Create an API key at console.anthropic.com
In Options, paste the key under Anthropic API key, set Default provider to Anthropic (defaults to claude-sonnet-5; pick another Claude model from the list), and click Save
Run the snippet above on an HTTPS or localhost page

Anthropic uses the Messages API (POST /v1/messages), not OpenAI Chat Completions.

OpenRouter

Create an API key at openrouter.ai/keys
In Options, paste the key under OpenRouter API key, set Default provider to OpenRouter (defaults to openrouter/auto; type to search for others), and click Save
Run the snippet above on an HTTPS or localhost page

Ollama (no API key)

Install Ollama and start it (default: http://localhost:11434)
Pull at least one chat model, for example:
```
ollama pull gemma4
```
In Options, set Default provider to Ollama (enabled only when Ollama is reachable and has at least one model)
Confirm the model field populates from installed models (type to filter)
Click Save
Run the snippet above on an HTTPS or localhost page

Inference Bridge strips the chrome-extension:// Origin header on requests to local Ollama (see the spec README “Local providers” section). After updating the extension, use Reload on chrome://extensions so that rule is installed. If you still see HTTP 403, you can fall back to restarting Ollama with OLLAMA_ORIGINS=chrome-extension://*, but origin stripping is preferred.

OpenAI-compatible servers

Use this for LM Studio, llama.cpp server, vLLM, LocalAI, or any self-hosted proxy that exposes OpenAI-style /v1/models and /v1/chat/completions.

Start your server and note its host URL (e.g. http://127.0.0.1:1234 or http://192.168.1.67:1234)
In Options, under OpenAI-compatible servers, enter a Name, Base URL, and optional API key — /v1 is appended automatically if you omit it (unusual paths like /openai/v1 are kept as entered)
Click Add server — Chrome prompts for host access to that origin only; deny means the endpoint is not saved
Set Default provider to the new named entry, pick or type a model, and click Save
Run the snippet above on an HTTPS or localhost page

Ollama remains a first-class built-in provider; you do not need to re-add it as a custom endpoint. Loopback servers get the same Origin/Referer stripping used for Ollama. Remote HTTPS endpoints do not.

Abort example:

const controller = new AbortController();
const iter = window.inference.request({
  method: "chat",
  messages: [{ role: "user", content: "Write a long poem." }],
  signal: controller.signal,
});

setTimeout(() => controller.abort(), 500);

try {
  for await (const chunk of iter) {
    console.log(chunk);
  }
} catch (err) {
  console.log(err.code); // "aborted"
}

Example apps: try the live IPA examples gallery (chat, social Ask AI, and more). Source lives in the specification repository.

For app-side helpers (TypeScript types, drain a stream to done, page-executed tool loops), see ipa-tools — optional; not required to use this extension.

Security

Injects only into top-level frames
Requires a secure context (https: or localhost / loopback http:)
Does not inject into file: pages
Permission is per HTTP(S) origin and records the chosen provider + model
Request validation happens in the extension before any provider call
OpenAI / Anthropic / OpenRouter credentials are read only inside the service worker
Ollama traffic stays on http://localhost:11434 / http://127.0.0.1:11434
Local Ollama and other loopback OpenAI-compatible requests drop the extension Origin / Referer headers via declarativeNetRequestWithHostAccess (host-scoped per endpoint)
Optional host permissions for custom OpenAI-compatible servers are requested only for the exact origin the user saves

If you are building your own IPA extension with local providers, follow the Origin-stripping guidance in the specification and reuse or adapt src/ollama-origin-bypass.js / src/loopback-origin-bypass.js. Keep host permissions and DNR rules tight; do not apply header stripping to remote APIs.

Architecture

.
  manifest.json
  package.json                 # vitest + packaging scripts
  icons/                       # toolbar / extension management PNGs (16, 48, 128)
  test/                        # validate / storage / permissions / registry
  background/service-worker.js   # permissions + orchestration
  content/inject.js              # MAIN world: window.inference
  content/content-script.js      # ISOLATED relay
  src/
    errors.js
    validate.js
    run-tools.js                 # page-side experimental.runTools (tests; inject.js mirrors)
    storage.js
    permissions.js
    ollama-origin-bypass.js      # strip chrome-extension Origin for local Ollama
    loopback-origin-bypass.js    # same for other loopback OpenAI-compatible hosts
    host-permissions.js          # optional host permission helpers for custom endpoints
    providers/
      types.js                   # Provider / ModelInfo contract
      registry.js                # built-ins + dynamic compat endpoints
      openai-compat-stream.js    # shared OpenAI-compatible SSE streaming
      openai-compat.js           # factory for user-named OpenAI-compatible servers
      openai.js                  # OpenAI streaming adapter
      anthropic.js               # Anthropic Messages API streaming adapter
      openrouter.js              # OpenRouter /api/v1 models + chat adapter
      ollama.js                  # Ollama /api/tags + /api/chat adapter
  ui/
    options.html|.js             # provider + model + API keys + compat endpoints
    approval.html|.js            # origin permission prompt
    model-input.js               # shared model autosuggest (input + datalist)
    shared.css
  scripts/
    package.mjs                  # Chrome Web Store ZIP packaging

Standards Compatibility

Area	Status
Spec contract (`window.inference.request`, `getFeatures`, streaming, abort, errors)	Implemented
Text chat	Implemented
Per-origin permission UX	Implemented (extension UX; not part of the API contract)
Feature discovery	Implemented; `getFeatures()` returns `{ toolCalling: false }`
Tools	Optional in IPA (`getFeatures().toolCalling`); Bridge-experimental only until graduation
Vision / audio / embeddings	Not implemented; treat as future experimental candidates

The specification remains intentionally small. Provider-specific or advanced capabilities should land here as experimental features first, then be proposed for the specification only after real multi-provider experience.

Experimental Features

Experimental APIs are Inference Bridge–specific. They are not part of the IPA contract. Apps that depend on them should call window.inference.experimental so the opt-in is visible in source. If a capability later graduates into IPA, migrate callers from experimental.request → request. The page-side tool loop already lives in ipa-tools for real apps; Bridge also exposes experimental.runTools for DevTools / no-bundler demos. Neither belongs on stable window.inference.

Named OpenAI-compatible servers are a first-class Bridge provider option (see Supported Providers); they are not part of this experimental page API.

Function tools

Stable IPA chat stays SPEC-faithful. Tool calling is only available through the experimental namespace:

window.inference.getFeatures()              // { toolCalling: false } — stable request surface only
window.inference.request(...)               // IPA-stable chat only
window.inference.experimental.request(...)  // Bridge experimental (tools, etc.)
window.inference.experimental.runTools(...) // optional page-side agent loop helper

getFeatures() reports what stable request accepts, not whether experimental.request can relay tools. Bridge returns { toolCalling: false } until tools graduate; apps that want tools today should keep calling experimental.

Stable window.inference.request rejects tools, toolChoice, assistant toolCalls, and role: "tool" messages (invalid_request). Streaming still follows accepted → optional reasoning_delta / delta → done. When the model ends on tools, done.message may include toolCalls.

Security: function tools are defined and executed by the page. Bridge only relays JSON schemas, toolCalls, and role: "tool" results — it never runs app code or widens host permissions for tools. Approval still lists tool names so the user can see what the site is authorizing the model to request.

Defaults: if tools is present and toolChoice is omitted, Bridge treats it as "auto" (model may reply in text or call tools).

`experimental.request` parameters

Param	Required	Type	Notes
`method`	yes	`"chat"`	Only chat is supported.
`messages`	yes	`ExperimentalMessage[]`	Non-empty. Roles: `system` / `user` / `assistant` / `tool`.
`tools`	no	`Tool[]`	Non-empty when present. Function tools only for now.
`toolChoice`	no	`"auto"` \| `"none"` \| `"required"` \| `{ type: "function", function: { name } }`	Defaults to `"auto"` when `tools` is present.
`signal`	no	`AbortSignal`	Abort is handled in the page bridge (does not cross realms).

messages shapes

Role	Fields
`system` / `user`	`content: string`
`assistant`	`content: string \| null`; optional `reasoning?: string`; optional `toolCalls?: ToolCall[]`
`tool`	`toolCallId: string`; `content: string` (usually JSON text)

tools / ToolCall

Kind	Shape
Function tool	`{ type: "function", function: { name, description?, parameters? } }` — `parameters` is JSON Schema
`ToolCall` (on assistant / `done.message`)	`{ id, type: "function", function: { name, arguments } }` — `arguments` is a JSON string

Returns the same streaming contract as stable request: AsyncIterable of accepted → optional reasoning_delta / delta → done. On a tool turn, done.message.toolCalls may be set (often with empty/null content).

`experimental.runTools` parameters

Page-side agent loop. Calls experimental.request internally; Bridge still does not execute tools. For apps, prefer ipa-tools runTools (npm package, TypeScript types, works across IPA implementations); pass experimental.request until tools graduate onto stable request. Bridge’s experimental.runTools is convenient for console / paste-ready demos without a bundler.

Param	Required	Type	Notes
`messages`	yes	`ExperimentalMessage[]`	Conversation seed (mutated copy returned).
`tools`	no	`Tool[]`	Forwarded on each round.
`execute`	usually	`Record<string, (args) => unknown \| Promise<unknown>>`	Map of tool name → page handler. Required for any tool the model calls.
`toolChoice`	no	same as `request`	Forwarded each round when set.
`maxRounds`	no	`number`	Default `5`. Positive finite.
`onDelta`	no	`(content: string) => void`	Text deltas from each round.
`onReasoningDelta`	no	`(content: string) => void`	Reasoning deltas when present.
`onToolCall`	no	`({ id, name, arguments }) => void`	Fired once per tool call after args are parsed, before `execute` runs. Useful for UI chips / logging; putting UI inside `execute` is still fine.
`signal`	no	`AbortSignal`	Aborts between / during rounds.
`method`	no	`"chat"`	Defaults to `"chat"`.

Returns Promise<{ messages, final }> where final is the last done chunk (text reply after tools, or the first turn if no toolCalls) and messages includes assistant/tool turns appended by the loop.

Manual multi-turn (page executes)

async function getWeather({ city }) {
  // Page-owned — Bridge never runs this
  return { city, tempC: 22 };
}

const tools = [
  {
    type: "function",
    function: {
      name: "get_weather",
      description: "Get the current weather for a city",
      parameters: {
        type: "object",
        properties: { city: { type: "string" } },
        required: ["city"],
      },
    },
  },
];

const messages = [
  { role: "user", content: "What's the weather in Austin?" },
];

let done;
for await (const chunk of window.inference.experimental.request({
  method: "chat",
  messages,
  tools,
  // toolChoice defaults to "auto" when tools are present
})) {
  if (chunk.type === "done") done = chunk;
}

if (done.message.toolCalls?.length) {
  messages.push({
    role: "assistant",
    content: done.message.content ?? null,
    toolCalls: done.message.toolCalls,
  });

  for (const call of done.message.toolCalls) {
    const args = JSON.parse(call.function.arguments);
    const result =
      call.function.name === "get_weather"
        ? await getWeather(args)
        : { error: "unknown tool" };

    messages.push({
      role: "tool",
      toolCallId: call.id,
      content: JSON.stringify(result),
    });
  }

  for await (const chunk of window.inference.experimental.request({
    method: "chat",
    messages,
    tools,
  })) {
    if (chunk.type === "delta") {
      console.log("[delta]", chunk.content);
    }
    if (chunk.type === "done") {
      console.log("[done]", chunk.message.content);
    }
  }
} else {
  console.log(done.message.content);
}

`runTools` helper

window.inference.experimental.runTools runs the same page-side loop for you (still page-executed handlers). Useful in DevTools without installing a package; for shipped apps, use ipa-tools instead. Register multiple tools in tools and matching handlers in execute — the model may call one or more per turn:

const { final, messages } = await window.inference.experimental.runTools({
  messages: [
    {
      role: "user",
      content: "What's the weather in Austin, and what time is it there?",
    },
  ],
  tools: [
    {
      type: "function",
      function: {
        name: "get_weather",
        description: "Get the current weather for a city",
        parameters: {
          type: "object",
          properties: { city: { type: "string" } },
          required: ["city"],
        },
      },
    },
    {
      type: "function",
      function: {
        name: "get_time",
        description: "Get the current local time for a city",
        parameters: {
          type: "object",
          properties: { city: { type: "string" } },
          required: ["city"],
        },
      },
    },
  ],
  execute: {
    async get_weather({ city }) {
      return { city, tempC: 22 };
    },
    async get_time({ city }) {
      return { city, localTime: "3:45 PM" };
    },
  },
  onDelta(content) {
    console.log("[delta]", content);
  },
  onToolCall({ id, name, arguments: args }) {
    console.log("[tool]", name, id, args);
  },
});

console.log("[final]", final.message.content);
console.log("[messages]", messages);

With Zod

Define args with Zod, convert to JSON Schema for parameters, and parse in your page-side handler. Requires Zod 4+ (z.toJSONSchema):

import { z } from "zod";

const WeatherArgs = z.object({
  city: z.string().describe("City name"),
});

const { final } = await window.inference.experimental.runTools({
  messages: [{ role: "user", content: "What's the weather in Austin?" }],
  tools: [
    {
      type: "function",
      function: {
        name: "get_weather",
        description: "Get the current weather for a city",
        parameters: z.toJSONSchema(WeatherArgs),
      },
    },
  ],
  execute: {
    async get_weather(raw) {
      const { city } = WeatherArgs.parse(raw);
      return { city, tempC: 22 };
    },
  },
  onDelta(content) {
    console.log("[delta]", content);
  },
});

console.log("[final]", final.message.content);

Function-tool provider matrix

Provider	Function tools
OpenAI	Chat Completions `tools`
Anthropic	Messages API `tools`
OpenRouter	Chat Completions `tools`
Ollama	`/api/chat` `tools`
OpenAI-compatible	Chat Completions `tools`

Approval shows an Experimental banner and a Tools preview (function names). Always-allow origins still re-prompt when a request includes tools (or a wider tool set than the grant covers).

Development

npm install
npm test

Focused Node tests cover request validation (stable vs experimental), storage/grants, permission decisions (including tools re-prompt), provider registry, the page-side runTools loop, and function-tool streaming/accumulation for OpenAI / Anthropic / OpenRouter / Ollama / OpenAI-compatible (no full MV3 e2e).

Package a release ZIP (runtime files only):

npm run package

Manual checks

Current limitations

Built-in providers are OpenAI, Anthropic, OpenRouter, and local Ollama (Ollama fixed at http://localhost:11434); additional OpenAI-compatible servers are user-configured
Text chat on the stable IPA path; function tools are Bridge-experimental only (window.inference.experimental)
No file: / opaque-origin pages
No cost estimate in the approval UI
Cross-realm errors are reconstructed as Error objects with a code property

Contributing

Issues and pull requests are welcome.

Keep the core IPA surface aligned with SPEC.md
Prefer experimental, clearly labeled features over expanding the normative API prematurely
Add unit tests for non-UI logic where practical

See also Chrome Web Store release checklist and the privacy policy.

License

MIT. See LICENSE.

Navigation Menu

Repository files navigation

Inference Bridge

Features

Installation

Chrome Web Store (recommended)

Development (Load unpacked)

Supported Providers

Try it

OpenAI

Anthropic

OpenRouter

Ollama (no API key)

OpenAI-compatible servers

Security

Architecture

Standards Compatibility

Experimental Features

Function tools

`experimental.request` parameters

`experimental.runTools` parameters

Manual multi-turn (page executes)

`runTools` helper

With Zod

Function-tool provider matrix

Development

Manual checks

Current limitations

Contributing

License

About

Releases

Packages

Contributors

Languages

Name		Name	Last commit message	Last commit date
Latest commit History 50 Commits
.cursor		.cursor
.github/workflows		.github/workflows
background		background
content		content
docs		docs
icons		icons
scripts		scripts
src		src
test		test
ui		ui
.gitignore		.gitignore
LICENSE		LICENSE
PRIVACY.md		PRIVACY.md
README.md		README.md
manifest.json		manifest.json
package-lock.json		package-lock.json
package.json		package.json
vitest.config.js		vitest.config.js

Folders and files

Latest commit

History

Repository files navigation

Inference Bridge

Features

Installation

Chrome Web Store (recommended)

Development (Load unpacked)

Supported Providers

Try it

OpenAI

Anthropic

OpenRouter

Ollama (no API key)

OpenAI-compatible servers

Security

Architecture

Standards Compatibility

Experimental Features

Function tools

experimental.request parameters

experimental.runTools parameters

Manual multi-turn (page executes)

runTools helper

With Zod

Function-tool provider matrix

Development

Manual checks

Current limitations

Contributing

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

`experimental.request` parameters

`experimental.runTools` parameters

`runTools` helper