Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,26 @@ All notable changes to WebBrain are documented in this file.

This changelog was generated from the repository Git history and release tags. Versions without a Git tag are inferred from version-bump commits and the current `package.json` / browser manifest versions.

## [26.0.11] - 2026-08-05

### Changed
- Routed advice and drafting follow-ups through response-only handling when trusted conversation context is sufficient, even while Act mode is selected.
- Closed built-in tool schemas and made the runtime mode authoritative during execute tasks.
- Kept Turkish deasciification opt-in and instruction-only, with skill instructions loadable only from the enabled catalog after explicit conversion intent.
- Bounded Chrome and Firefox conversation, chat, and run-replay session snapshots so live work can continue safely when recovery persistence is unavailable.

### Fixed
- Rejected undeclared tool arguments and mixed click targets before dispatch with structured `invalid_tool_arguments` / `noDispatch` results.
- Suppressed planner-shaped JSON beside tool calls, retried planner-shaped terminals once, and replaced raw execute-protocol failures with a user-facing unverified-completion message.
- Prevented unknown required form values from being represented by empty focus, clear, or write actions.
- Normalized nested and object-shaped failures before UI, trace, and dedupe handling so `[object Object]` is never rendered.
- Made assistant-message Copy controls idempotent, added a localized **Copy message** label, and collapsed rejected `done` retries into one visible diagnostic row.
- Preserved acknowledged replay boundaries without false warnings while deduplicating genuine discarded-event gaps per request.
- Retried quota failures with compact snapshots, marked unrecoverable runs non-durable, warned once, and prevented consequential action replay after connection loss without deleting other tab/session data.

### Tests
- Added mirrored Chrome/Firefox coverage for closed schemas, disabled skill arguments, Act/planner enforcement, nested errors, Copy deduplication, replay boundaries, multi-tab quota exhaustion, attachment/screenshot compaction, and fail-closed reconnect durability.

## [26.0.0] - 2026-07-26

### Added
Expand Down
Binary file not shown.
Binary file not shown.
Binary file not shown.
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "webbrain",
"version": "26.0.10",
"version": "26.0.11",
"description": "Open-source AI browser agent — chat with pages, automate tasks, multi-provider LLM support.",
"private": true,
"type": "module",
Expand Down
2 changes: 1 addition & 1 deletion src/chrome/ARCHITECTURE.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# WebBrain Chrome/Edge Extension — Architecture

> Version 26.0.10 · Manifest V3 · Service Worker background
> Version 26.0.11 · Manifest V3 · Service Worker background

## High-Level Overview

Expand Down
2 changes: 1 addition & 1 deletion src/chrome/manifest.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"manifest_version": 3,
"name": "WebBrain",
"version": "26.0.10",
"version": "26.0.11",
"description": "Open-source AI browser agent — chat with pages, automate tasks, multi-provider LLM support.",
"permissions": [
"sidePanel",
Expand Down
211 changes: 174 additions & 37 deletions src/chrome/src/agent/agent.js

Large diffs are not rendered by default.

123 changes: 123 additions & 0 deletions src/chrome/src/agent/conversation-persistence.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
export const SESSION_CONVERSATION_BUDGET_BYTES = 1_500_000;
export const SESSION_CONVERSATION_RETRY_BUDGET_BYTES = 450_000;

const DATA_URL_RE = /data:(?:image|application)\/[a-zA-Z0-9+.-]+(?:;[^,\s]*)?;base64,[A-Za-z0-9+/=\s]+/g;

function byteLength(value) {
return new TextEncoder().encode(JSON.stringify(value)).byteLength;
}

function capText(value, maxChars, marker, state) {
const sanitized = String(value || '').replace(DATA_URL_RE, () => {
state.compacted = true;
return '[embedded binary data omitted from session recovery]';
});
if (sanitized.length <= maxChars) return sanitized;
state.compacted = true;
return `${sanitized.slice(0, Math.max(0, maxChars - marker.length - 1))}\n${marker}`;
}

function attachmentPlaceholder(message, kind) {
const handles = Array.isArray(message?.attachmentHandles) ? message.attachmentHandles : [];
if (handles.length) {
const ids = handles.map(handle => String(handle?.attachmentId || '')).filter(Boolean).slice(0, 8);
return `[User ${kind} attachment bytes omitted from session recovery; durable attachment handle(s): ${ids.join(', ') || 'available in chat history'}.]`;
}
return `[${kind === 'image' ? 'Screenshot/image' : 'Document'} bytes omitted from session recovery.]`;
}

function sanitizeValue(value, state, depth = 0) {
if (typeof value === 'string') return capText(value, 32_000, '[large value truncated for session recovery]', state);
if (!value || typeof value !== 'object' || depth > 6) return value;
if (Array.isArray(value)) return value.slice(0, 100).map(item => sanitizeValue(item, state, depth + 1));
const out = {};
for (const [key, child] of Object.entries(value).slice(0, 100)) {
if (typeof child === 'string' && (/^(?:data|url)$/i.test(key)) && /^data:.*;base64,/i.test(child)) {
state.compacted = true;
out[key] = '[embedded binary data omitted from session recovery]';
} else {
out[key] = sanitizeValue(child, state, depth + 1);
}
}
return out;
}

function sanitizeContent(message, state, caps) {
if (message?.transientCompletionVerification === true) {
state.compacted = true;
return '[Completion verification screenshot omitted from persisted history.]';
}
const content = message?.content;
if (typeof content === 'string') {
const cap = message.role === 'tool' ? caps.toolChars : caps.textChars;
return capText(content, cap, '[large content truncated for session recovery]', state);
}
if (!Array.isArray(content)) return sanitizeValue(content, state);
return content.slice(0, 100).map(block => {
if (block?.type === 'image_url' || block?.type === 'image') {
state.compacted = true;
return { type: 'text', text: attachmentPlaceholder(message, 'image') };
}
if (block?.type === 'document' || block?.source?.type === 'base64') {
state.compacted = true;
return { type: 'text', text: attachmentPlaceholder(message, 'document') };
}
return sanitizeValue(block, state);
});
}

function sanitizeMessage(message, state, caps) {
if (!message || typeof message !== 'object') return message;
const out = { ...message, content: sanitizeContent(message, state, caps) };
if (Array.isArray(message.tool_calls)) {
out.tool_calls = message.tool_calls.slice(0, 50).map(call => ({
...call,
function: call?.function ? {
...call.function,
arguments: capText(call.function.arguments || '', caps.toolArgsChars, '[tool arguments truncated for session recovery]', state),
} : call?.function,
}));
}
if (Array.isArray(message.responseItems)) out.responseItems = sanitizeValue(message.responseItems, state);
return out;
}

function reduceToBudget(messages, maxBytes, state) {
if (byteLength(messages) <= maxBytes) return messages;
const out = messages.map(message => ({ ...message }));
const keepRecentFrom = Math.max(1, out.length - 14);
for (let index = 1; index < keepRecentFrom && byteLength(out) > maxBytes; index++) {
const message = out[index];
if (!message || message.role === 'system') continue;
state.compacted = true;
out[index] = {
role: message.role,
...(message.tool_call_id ? { tool_call_id: message.tool_call_id } : {}),
content: '[Earlier message omitted from bounded session recovery snapshot.]',
};
}
for (let index = keepRecentFrom; index < out.length && byteLength(out) > maxBytes; index++) {
const message = out[index];
if (!message || typeof message.content !== 'string' || message.content.length <= 4_000) continue;
state.compacted = true;
out[index] = { ...message, content: `${message.content.slice(0, 3_900)}\n[content truncated for session recovery]` };
}
return out;
}

export function serializeConversationForSession(messages, options = {}) {
const maxBytes = Number.isFinite(options.maxBytes) ? Math.max(100_000, options.maxBytes) : SESSION_CONVERSATION_BUDGET_BYTES;
const tight = maxBytes <= SESSION_CONVERSATION_RETRY_BUDGET_BYTES;
const caps = tight
? { textChars: 16_000, toolChars: 8_000, toolArgsChars: 8_000 }
: { textChars: 96_000, toolChars: 32_000, toolArgsChars: 24_000 };
const state = { compacted: false };
const sanitized = Array.isArray(messages) ? messages.map(message => sanitizeMessage(message, state, caps)) : [];
const bounded = reduceToBudget(sanitized, maxBytes, state);
return { messages: bounded, bytes: byteLength(bounded), compacted: state.compacted };
}

export function isSessionQuotaError(error) {
const message = String(error?.message || error || '');
return /quota|QUOTA_BYTES|bytes? exceeded|storage limit/i.test(message);
}
3 changes: 3 additions & 0 deletions src/chrome/src/agent/planner.js
Original file line number Diff line number Diff line change
Expand Up @@ -114,10 +114,13 @@ Rules:
- Classify the user's semantic intent across any language; never rely on literal keywords or UI labels.
- execute means the user authorizes action. A request to plan and then perform is execute.
- respond means the user asks only for a natural-language answer or recoverable artifact from existing conversation/working-note context, with no fresh page read or browser action.
- Runtime mode does not force execute. In Act mode, an advice, explanation, or drafting follow-up is still respond when trusted conversation context already contains everything needed.
- Require execute only when the answer genuinely needs fresh page, browser, or network evidence. Do not reread a page merely because Act mode is selected.
- plan_only means the user asks for a plan, outline, strategy, or discussion without authorizing action.
- clarify means missing or conflicting user information prevents a useful plan; localized.summary must be the concise question to ask.
- A request to answer, summarize, explain, analyze, or draft a response about currently visible/open page content is execute when producing the answer needs a fresh page or browser read, even if the final deliverable is only text and requires_state_change is false. Example: "How should I respond to this open email?" is execute because the email must be read now.
- respond must not include steps that need page, browser, network, memory, or scheduling tools. If any such tool is needed to produce the requested answer, classify the request as execute instead.
- When a required form value is unavailable from trusted or public evidence, leave the field untouched and classify as clarify. Never plan to focus, clear, or write an empty value as a stand-in for missing personal information.
- requires_state_change is true only when an execute request needs a mutation such as interacting with form/account state, modifying page data, downloading/uploading a file, a write-method network request, a Dev patch, or scheduling work. It is false for reads, analysis, summaries, navigation, scrolling, hovering, window/viewport changes, plan_only, and clarify.
- requires_submission is true only when an execute request must explicitly commit a form/dialog with an action such as Submit, Save, Send, Publish, Post, or Confirm. It is false for filling, editing, checking, or selecting without committing, including explicit do-not-submit tasks and autosave UIs, and false for non-execute requests.
- allows_planner_shaped_result is true only when the user explicitly requests planner-like final data (summary/steps JSON or Plan/Steps/Workflow markdown). Never changes request_kind.
Expand Down
126 changes: 126 additions & 0 deletions src/chrome/src/agent/tool-arguments.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
function isPlainObject(value) {
return !!value && typeof value === 'object' && !Array.isArray(value);
}

function valueMatchesType(value, type) {
if (type === 'object') return isPlainObject(value);
if (type === 'array') return Array.isArray(value);
if (type === 'integer') return Number.isInteger(value);
if (type === 'number') return typeof value === 'number' && Number.isFinite(value);
if (type === 'null') return value === null;
return typeof value === type;
}

function validationFailure(toolName, invalidArguments, detail) {
const fields = [...new Set(invalidArguments.map(String))];
return {
ok: false,
result: {
success: false,
invalidArguments: true,
invalidToolArguments: true,
noDispatch: true,
dispatched: false,
errorCode: 'invalid_tool_arguments',
invalidArgumentNames: fields,
error: `${toolName || 'Tool'} could not run because its arguments do not match the advertised schema. Re-emit the call with only declared, valid arguments; do not assume the action happened.`,
detail,
},
};
}

function validateValue(value, schema, path, failures) {
if (!schema || typeof schema !== 'object') return;
const acceptedTypes = Array.isArray(schema.type) ? schema.type : (schema.type ? [schema.type] : []);
if (acceptedTypes.length && !acceptedTypes.some(type => valueMatchesType(value, type))) {
failures.push(path);
return;
}
if (Array.isArray(schema.enum) && !schema.enum.some(candidate => Object.is(candidate, value))) {
failures.push(path);
return;
}
if (typeof value === 'string') {
if (Number.isFinite(schema.minLength) && value.length < schema.minLength) failures.push(path);
if (Number.isFinite(schema.maxLength) && value.length > schema.maxLength) failures.push(path);
}
if (Array.isArray(value) && schema.items) {
value.forEach((item, index) => validateValue(item, schema.items, `${path}[${index}]`, failures));
}
if (!isPlainObject(value)) return;
const properties = isPlainObject(schema.properties) ? schema.properties : {};
for (const required of Array.isArray(schema.required) ? schema.required : []) {
if (!Object.prototype.hasOwnProperty.call(value, required)) failures.push(`${path}.${required}`);
}
if (schema.additionalProperties !== true && typeof schema.additionalProperties !== 'object') {
for (const key of Object.keys(value)) {
if (!Object.prototype.hasOwnProperty.call(properties, key)) failures.push(`${path}.${key}`);
}
}
for (const [key, child] of Object.entries(value)) {
if (Object.prototype.hasOwnProperty.call(properties, key)) {
validateValue(child, properties[key], `${path}.${key}`, failures);
}
}
}

function validateClickTarget(args) {
const text = typeof args.text === 'string' && args.text.trim() !== '';
const selector = typeof args.selector === 'string' && args.selector.trim() !== '';
const index = Number.isInteger(args.index) && args.index >= 0;
const hasX = typeof args.x === 'number' && Number.isFinite(args.x);
const hasY = typeof args.y === 'number' && Number.isFinite(args.y);
const coordinates = hasX && hasY && !(args.x === 0 && args.y === 0);
const strategies = [text, selector, index, coordinates].filter(Boolean).length;
const invalidCoordinates = hasX !== hasY || ((hasX && hasY) && args.x === 0 && args.y === 0);
if (strategies !== 1 || invalidCoordinates || (args.from_screenshot === true && !coordinates)) {
return validationFailure('click', ['target'], 'Provide exactly one target strategy: non-empty text, non-empty selector, a non-negative integer index, or a complete non-zero x/y coordinate pair.');
}
return null;
}

export function closeToolDefinition(tool) {
if (!tool?.function) return tool;
const parameters = tool.function.parameters;
if (!isPlainObject(parameters)) return tool;
const closeSchema = (schema) => {
if (!isPlainObject(schema)) return schema;
const closed = { ...schema };
if (isPlainObject(schema.properties)) {
closed.properties = Object.fromEntries(Object.entries(schema.properties).map(([key, child]) => [key, closeSchema(child)]));
}
if (schema.items) closed.items = closeSchema(schema.items);
if (schema.type === 'object' && schema.additionalProperties === undefined) closed.additionalProperties = false;
return closed;
};
return {
...tool,
function: {
...tool.function,
parameters: closeSchema(parameters),
},
};
}

export function closeToolDefinitions(tools) {
return Array.isArray(tools) ? tools.map(closeToolDefinition) : [];
}

export function validateToolArguments(toolName, args, parameters) {
if (!isPlainObject(args)) {
return validationFailure(toolName, ['$'], 'Arguments must be a JSON object.');
}
const closedParameters = isPlainObject(parameters)
? { ...parameters, additionalProperties: false }
: { type: 'object', properties: {}, additionalProperties: false };
const failures = [];
validateValue(args, closedParameters, '$', failures);
if (failures.length) {
return validationFailure(toolName, failures, `Invalid or undeclared argument(s): ${[...new Set(failures)].join(', ')}.`);
}
if (toolName === 'click') {
const clickFailure = validateClickTarget(args);
if (clickFailure) return clickFailure;
}
return { ok: true, args };
}
10 changes: 7 additions & 3 deletions src/chrome/src/agent/tools.js
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import { closeToolDefinitions } from './tool-arguments.js';

/**
* Tool definitions for the WebBrain agent.
* These are sent to the LLM in OpenAI function-calling format.
Expand Down Expand Up @@ -1316,15 +1318,15 @@ export function getToolsForMode(mode, opts = {}) {
base = [...base, ...extras];
}
const useDoneJson = normalizedMode === 'act' && tier === 'full' && opts.cloudRun === true && !!opts.outputSchema;
if (useDoneJson) return base.map(tool => (tool.function.name === 'done' ? DONE_JSON_TOOL : tool));
if (useDoneJson) return closeToolDefinitions(base.map(tool => (tool.function.name === 'done' ? DONE_JSON_TOOL : tool)));
const useOutcomeDone = normalizedMode !== 'ask';
if (!opts.strictSecretMode && !useOutcomeDone) return base;
if (!opts.strictSecretMode && !useOutcomeDone) return closeToolDefinitions(base);
const replacement = opts.strictSecretMode
? (useOutcomeDone
? (tier === 'compact' ? DONE_TOOL_COMPACT_STRICT_WITH_OUTCOME : DONE_TOOL_STRICT_WITH_OUTCOME)
: DONE_TOOL_STRICT)
: (tier === 'compact' ? DONE_TOOL_COMPACT_WITH_OUTCOME : DONE_TOOL_WITH_OUTCOME);
return base.map(t => (t.function.name === 'done' ? replacement : t));
return closeToolDefinitions(base.map(t => (t.function.name === 'done' ? replacement : t)));
}

const SENSITIVE_PAGE_DATA_GUIDANCE = `SENSITIVE PAGE DATA:
Expand All @@ -1333,7 +1335,9 @@ const SENSITIVE_PAGE_DATA_GUIDANCE = `SENSITIVE PAGE DATA:

const PLAN_TO_EXECUTION_GUIDANCE = `PLAN TO EXECUTION:
- In Act/Dev, an approved or pinned plan is context for doing the task, not a completed user outcome. When the user authorized action, do not end by returning the plan, planner JSON, action-policy metadata, or a promise to act; call the first permitted tool and continue until done, an explicit blocker, cancellation, or required user input.
- The trusted runtime mode is authoritative. Never claim that the run is in Ask mode or tell the user to switch to Act when the runtime prompt says Act/Dev.
- Do not call done with the plan, planner JSON, action-policy metadata, or a promise to act as its summary. Call a permitted non-done tool first; use clarify or stop only for a real blocker or required user input.
- If a required form value is unavailable, leave that field untouched and call clarify. Never focus, clear, or write an empty value merely because the value is unknown.
- Respect user boundaries: if the user asked only for a plan, or said to wait for approval or confirmation, return the plan or wait and do not execute.
- Structured output can be legitimate user-requested data. Honor requested JSON or markdown formats; never treat an answer as leaked planner metadata merely because it looks like a plan or policy.`;

Expand Down
Loading
Loading