stage.md

Browser panel

← Docs · 中文

When the model uses a browser-type MCP (such as playwright) to go online or test the web pages it builds, that browser's view is docked in a panel on Yan's right side (游目, the browser panel) instead of being a separate window on screen getting in the way. You can click, scroll and type in the panel (input methods work as usual), and tabs stay in sync with the browser: when the model opens or switches to another page, the panel follows; click another tab to look at that page.

Setup

The browser isn't set up in MCP but in Settings → Browser: turn on the browser panel, choose which browser to use (Edge, Chrome, or Bundled Chromium if neither is installed), and click Install on the Driver row. Yan starts a Playwright MCP server of its own, named 游目, which isn't listed in the MCP tab but can still be chosen in presets. Everything it owns lives in the 游目/ folder under the storage root:

• 依赖/ (dependencies): the driver @playwright/mcp is installed here (via npm; the download source follows the Environment tab, using mirrors in mainland China by default). Update reinstalls the latest.
• 浏览器/ (browser): sign-ins and bookmarks (--user-data-dir).
• 下载/ (downloads): files downloaded from web pages land here (--output-dir).
• 内核/ (engine): with Bundled Chromium, Playwright's browser is installed here, and the Browser row gets an extra button to install the engine. The download source is chosen automatically (server/mirror.js): before installing, it fetches 2 MB of the engine itself from both the mirror and the official source to see which is faster — mirrors often have to fetch newly released engines from the origin on the spot at tens of KB/s, looking stuck; if 90 seconds pass with no progress, it ends that download and starts over from the other source. The button shows how far it's got. The driver follows the comparison made on a common package. If you reload or click again while it's installing, you follow the same run rather than starting another. If you pick an Edge / Chrome that isn't installed, the same row offers a link to the official site (installing a system browser needs administrator rights; Yan doesn't do it for you).
• playwright.json: the connection settings, see below.

These folders can't be relocated; missing ones are created. Open folder on the Storage row takes you there. Local files can always be opened (--allow-unrestricted-file-access): the agent can open the pages it builds via file://, change them and look again, without anyone's help.

The panel needs three extras from that browser: debugging port 9288, allowing Yan's page as an origin (and the debugging port itself — the front end that Developer tools opens is served by it and must connect back), and the window moved off screen. You don't write these: Yan adds them when starting Playwright's MCP server (prepareBrowser in server/stage.js; the panel's own server is first assembled whole by builtinConfig). If the arguments carry a --config, it merges into that file, keeping what you wrote (port, window position, other settings); the merged result is written to 游目/playwright.json, and --config in the arguments is replaced with it when the server starts. Connecting to a browser started elsewhere (--cdp-endpoint) adds nothing, and headless mode doesn't move the window. A Playwright server connected separately in MCP gets the same additions; other servers start as is.

• Yan reads the debugging port itself: --cdp-endpoint in the server arguments, or --remote-debugging-port in launchOptions.args of the --config file; if neither is given, 9288. The profile folder holding bookmarks is read from --user-data-dir in the arguments (or userDataDir in the file). To change the port, write it in your own --config file, and Yan adds the rest around it.
• The debugging port is 9288 rather than the usual 9222 / 9223: those two are often taken by port forwarding for remote debugging (netsh interface portproxy), and if IPv4 is taken, the browser can only fall back to listening on IPv6, where the panel can't find it.
• The window is off screen and doesn't take a taskbar button either: on Windows the bridge runs a small invisible process that finds this browser by its debugging port and removes its windows from the taskbar (server/taskbar.js). When it opens a new window, a button may flash for up to a second before it's removed.
• The server's Note to the model says "when I say open the browser, I mean this one" (already written for one-click setup), so the model won't open the system's default browser.
• This browser can't open Yan itself: Yan's page can change settings, so a model that opened it could turn off review or change permissions. Yan adds network.blockedOrigins to the config to block the bridge's address, and the port follows YAN_PORT. The same applies to a Playwright server you connect yourself. This stops opening and navigating to the page, but it isn't a security boundary: code run in the page (browser_run_code_unsafe) can still send requests directly, the same class as node -e, which static screening can't reach.
• The window is moved off screen rather than running headless: headless Edge includes HeadlessChrome in its UA, which some sites detect; an off-screen window is like any other, and still renders.
• --remote-allow-origins only allows Yan's page and the debugging port itself (Developer tools in the menu opens the browser's own front end, served by the debugging port, which must connect back); other websites can't connect to this debugging port. If you change the bridge port (YAN_PORT), the additions follow.
• Web pages for projects that need building still start a dev server as usual, and the model opens localhost in the browser.

Usage

• With a browser-type MCP connected (one whose tools include browser_navigate), a small hanging scroll drops from the top of the page at the right of the text. When the pointer comes near, the word 游目 appears on a slip; click it to open the panel. Close the panel and the scroll comes back.
• If the browser isn't open or has been closed, click Open browser in the panel, and Yan calls that server's browser_navigate once for you; the browser starts with its configuration (sign-ins and bookmarks intact).
• Tabs are a row of text separated by thin rules; the current one has a vermilion stroke underneath, and the one the agent is on has a vermilion dot after its title; × appears only on hover. ＋ at the end opens a new page.
• Under an ink line is the address: normally just one line, "domain › page title" (local pages show localhost:port and the path, local files the file name); hover for the full URL, click it (or Ctrl+L) to turn it into an editable, copyable input. The bookmark ribbon at the right end is for bookmarks: vermilion when this page is bookmarked. Clicking it opens a bookmarks tag: the first line is this page's title with a ribbon at the right — ink-colored, click it to bookmark; already bookmarked, it's vermilion, with two small icons beside it for edit and delete. Below, each folder is a thin folded corner, a small case, the folder name in darker ink and its count; click the header to open or close it (which ones are closed is remembered locally). Items in the same folder are gathered by a faint vertical ink trace, each led by a faint ribbon; click one to open it in the current page. Hovering an item shows two small icons: the brush is edit (expands name and address fields in place; Enter saves, Esc discards), and the struck-through text is delete (deleting a non-empty folder first asks "including n items, click again to delete"). At the bottom, New folder and Organize…, which opens the browser's own bookmarks page, where dragging to sort and bulk actions live.
• When the model switches pages (selecting a tab, opening a new one), the panel follows; if you clicked another tab yourself and the model carries on in the original page, the panel doesn't drag you back.
• Dialogs from web pages (alert, confirm, prompt, leave-page prompts) are drawn by the browser on the off-screen window, and the page stops there: the panel draws its own over the view, Enter to confirm, Esc to cancel. The model receives it too; whoever answers first wins.
• When a page asks you to choose a file (after clicking "Upload" and the like), the browser's file picker is off screen too, so the panel opens the system's file dialog through the bridge, and the chosen paths go straight into the page (nothing is copied or read by Yan's page). When the agent triggered it, no dialog pops up in your face: the panel floats a paper tag instead, and Choose file opens the dialog.
• Dropdowns (<select>) in a page: the browser draws their options in a separate little window, also off screen. The panel floats a paper tag right under the dropdown listing the options (grouped as on the page, the current one in vermilion, a filter box on top when there are more than twelve); click one, or use the arrow keys and Enter, and Esc to cancel. The page gets its usual change event.
• Right-click: the browser's menu is off screen too, so the panel floats a paper tag at the pointer with a few simple items — on a link, Open link in new tab (in the background) and Copy link; on an image, Open image in new tab; on selected text, Copy and Quote to agent (a quote carrying the address, the same path as circling); in an input, Cut, Copy, Paste (sent as Ctrl+X / C / V); elsewhere, Back, Forward, Reload. Pages with their own right-click menu keep theirs.
• Middle-click or Ctrl-click a link and the new tab opens in the background while the panel stays put; when the pointer leaves the picture, the page knows it left (hover menus close).
• While loading, a vermilion line runs under the address bar; back and forward fade when there's nowhere to go. The pointer follows the page: a hand on links, a text cursor in inputs.
• The agent's actions don't take up a line: where it clicks on the page, a vermilion brush circle appears labeled "click"; while it types, a vermilion line appears under that field labeled "type", fading after a second or two. At the right end of the address row hangs 执事 with a vermilion dot (blinking while active, hidden after eight idle seconds); hover to see the latest few steps ("go to github.com", "click "Sign in"", "type "…"").
• Marking up: the pen at the top of the panel (or holding Alt) starts drawing — drag a loop to circle, drag an open stroke for a line (underline a word, draw an arrow — the text just above and below it is read, and the model gets its start and end), click to dot, drawn in vermilion over the view, as many places as you like. Each mark drops a quote into the input box (which page, which words were circled), and the view with the marks is saved as an image attachment. Sending carries it to the agent like any other quote; removing the quote clears the marks from the view. Click the pen again or press Esc to stop drawing.
• The browser window follows the panel's size, showing the page at its natural size; when the panel is smaller than 960×600 the window stops shrinking (so sites don't switch to their mobile version) and the view is scaled down proportionally.
• Fit page and which search provider to use are in the menu (adjustable while the panel is open) and saved to settings; the last line of the menu, Browser panel settings…, configures the panel's own browser.
• Fit page (a switch in the menu, on by default): when the agent tests a page it often pins the viewport (setViewportSize, browser_resize), after which the page no longer follows the window and the view has blank margins. The margin shows a line saying the agent pinned the viewport at 1360×900. With Fit page on and the agent having actually pinned it (one of the two calls above), once the agent is idle (no browser calls in flight, no chats answering), it's released — the same URL is opened in a new page and the old one closed, filling the panel. With it off, or when the viewport only measures as pinned without the agent having pinned it, that line carries a Release button for you to click (a new page wipes scripts running in the page and half-filled forms, so it isn't done on a guess).
• The tuning icon at the top of the panel opens a menu: Find (Ctrl+F), Zoom, History, downloads, organizing bookmarks, opening a local file (Ctrl+O; pick it in the system's file dialog and it opens as file://, so the stylesheets, scripts and images next to it still load; typing a path into the address bar works too. Files the browser can't show (Office documents, archives, csv and so on), which it would treat as downloads, are cancelled on the spot and the original opens in its app instead of being copied into downloads; executables aren't opened), copying the URL, opening in the system browser (handed to the system's default browser through the bridge), Developer tools, browser settings, browser panel settings. History, downloads, bookmarks and settings all open the browser's own pages (in a new tab); cookies, passwords, site permissions and clearing browsing data are all in its settings page, and Yan doesn't build its own.
• Find: the address row turns into a find bar; matches on the page turn pale yellow, the current one vermilion and scrolled into view, with "2 / 7"; Enter for next, Shift+Enter for previous, Esc to close.
• Zoom: "− 100% ＋" in the menu, or Ctrl + / − / 0 while the keyboard is in the page. Zooming in one step narrows the window one step, so the page reflows at its own breakpoints and the text appears one step larger in the panel; the view may look slightly soft when zoomed in.
• Downloads: once a download starts in a page (whether you or the agent clicked), the menu's downloads entry shows the new count. The downloads tag lists name, size, origin and time; click one to show it in File Explorer (shown, not opened); "All downloads…" opens the browser's own downloads page.
• While the panel is closed, the scroll shows the browser's state: rolled up means not open; unrolled means open; swaying, with a glow around its red sun, means the model is using it — from its first browser_* call until that reply ends, so it doesn't go out between steps, and helpers still working count too.
• Drag the panel's left edge to resize it; the width is remembered locally. The icon at the top that stretches toward the corners widens the panel to fill the page; Esc returns. While the keyboard belongs to the page, each corner of the view shows a viewfinder stroke. Side threads also use the right side, and the panel gives way while a side thread is open.
• Clicking the view hands the keyboard to that page: key presses (including Ctrl+C / V) and text from input methods go in; clicking elsewhere in Yan takes it back. The panel handles a few keys the browser would normally take: Ctrl+L / Alt+D / F6 for the address bar, Ctrl+O to open a local file, F5 / Ctrl+R to reload, Alt+← / → and the mouse side buttons for back and forward. Ctrl+T and Ctrl+W can't be intercepted from outside the browser, so they aren't handled.
• The address bar takes URLs, localhost:port and local paths (E:\…\index.html); anything that doesn't look like a URL (spaces, no dot) is treated as a search and sent to the provider chosen in the menu (Bing, Baidu, Google; click to switch).

How it works

• The panel is another viewer of that browser: the page connects straight to the debugging port, receives frames with Page.startScreencast, sends input with Input.*, and copies tabs from Target.setDiscoverTargets. Frames don't go through the bridge; the bridge only looks up the connection address for the page (server/stage.js, since the debugging port's /json doesn't allow cross-origin reads).
• Page size: playwright sets no viewport when headed, so the page is as big as the window. When the panel changes size (250 ms after you stop), Browser.setWindowBounds resizes the off-screen window to the panel's size, adding the tab bar and address bar by their current height. For browsers with a fixed viewport, resizing the window does nothing, and the view is still scaled by the tighter of width and height; clicks are converted back by the same factor. With enough room the picture is one to one: the window is sized to the panel rounded down, the viewport is measured by innerWidth (scrollbar included, the same width as the frames), and the picture is nudged onto whole device pixels — a pixel or two off and the whole frame gets resampled and text goes soft. While the page moves it streams JPEG (on photo-heavy pages a PNG frame is nearly 2 MB and only a dozen or so arrive per second); once it has been still for 400 ms a PNG screenshot replaces it. The screenshot triggers one more frame of the same picture, and the same picture encodes to byte-identical JPEG, so a frame identical to the last is skipped.
• When to connect: on page load, when the panel opens, and after each browser_* call (that's usually when the browser starts). When the browser closes the panel clears; there's no polling. Yan itself never starts a browser; Open browser also starts it through that MCP server.
• Following pages: after each browser_* call, Yan asks which page is in front (the debugging port's /json/list is ordered by recent activity, and the model selecting a tab brings that page to the front), and follows only if it differs from last time. When you click a tab, the panel also brings that page to the front and notes it, so it isn't taken as the model switching.
• The few things the browser draws itself (dialogs, loading, whether back is possible) are off screen: the panel turns on Page.enable for the page being viewed and draws them itself from events such as javascriptDialogOpening, frameStartedLoading and frameNavigated; back and forward use Page.navigateToHistoryEntry, not scripts in the page. File choosing turns on Page.setInterceptFileChooserDialog; on fileChooserOpened, if a person just pressed something in the panel (within 1.5 seconds), the bridge opens the system's file dialog (POST /api/work/pick with files: true, the same dialog as picking the working folder), otherwise the paper tag floats; the real paths chosen are handed to the page with DOM.setFileInputFiles. A dropdown's little window isn't part of the page picture either: the page-side listener (see "the agent's moves" below) also catches a person opening one (pressing it, or F4, Alt+↓, Space), reports the options and where the box is, and the panel floats the paper tag; the choice goes back through the same listener, which sets selectedIndex and fires input and change. Dropdowns the page handles itself (default prevented) or inside cross-origin frames are left to the browser; ones the agent clicks don't pop up (it uses select_option).
• Pointer: the page-side listener watches pointer moves and reports only when the cursor of what's under it changes (a hand on links, a text cursor in inputs); the panel applies it to the picture instead of polling.
• The model's actions: arguments of browser_* calls are written, following the playwright MCP's conventions, as a short phrase with a verb (stageActionText), and the latest few are kept. Where it acts on the page comes from a listener in the page only the panel hears (src/26-stage/50-ink.js): in an isolated script world it listens for real presses and input and reports the element's position back through Runtime.addBinding; if the agent is calling the browser and it wasn't a person pressing on the view just now, a stroke is drawn.
• Marking up takes the same path as selecting text to Quote, with no separate annotation system. Marks are recorded by position in the page (viewport coordinates plus the scroll reported in screencast frames), so the vermilion strokes follow when the page scrolls; when drawn, the page is asked for the visible text in that area (for a dot, the text of the element at that point). The quote comes in two parts: one line for the interface, and a version for the model that also carries the URL and the positions of each mark (CSS pixels, framed the same as the attached image). The attached image is the current view with the strokes on top, saved as an ordinary attachment; marking another place replaces both the quote and the image. Clicking the quote above that question returns to that page in the panel.
• Frames are received only when visible: when the panel is closed, pushed aside by a side thread, or Yan's page is in the background, screencast stops, and resumes on return.
• Bookmarks: the panel only relays web pages, so the browser's own bookmarks bar is outside the view. Bookmarks are stored in Default/Bookmarks in the browser's profile folder, taken from --user-data-dir in that server's arguments and read by the bridge (POST /api/stage/bookmarks, which, like finding the debugging port, only passes that server's arguments), fresh each time. Editing bookmarks borrows the browser's own bookmarks page: chrome.bookmarks is available there, and additions, deletions and edits take effect in the browser immediately, without touching the profile files. That page is opened in a separate browser context (Target.createBrowserContext) — the agent (Playwright) doesn't know it and the panel doesn't list it; bookmarks are shared across the profile, so they still go into the main bookmarks, and it closes when done. For ten seconds after an edit, the tree returned by the browser is trusted (the file is written a second or two later).
• Fit page: Playwright remembers a pinned viewport and pins it again on navigation, and a separate connection from the panel can't clear it (tested) — so releasing goes through that MCP server's browser_tabs: first list to find which of the agent's pages this is, then new with the same URL and close the old one, so the agent's tab bookkeeping stays right. When the agent calls browser_resize, or setViewportSize inside browser_run_code_unsafe, a line is added after the tool result (stagePinned in prompts/mcp.js) telling it to open a new page after testing rather than setting a desktop size again to "restore".
• Zoom: the browser's own zoom can't be set through the debugging port (calling chrome.tabs.setZoom from the bookmarks page hangs), so it's done by resizing the window: the window width is narrowed by the factor, and the view's maximum width is widened by the same factor.
• Find: a small script runs in the page, marking matches with CSS highlights (::highlight, styles attached through adoptedStyleSheets) without changing the page's DOM; it only finds matches within the same run of text.
• Downloads: with the Page domain on for the page being viewed, downloadWillBegin / downloadProgress report names and progress; for downloads made by the agent through MCP, "Downloaded file … to …" in the result says where it was saved, which is filled in. Records are kept locally (the latest thirty). Showing a file goes through the bridge (POST /api/stage/reveal, explorer /select); with no recorded path, it opens the folder where that server saves downloads (--output-dir, or .playwright-mcp under the server's working directory). The bubble Edge pops up when downloading (edge://downloads-hub) is also reported as a page; the panel doesn't list it.
• Browsers don't announce title changes: while the panel is open, it asks for the tab list every second and a half.
• Code: src/26-stage/, styles/56-stage/; the test test/stage.mjs starts a separate headless browser instance as the model's browser (a single window), while Yan stays in the test browser.