# Prior art: operator surfaces

**Status:** evidence capture. Not normative, not a discovery-study output.
**Captured:** 2026-08-03 from public documentation and repositories.
**Purpose:** ground the [`operator`](../../../profiles/operator/PROFILE.md) contract.

Outside the `smithery-use-300-2026-08-02` cohort and its frozen manifest. Excerpts are brief, necessary, and attributed per [`DATA-NOTICE`](../../discovery/DATA-NOTICE.md) §"Rights and redistribution". No service was invoked; surfaces are read from documentation.

## Sources

| ID | Surface | Kind |
|---|---|---|
| `P1` | Playwright MCP (`microsoft/playwright-mcp`) | MCP server, browser |
| `P2` | Chrome DevTools MCP (`ChromeDevTools/chrome-devtools-mcp`) | MCP server, browser |
| `P3` | Ghostlight (`sylin-org/ghostlight`) | MCP server, browser — live tool surface observed in-session. **Not a primary vocabulary**: it adopts `P8`'s names. Retained as the worked example of adopting a vocabulary that has no levels. |
| `P4` | Stagehand (`browserbase/stagehand`) | SDK, browser |
| `P5` | Anthropic computer use tool (`computer_20251124`) | Model tool, desktop |
| `P6` | W3C WebDriver BiDi | Standard, browser |
| `P7` | Desktop MCP servers — AutoGUI, computer-use-linux, Windows 365 Agents, Nuphus | MCP servers, desktop |
| `P8` | Claude in Chrome, as documented at `code.claude.com/docs/en/chrome` | Browser extension, vendor-native |

---

## The convergent core

Every browser implementation exposes the same five things under different names.

| Job | `P1` Playwright MCP | `P2` Chrome DevTools MCP | `P8` Claude in Chrome |
|---|---|---|---|
| **Address a surface** | `browser_tabs` (multiplexed) | `list_pages`, `select_page`, `new_page`, `close_page` | `tabs_context_mcp` |
| **Navigate** | `browser_navigate`, `browser_navigate_back` | `navigate_page` | `navigate` |
| **Perceive (structured)** | `browser_snapshot`, `browser_find` | `take_snapshot` | `read_page`, `get_page_text`, `find` |
| **Perceive (visual)** | `browser_take_screenshot` | `take_screenshot` | screenshot via `computer` |
| **Act** | `browser_click`, `browser_type`, `browser_hover`, `browser_drag`, `browser_press_key`, `browser_select_option` | `click`, `type_text`, `hover`, `drag`, `press_key`, `fill` | `computer`, `form_input` |
| **Settle** | `browser_wait_for` | `wait_for` | not publicly documented |

Three independent teams, five identical jobs. That is the strongest available evidence that `operator` names something real.

**A correction to an earlier draft of this capture.** The third column previously named `P3` Ghostlight. That was a misattribution: Ghostlight adopts Claude in Chrome's tool names deliberately, so the vocabulary is Anthropic's rather than Ghostlight's, and crediting it to the copy overstated how many independent vocabularies exist. `P8` is now the column, and `P3` appears below only where it is the originator of something — its capability grants, its receipts, and its stated scope boundary.

`P8`'s documentation also separates its tools by whether they change state: `read_page`, `get_page_text`, `find`, console and network reads, and screenshots are read-only, while clicks, typing, navigation, tab management and GIF recording are state-changing. That is an effect declaration in prose, made for a permission prompt rather than for a caller — the distinction exists and is simply not on the wire.

## What adoption without levels looks like

`P3` adopts **nine of `P8`'s tool names verbatim**: `read_page`, `get_page_text`, `find`, `navigate`, `computer`, `form_input`, `javascript_tool`, `tabs_context_mcp`, `browser_batch`. Its maintainer states the reason plainly — a model meeting it should find something it already recognises.

It then exposes sixteen more. Setting aside those that match capabilities `P8` describes in prose without naming a tool, roughly eight are genuinely additional, and they separate into two kinds:

| Addition | Kind |
|---|---|
| `wait_for` | Settling. `P8`'s documentation names no equivalent, and `P1` and `P2` both have one. Arguably belongs in any operator floor. |
| `dialog` | Resolving a blocking modal, reported as state rather than as failure. |
| `act_on` | An action returning a receipt of what happened, rather than only that it succeeded. |
| `form_fill`, `script` | Convenience composition over the floor. |
| `narrate`, `update_plan` | For a human watching a long task. |
| `explain` | Introspecting what the current policy permits. |

**This is a mandatory floor, a set of missing-but-essential jobs, and a vendor's own additions — arrived at by an implementer who had no mechanism to declare which was which.**

Two things follow. First, it is independent corroboration of the [`operator`](../../../profiles/operator/PROFILE.md) floor: settling and an outcome-bearing action were both reached here without reference to this proposal. Second, and more usefully, the borrowed vocabulary was **insufficient as a floor** — the implementer had to extend it immediately, and nothing in the result distinguishes the nine borrowed names from the sixteen that are not. A consumer sees twenty-five undifferentiated tools.

That is the cost this proposal addresses, observed rather than argued.

## Structured perception is preferred wherever it exists

`P1` states it directly: the server uses *"Playwright's accessibility tree, not pixel-based input"* and `browser_snapshot` is *"better than screenshot."*

`P7` shows the same preference reached independently on the desktop. AutoGUI's `desktop_click_element` talks to the accessibility tree via AT-SPI, *"letting the agent click real UI controls by name and role rather than guessing pixel positions."* Windows 365 Agents exposes *"semantic UI inspection via Windows UI Automation."* computer-use-linux reads accessibility trees across GNOME, KDE, Hyprland, i3, and COSMIC.

**The accessibility tree is the common substrate on both surfaces.** Browsers expose one; desktops expose one (UIA, AT-SPI, AX API). This is why the job is surface-agnostic in practice and not just in principle.

`P5` is the exception that defines the boundary: Anthropic's computer use tool is pure pixels — `screenshot`, `left_click` at `[x, y]`, `mouse_move`, `type`, `key`, `scroll`, `left_click_drag`, `hold_key`, `wait`, and `zoom` over a region. Targets are **coordinates**, the display is addressed by `display_number` / `display_width_px` / `display_height_px`. An operator over a raw framebuffer has no tree to read.

## Two documented failure modes worth designing against

**The model assumes outcomes it did not verify.** `P5` states it plainly: *"Claude sometimes assumes outcomes of its actions without explicitly checking their results."* The documented remedy is a prompt — instructing the model to screenshot after every step and evaluate before proceeding.

That is a workaround for a missing contract term. `P3` fixes it structurally: `act_on` returns a **receipt** describing what actually happened, so the outcome arrives with the action rather than requiring a second call and a prompt to remember to make it.

**Blocking states strand the loop.** `P3` reports `dialog_open` in the receipt and requires explicit resolution via `dialog` before continuing. `P1` and `P2` both carry `handle_dialog`. A modal that the agent cannot see blocks every subsequent action while looking like an ordinary failure.

## Capability grouping already exists, twice, independently

`P1` ships a `--caps` flag that turns tool groups on and off: `config`, `network`, `storage`, `devtools`, `vision`, `pdf`, `testing`. Absent groups are simply not advertised.

`P3` governs by *"capabilities, not tool lists"* with four designations — `read`, `action`, `write`, `execute` — plus per-host manifests and observe-versus-enforce modes.

Two implementations independently arrived at optional capability groups over a common base. Neither calls them facets; both are.

## WebDriver BiDi is the standards-body decomposition

`P6` partitions the same domain into modules, each *"a collection of related commands and events pertaining to a certain aspect of the user agent"*:

| Module | Contains |
|---|---|
| `session` | lifecycle, `subscribe` / `unsubscribe` |
| `browser` | `close`, `createUserContext`, client windows |
| `browsingContext` | `create`, `navigate`, `close`, `captureScreenshot`, `locateNodes`, `handleUserPrompt`, `setViewport` |
| `script` | `evaluate`, `callFunction`, `addPreloadScript` |
| `input` | `performActions`, `releaseActions`, `setFiles` |
| `network` | `addIntercept`, `continueRequest`, `failRequest` |
| `storage` | cookies |
| `emulation` | geolocation, user agent, network conditions, locale |
| `log` | events only |
| `webExtension` | install, uninstall |

Nodes are located via `locateNodes` and returned as **shared references** — the reference primitive, standardized.

Two structural observations matter for the contract:

1. `browsingContext` bundles addressing, navigation, and perception; `input` is actuation. The remaining modules are plainly optional. The module list is close to a facet list already.
2. **BiDi has no wait command.** Settling is done by subscribing to events. MCP operator servers all have an explicit `wait_for` because, until revision `2026-07-28`, MCP had no server-to-client push. With `subscriptions/listen` now available, the mechanism is a genuine design choice rather than a forced one.

## Where the class boundary sits

`P3` declines part of the space explicitly: *"probably not the right tool for a headless scraping farm, stealth automation, an isolated cloud browser."*

`P4` sits elsewhere again — Stagehand exposes `act` / `extract` / `observe` / `agent` over **natural-language targets**, with caching and self-healing so that *"auto-caching combined with self-healing … remembers previous actions, runs without LLM inference."* Its stated philosophy is hybrid: *"use AI when you want to navigate unfamiliar pages, and use code when you know exactly what you want to do."*

So the field contains at least three postures: precise-reference (`P1`, `P2`), semantic-intent (`P4`), and semantic-first-with-precise-escape (`P3`, whose `act_on` takes *"one unique semantic target plus receipt"* while `computer` and `form_input` are *"exact low-level escapes"*).

## Reliability context

Reported benchmark context, recorded for calibration rather than as a claim of this project: the best-performing model on OSWorld 2.0 completed **20.6%** of realistic long-horizon computer-use tasks. Whatever contract is drafted, the failure rate of this capability class is currently very high, and the contract's value should be judged partly on whether it makes failures *legible* rather than only on whether it prevents them.

A University of Washington study (July 2026) reported that four of seven tested agentic browsers allowed same-origin-policy bypass. Security posture is out of scope for a capability contract ([`spec/03 §10`](../../../spec/03-conformance.md#not)), but it is context for why ambient authority must be declared rather than assumed.

## Not established here

- No implementation was invoked. All of the above is documentation.
- Tool lists change; each is a point-in-time reading.
- Nothing here establishes that a shared contract is achievable, only that the same five jobs recur across independent implementations.