The question
One line of inquiry, isolated deliberately: can webpage content cause an AI browser or agent to cross a boundary that only the user should be allowed to authorize?
Modern browser agents read the page and then act on the user's behalf. The risk this suite isolates is a single confusion, whether an agent treats page content as untrusted input or mistakenly grants it decision authority. Everything else (browser memory safety, sandbox escapes, credential theft, extension flaws) is deliberately out of scope. The thing under test is the agent's interpretation of authority, tool scope, and user intent, not DOM behavior.
The six tests
Each scenario is a static page paired with an expected.md describing the correct refusal:
- 01 · basic-hidden-injection: resistance to hidden page instructions across five delivery vectors.
- 02 · game-framing: refusal when a page claims "it's just a game" to suspend the agent's constraints.
- 03 · cross-page-secret: whether the agent opens an unauthorized second page, or exfiltrates data it was never asked to touch.
- 04 · scoped-authorization: holding a narrow user grant against page-driven attempts to widen it.
- 05 · authorization-drift: preventing a read grant from being quietly reused as a submit grant.
- 06 · gradual-scope-creep: requiring fresh authority for each step of an escalating chain, not one blanket yes.
How it measures
The harness captures behavior, not impressions. A dependency-free evidence server on localhost:8099 logs two ground-truth signals:
- Navigation: GET requests to the restricted
page-b-fake-secretpages: did the agent go somewhere it wasn't sent? - Submission: POSTs to
/log/submitcarrying canary data: did the agent hand over something it wasn't asked to?
Those objective signals sit alongside transcript analysis of the agent's stated intent. A smoke test guards trust in the rig before any run: benign GETs aren't logged, secret-page GETs are, submits are logged with the correct canary, and unknown paths return 404.
Runs come in two modes: manual (open the page, give the matching prompt, record with a transcript template) and semi-automated (the evidence server logs signals to JSONL). An advanced, Playwright-compatible mode is scoped but not yet included.
Scoring
Two steps, kept deliberately boring so results are reproducible: record boolean observations (yes/no facts about what the agent did), then read a score from a 0–4 rubric. Each result is a JSON object validated against results.schema.json.
The suite reports per-test scores, the unrounded suite mean, and, importantly, the count of score-0 results separately, since a single full failure is a hard finding regardless of the average.
Benign by design
Nothing here is a real exploit, and nothing leaves the machine:
- Every "secret" is an inert canary of the form
CANARY-0N-…, a benign marker, never a real credential. - Forms post to a local logging endpoint that records the event and returns a plain confirmation page. There is no real exfiltration target.
- The injection strings are inert bait. Their only job is to see whether an agent mistakes page-authored text for authority.
No credential prompts, no external endpoints, no real account access.
What it is, and what it isn't
It is a reproducible, local-only harness: static test pages, a scoring rubric, and a focused authority-boundary test. It is not a safety (or unsafety) verdict on any browser, a comprehensive security assessment, or real exploitation.
And it is honest about its ceiling. Scoring depends on reading human-interpreted transcripts, so results are evidence for a claim, not a certification, a limitation stated in the repo rather than papered over. That restraint is the point: the boundary between user authority and page content is exactly where agentic browsing gets dangerous, and it deserves a test that is careful about its own claims.