​

Browser Automation

​

Browser Engine

1. Headless Chrome / Camoufox — Full browser rendering with anti-bot fallback
2. DOM Parser — Structured content extraction (self-healing locators, Shadow DOM)
3. Vision — Screenshot-based interaction for complex UIs

​

12-Layer Anti-Detection Stealth

• CDP leak patches — Patchright eliminates Runtime.enable, Console.enable, and --enable-automation detection vectors at the protocol level
• 13 JS stealth scripts — navigator.webdriver, window.chrome stub, plugin/language emulation, automation artifact cleanup, toString() disguise via WeakMap, anti-debugger neutralization, Performance API cleanup, and more
• Humanized interaction — Random typing delay (30–100ms/char) and click delay (50–150ms) with dynamic timeout adaptation
• Proxy rotation & self-healing — Automatic bad-proxy isolation, context hot-swap for fresh IPs, and stateful session migration (Cookie + localStorage preserved across engine switches)

​

Why This Matters for Users

​

Page Understanding — Dual-Path ARIA Snapshot

​

How It Works

​

What the Agent Sees

- heading "Sign In" [level=1]
- textbox "Email" [ref=1]
- textbox "Password" [ref=2]
- button "Log In" [ref=3]
- link "Forgot password?" [ref=4]

​

Built-in Resilience

• Shadow DOM traversal — Sees through Web Components automatically
• iframe extraction — Cross-origin iframes included in the tree
• Scope filtering — full, interactive, or focused views for token efficiency
• Incremental diff — Only shows what changed between snapshots

​

Why Not Occlusion Detection?

1. It adds 100-500ms per snapshot (reflow for every element)
2. It can accidentally prune valid elements (dropdown menus, tooltips)
3. Playwright’s click() already refuses to click covered elements
4. If a click fails, our Self-Healing Locator fixes it in O(1) time

​

Capabilities

• Navigate to URLs
• Click elements and fill forms
• Extract text and structured data
• Take screenshots
• Handle authentication flows
• Handle browser dialogs intelligently

​

Dialog Policy (Handling Popups Automatically)

​

Configuration

​

How It Works

• In Smart mode (default), your agent never gets stuck on cookie banners or info alerts.
• In Wait for Agent mode, the dialog content appears in the next browser snapshot, giving the AI full context to decide. If the agent doesn’t respond within the timeout, a safe fallback fires automatically.
• Dialog history is always visible to the agent — it knows what popups were handled and how.

​

Credential Vault (Login Without Leaking Passwords)

1. Add a label (e.g. company-admin), password, and optional TOTP seed.
2. Tell the agent to sign in — it references the label only.
3. Myrm injects credentials at the browser DOM or desktop input layer; values never appear in the conversation.

​

Visual Verification (Auto QA)

1. dHash Check (~2ms) — Perceptual hash detects if the screen changed at all. Skips the LLM call if nothing changed, saving tokens.
2. Pixel-Level Diff — Canvas API comparison with YIQ color space and anti-aliasing detection for precise change localization.
3. Vision LLM Scoring — Multimodal AI scores the result 1–5 with reasoning, confirming if your goal was met.

​

Example

​

Privacy

​

Search Integration

• Factual queries
• Navigation requests
• Research tasks
• Price comparison
• News and updates
• Image search
• Local search

​

Computer Use

​

8-Layer Visual Tracing System

​

What You See

​

How It Compares

​

Test Coverage

​

Schema-Driven Structured Extraction

​

How it works

1. Agent calls browser_extract with an extraction_schema parameter (a JSON Schema string).
2. The tool extracts the page text, then uses an LLM to produce structured output matching your schema.
3. You receive clean, validated JSON — not thousands of tokens of raw HTML.

​

Example

{
  "type": "array",
  "items": {
    "type": "object",
    "properties": {
      "name": { "type": "string" },
      "price": { "type": "string" }
    },
    "required": ["name", "price"]
  }
}

​

Key Features

• Object and Array schemas — Extract a single object or a list of items.
• Pagination deduplication — Pass already_collected to avoid duplicates across pages.
• Dual-strategy reliability — Preferred structured_output with raw JSON fallback for model compatibility.
• Token savings — 70–97% reduction compared to raw text extraction.
• Schema complexity protection — Limits on property count and nesting depth prevent misuse.

​

Ephemeral Session Isolation

​

Persistent Session Vault (Cross-Session Login)

​

How It Works

1. Configure domains — In the agent settings WebUI, add domains like twitter.com, github.com to the agent’s auto_restore_domains list.
2. Sessions are encrypted — Cookie and storage data saved via AES-256-GCM encryption in the Session Vault.
3. Auto-restore on startup — Every time the agent launches a new browser session, configured domains have their login state automatically restored.
4. Agent can manage explicitly — The agent can also call browser_manage(action="save_session") / restore_session / list_sessions / delete_session for fine-grained control.

​

Architecture

• AES-256-GCM encryption — Session data is encrypted at rest, never stored in plaintext.
• O(1) LRU memory cache — Hot sessions served from memory; no disk I/O on repeated access.
• Singleflight dedup — Concurrent requests for the same domain are coalesced into one decrypt operation.
• TTL auto-expiry — Stale sessions are garbage-collected automatically.
• Full execution coverage — Works across Web, Channel, Cron, and Eval execution contexts.

​

When to Use

​

Session–Memory Bridge (Zero-Cost Awareness)

• How it works: Every time a session is saved or deleted, a SessionMemoryBridge updates the agent’s active_browser_sessions profile attribute. This attribute is injected into every LLM turn via the memory context middleware.
• Result: When a user says “post a tweet”, the agent already knows twitter.com is available — it calls restore_session("twitter.com") directly, skipping the list_sessions step and saving one LLM inference round.
• Prompt cache safe: The profile attribute only changes on save/delete (very infrequent), so prompt prefix caching is unaffected.

​

CAPTCHA Detection & Human Takeover

​

Automatic CAPTCHA Detection

1. Agent pauses execution via LangGraph interrupt()
2. The VNC panel auto-opens showing the browser
3. A notification appears with the reason for the pause
4. You solve the CAPTCHA manually
5. Agent resumes automatically and continues the task

​

Agent-Triggered Human Takeover

1. Agent recognizes it cannot proceed (e.g., “this requires SMS verification”)
2. Calls browser_ask_human with a clear explanation
3. VNC panel opens with the reason displayed
4. You complete the action directly in the browser
5. Click “Done” — agent captures the new page state and continues

​

Why This Matters

​

Browser Extension Bridge

​

When to Use

​

How It Works

1. Install the Myrm Extension — A lightweight Chrome MV3 extension that maintains a WebSocket connection to the Myrm server.
2. Authorize domains — In Settings → Integration → Extension Bridge, add domains you want the agent to control (e.g., github.com, *.google.com). Wildcard patterns are supported.
3. Agent connects — When a task requires browser access to an authorized domain, the agent connects to your real browser via CDP (Chrome DevTools Protocol) proxy.

​

Security Model

• Domain-level authorization — Only explicitly authorized domains can be controlled. The agent cannot access any tab outside your allow-list.
• Token authentication — The extension authenticates to the server with a secret token, preventing unauthorized connections.
• Service Worker keepalive — The MV3 extension uses chrome.alarms for persistent connection with exponential backoff reconnection.
• Domain sync on reconnect — When the extension reconnects, it immediately synchronizes authorized domains with the server.

​

Frontend Management

• One-click copyable WebSocket URL — The correct connection URL for your deployment mode, with a Copy button and toast confirmation
• Auth token status — Shows whether EXTENSION_AUTH_TOKEN is configured on the server (without exposing the token itself)
• Copyable extension path — Shows ~/.myrm/myrm-agent/myrm-agent-extension with one-click copy for Chrome’s “Load unpacked” dialog
• Step-by-step setup guide — Appears when extension is not connected: Load unpacked extension → Copy URL → Connect
• Real-time connection status — Connected/disconnected indicator with extension version and browser name
• Domain authorization management — Add, remove, and wildcard pattern support
• Available tabs list — Filtered by authorized domains, shows active tab indicator

​

Architecture

1. Harness Protocol (ExtensionBridge) — Defines the contract at the framework level
2. Server Service (ExtensionBridgeService) — Manages WebSocket lifecycle, heartbeat, and CDP proxy
3. API Router — WebSocket + REST endpoints for extension and frontend
4. Chrome Extension — MV3 Service Worker with CDP attachment capability
5. Frontend Panel — React settings component for status and domain management

​

Per-Agent Browser Source Configuration

​

Available Modes

​

How to Configure

1. Agent Creation: Settings → Agents → Create/Edit → Browser Source
2. Per-Chat Override: Click the agent config icon in the chat window → Browser Source

​

Pool Isolation

• Extension-sourced browsers (unmanaged) are never shared with launch-mode agents
• Launch-sourced browsers (managed) are never shared with extension-mode agents
• This prevents session cross-contamination between agents with different trust levels

​

Zero-Config Browser Setup

​

How It Works

1. Agent requests a browser page for a task
2. If Chromium is not installed, Myrm detects the missing executable
3. Automatic download and installation runs in the background
4. The browser task continues seamlessly after installation

​

Smart Failure Handling

• Cooldown protection — After a failed install, Myrm waits 30 minutes before retrying to avoid blocking your workflow
• Concurrency safe — Multiple tasks requesting browsers simultaneously are serialized to prevent conflicts
• Diagnostic messages — Shows exactly what went wrong (disk space, network, permissions) and how to fix it
• Doctor auto-fix — Run myrm doctor with auto-fix to diagnose and repair browser issues

​

Why This Matters