Give any AI agent
a real browser

OpenClaw

OpenClaw is an open-source AI agent framework that uses tool-calling to interact with external services. AgentBrowser integrates as a set of browser control tools.

1. Start AgentBrowser: bun run dev (default: http://localhost:3000)
2. Create a tool definition file in your OpenClaw project (tools/browser.yaml):

# tools/browser.yaml
name: browser_navigate
description: "Navigate the browser to a URL"
endpoint: "http://localhost:3000/api/browser/sessions/{session_id}/action"
method: POST
parameters:
  session_id:
    type: string
    description: "Active browser session ID"
  action:
    type: string
    default: "navigate"
  target:
    type: string
    description: "URL to navigate to"

3. Register tools for all actions: browser_click, browser_type, browser_vision, browser_screenshot, browser_scroll, etc.
4. Create a session at agent startup and pass the session_id to all tool calls
5. Use the Vision AI tool (/vision endpoint) to let the agent "see" the page before deciding what to do
6. Close the session when the agent's task is complete via DELETE /sessions/{id}

Hermes

Hermes is a modular AI agent runtime that supports MCP (Model Context Protocol) servers. AgentBrowser can be exposed as an MCP server for native integration.

1. Install and start AgentBrowser: bun run dev
2. Add the MCP server configuration to your Hermes config file (hermes.config.yaml):

# hermes.config.yaml
mcp_servers:
  agentbrowser:
    type: "rest"
    base_url: "http://localhost:3000/api/browser"
    tools:
      - name: "create_session"
        path: "/sessions"
        method: "POST"
        description: "Create a new browser session"
      - name: "execute_action"
        path: "/sessions/{session_id}/action"
        method: "POST"
        description: "Execute a browser action"
      - name: "get_vision"
        path: "/sessions/{session_id}/vision"
        method: "POST"
        description: "Get AI vision snapshot of page"
      - name: "close_session"
        path: "/sessions/{session_id}"
        method: "DELETE"
        description: "Close a browser session"

3. Restart Hermes to load the new MCP server
4. Ask Hermes to browse the web: "Go to Hacker News and find the top 5 stories about AI"
5. Hermes will: Create a session → Navigate → Get vision snapshot → Read interactive elements → Extract data → Close session

Custom Python Agent

Build your own AI agent with any LLM (OpenAI, Anthropic, Ollama, etc.) using the REST API. Here's a complete working example.

import requests

BASE = "http://localhost:3000/api/browser/sessions"

# 1. Create session
s = requests.post(BASE, json={
    "name": "python-agent",
    "browserType": "chromium",
    "headless": True
}).json()
sid = s["id"]

# 2. Navigate
requests.post(f"{BASE}/{sid}/action", json={
    "action": "navigate",
    "target": "https://news.ycombinator.com"
})

# 3. Get vision (for LLM)
vision = requests.post(
    f"{BASE}/{sid}/vision", json={}
).json()

# 4. Pass vision.interactiveElements
#    to your LLM for decision-making
elements = vision["interactiveElements"]
for el in elements:
    print(f"{el['type']}: {el['text']}")

# 5. Clean up
requests.delete(f"{BASE}/{sid}")

OpenAI Function Calling

Define AgentBrowser as an OpenAI tool-calling function. Compatible with GPT-4o, GPT-4, GPT-3.5-turbo, and any model that supports function calling.

# Define tools for your OpenAI agent
tools = [{
    "type": "function",
    "function": {
        "name": "browser_navigate",
        "description": "Navigate browser to URL",
        "parameters": {
            "type": "object",
            "properties": {
                "url": {"type": "string"}
            },
            "required": ["url"]
        }
    }
}, {
    "type": "function",
    "function": {
        "name": "browser_vision",
        "description": "See the current page",
        "parameters": {
            "type": "object",
            "properties": {
                "full_page": {
                    "type": "boolean",
                    "default": false
                }
            }
        }
    }
}]