Browsercontrol

🚀 BrowserControl

BrowserControl is an MCP server that empowers your AI agent with full browser access through a vision-first approach. It allows the AI to truly "see", "click", "type", and "interact" with any website, just like a human, without relying on CSS selectors, XPath, or guesswork.

🚀 Quick Start

Installation

# Using pip
pip install browsercontrol

# Or with uv (recommended for faster installs)
uv add browsercontrol

# Chromium is auto-installed on first run—no extra steps needed!

Run the Server

# Using the CLI
browsercontrol

# Or as a Python module
python -m browsercontrol

# Or with FastMCP
fastmcp run browsercontrol.server:mcp

Connect to Your AI Agent

BrowserControl is compatible with any MCP-compatible AI agent or IDE. Here are the connection methods for different platforms:

{
  "mcpServers": {
    "browsercontrol": {
      "command": "browsercontrol"
    }
  }
}

# Set up MCP configuration
export MCP_SERVERS='{"browsercontrol": {"command": "browsercontrol"}}'

# Or add to your Gemini config file

{
  "browsercontrol": {
    "command": "browsercontrol"
  }
}

{
  "mcpServers": [
    {
      "name": "browsercontrol",
      "command": "browsercontrol"
    }
  ]
}

{
  "browsercontrol": {
    "command": "browsercontrol"
  }
}

{
  "context_servers": {
    "browsercontrol": {
      "command": {
        "path": "browsercontrol"
      }
    }
  }
}

from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client

# Connect to BrowserControl
server_params = StdioServerParameters(
    command="browsercontrol",
    args=[],
)

async with stdio_client(server_params) as (read, write):
    async with ClientSession(read, write) as session:
        # Initialize
        await session.initialize()

        # List available tools
        tools = await session.list_tools()

        # Call a tool
        result = await session.call_tool("navigate_to", {
            "url": "https://github.com"
        })

{
  "mcpServers": {
    "browsercontrol": {
      "command": "uvx",
      "args": ["browsercontrol"]
    }
  }
}

{
  "mcpServers": {
    "browsercontrol": {
      "command": "pipx",
      "args": ["run", "browsercontrol"]
    }
  }
}

{
  "mcpServers": {
    "browsercontrol": {
      "command": "browsercontrol",
      "env": {
        "BROWSER_HEADLESS": "false",
        "BROWSER_VIEWPORT_WIDTH": "1920",
        "BROWSER_VIEWPORT_HEIGHT": "1080",
        "LOG_LEVEL": "DEBUG"
      }
    }
  }
}

✨ Features

❌ Traditional Approach

"Find the button with class 'btn-primary'
that contains 'Submit' and is inside
form#contact-form..."

• Parse complex DOM structures.
• Guess at CSS selectors.
• No JavaScript support.
• No login persistence.
• No debugging tools.

✅ BrowserControl

"click(7)"

• See the rendered page with numbered elements.
• Just say "click 5" or "type in 3".
• Full dynamic JavaScript support.
• Persistent sessions across restarts.
• Complete DevTools access.

🎯 The Secret: Set of Marks (SoM)

Every screenshot comes annotated with numbered red boxes on interactive elements:

Found 15 interactive elements:
  [1] button - Sign In
  [2] input - Search...
  [3] a - Products
  [4] a - Pricing
  [5] button - Get Started

Your agent sees the numbers and simply calls click(1) to sign in. No CSS selectors. No XPath. No guessing.

🥊 Head-to-Head Comparison

💪 Key Advantages

1. Multi-Tab Orchestration

Unlike other tools that get "lost" when a new window opens:

• list_tabs() — See every open page, title, and URL.
• switch_tab(index) — Multitask between different sites.
• create_tab(url) — Open references or parallel workflows.

2. Session & Cookie Management

Stop struggling with login forms. Inject or inspect session state directly:

• set_cookie() — Log in instantly by injecting an auth token.
• get_cookies() — Debug session issues or export state.
• clear_cookies() — Start fresh without clearing the whole profile.

3. Reliable File Uploads

Most AI agents fail when they encounter a <input type="file">. BrowserControl uses native browser engine hooks:

• upload_file(id, path) — Just point at the button and the local file.

4. Developer Tools Suite

Debug like a pro with tools no one else provides:

get_console_logs()      # See browser errors
get_network_requests()  # Monitor API calls
get_page_errors()       # Catch JS exceptions
run_in_console(code)    # Debug in real-time
inspect_element(5)      # Get computed styles
get_page_performance()  # Core Web Vitals

5. Session Recording

start_recording()  →  Browse around  →  stop_recording()
                                              ↓
                               session_20260202.zip
                         (View with Playwright trace viewer)

6. Dynamic Viewport Control

Test responsive designs or emulate mobile screens on the fly:

• set_viewport(width, height) — Change resolution without restarting.

7. True Persistence

Result: Log in once, stay logged in across sessions.

🛠️ Available Tools

Navigation

Interaction

Tab Management

Forms

Content Extraction

Developer Tools

Recording

⚙️ Configuration

Configure via environment variables:

Examples:

# Run with visible browser (for debugging)
BROWSER_HEADLESS=false browsercontrol

# Mobile viewport emulation
BROWSER_VIEWPORT_WIDTH=375 BROWSER_VIEWPORT_HEIGHT=812 browsercontrol

# Verbose logging
LOG_LEVEL=DEBUG browsercontrol

💻 Usage Examples

Web Research

You: "Go to Wikipedia and find out when Python was created"

Claude: *navigates to wikipedia.org*
        *types "Python programming language" in search*
        *clicks search result*
        *reads the page*

        "Python was created by Guido van Rossum and first released in 1991."

Debug a Web App

You: "Go to my app at localhost:3000 and check for any errors"

Claude: *navigates to localhost:3000*
        *calls get_console_logs()*

        "I found 2 errors:
         [ERROR] Uncaught TypeError: Cannot read property 'map' of undefined
         [ERROR] Failed to load resource: 404 /api/users"

        *calls get_network_requests()*

        "The /api/users endpoint returned 404. Check your API route."

Automated Testing with Recording

You: "Test the login flow on my site. Start recording."

Claude: *calls start_recording()*
        *navigates to site*
        *types email in element 3*
        *types password in element 4*
        *clicks Login button (element 5)*
        *verifies successful redirect*
        *calls stop_recording()*

        "Login test passed! Recording saved to session_20260202.zip"

Form Automation

You: "Fill out the contact form on example.com"

Claude: *navigates to example.com/contact*
        *types name in element 2*
        *types email in element 3*
        *types message in element 4*
        *clicks Submit (element 5)*

        "Form submitted successfully!"

📚 Documentation

Architecture

┌─────────────────┐     ┌──────────────────┐     ┌─────────────┐
│   AI Agent      │────▶│  BrowserControl  │────▶│   Browser   │
│ (Claude/Gemini) │◀────│   MCP Server     │◀────│ (Chromium)  │
└─────────────────┘     └──────────────────┘     └─────────────┘
        │                        │                      │
        │   "click(5)"           │   mouse.click()      │
        │◀───────────────────────│◀─────────────────────│
        │   [annotated           │   [screenshot +      │
        │    screenshot]         │    element map]      │

How It Works

1. AI sends command — click(5)
2. Server finds element — Looks up element #5 from the last screenshot.
3. Browser acts — Clicks at the element's coordinates.
4. Capture state — Takes new screenshot, detects elements.
5. Annotate — Draws numbered boxes on interactive elements.
6. Return to AI — Sends annotated image + element list.

Project Structure

browsercontrol/
├── __init__.py          # Package exports
├── __main__.py          # CLI entry point
├── server.py            # MCP server setup
├── browser.py           # BrowserManager with SoM
├── config.py            # Environment configuration
└── tools/
    ├── navigation.py    # Navigation tools
    ├── interaction.py   # Click, type, hover tools
    ├── forms.py         # Form handling tools
    ├── content.py       # Content extraction tools
    ├── devtools.py      # Developer tools
    ├── recording.py     # Session recording tools
    └── tabs.py          # Tab management tools

🔧 Technical Details

Troubleshooting

xvfb-run browsercontrol

python -m playwright install chromium

ls -la ~/.browsercontrol/

pkill -f browsercontrol
browsercontrol

npx playwright show-trace ~/.browsercontrol/recordings/session.zip

🤝 Contributing

Contributions are welcome! Check out our Contributing Guide for details.

Ideas for contributions:

• [ ] Firefox/WebKit support
• [ ] DOM diffing (detect changes)
• [ ] Accessibility audit tools
• [ ] Mobile emulation presets
• [ ] Cookie import/export files

# Clone and install
git clone https://github.com/adityasasidhar/browsercontrol
cd browsercontrol
uv sync

# Run tests
uv run pytest

# Run in development
uv run fastmcp dev browsercontrol/server.py

📄 License

MIT License — Use it however you want.

🙏 Acknowledgments

• Vision-first approach inspired by Google's AntiGravity IDE.
• Built with FastMCP and Playwright.
• Thanks to the MCP community for making AI-tool integration accessible.

Built for AI agents that need to see the web.

⭐ Star on GitHub • 🐛 Report Bug • 💡 Request Feature