MCP Web Search

🚀 Web Search MCP Server

An MCP server that provides Google search functionality and web content viewing, equipped with advanced anti - scraping mechanisms.

🚀 Quick Start

This is an MCP server that offers Google search functionality and web content viewing, along with advanced anti - scraping mechanisms.

✨ Features

• Google Custom Search: Supports advanced filtering.
• Web Content Viewing: Comes with Markdown conversion functionality.
• Rate Limiting and Caching
• Browser Instance Pool
• Anti - Scraping Detection: Utilizes rebrowser - puppeteer technology.

📦 Installation

Prerequisites

• Bun runtime environment v1.0 or higher.
• Google API credentials (API Key and Search Application ID).

Installation Steps

# Run in development mode and watch for file changes
bun --watch run dev

# Execute tests
bun run test

# Run code checks
bun run lint

📚 Documentation

Configuration Guide

1. Cookie Settings

To implement authentication, configure as follows:

• Windows: Copy the config - example.json file to the %APPDATA%\Claude\mcp - clients\web - search directory and rename it to config.json.
• Mac/Linux: Copy the file to ~/.claude/mcp - clients/web - search and rename it to config.json.

2. Server Configuration

Add the following content to the config.json file:

{
  "browserLimits": {
    "globalLimit": 5,
    "requesterLimit": 2
  },
  "proxySettings": {
    "useProxy": true,
    "proxyUrl": "http://your - proxy - server:port",
    "proxyEnabledForDomains": ["example.com", "another - site.com"]
  }
}

Log Configuration

# View log files
# MacOS/Linux:
tail -n 20 -f ~/Library/Logs/Claude/mcp*.log

# Windows:
type %APPDATA%\Claude\Logs\mcp*.log

Anti - Scraping Mechanism Explanation

• Request Frequency Limitation: Control concurrent and total request numbers through the browserLimits configuration.
• Proxy Settings: Configure proxySettings to bypass anti - scraping detection, supporting proxy usage for specified domains.

Tool Interfaces

Search Tool

{
  name: "search",
  params: {
    query: string;          // Required: Search keyword
    trustedDomains?: string[]; // Optional: List of trusted domains
    excludedDomains?: string[]; // Optional: List of excluded domains
    resultCount?: number;   // Optional: Number of returned results, default is 10
    safeSearch?: boolean;   // Optional: Enable safe search, default is true
    dateRestrict?: string;  // Optional: Time limit, supports options like "d" (last day), "w" (last week)
  }
}

Web Page Viewing Tool

{
  name: "view_url",
  params: {
    url: string;            // Required: Target web page address
    includeImages?: boolean; // Optional: Whether to include images, default is true
    includeVideos?: boolean; // Optional: Whether to include videos, default is true
    preserveLinks?: boolean; // Optional: Keep links intact, default is true
    formatCode?: boolean;   // Optional: Format code output, default is false
  }
}

Notes

⚠️ Important Note

• The server has multiple built - in anti - scraping mechanisms, but you still need to combine proxy and user - agent settings to enhance security.
• Some websites may still detect automated behavior.

💡 Usage Tip

• The browser instance pooling design can improve resource utilization.
• Smartly clean idle browsers to avoid resource exhaustion.

📄 License

This project follows the MIT open - source license. Please refer to the LICENSE file for specific terms.