Eget MCP

🚀 Using the Claude Web Crawler MCP

A simple Model Context Protocol (MCP) server that connects Claude for Desktop with the locally running eGet web crawler. This enables Claude to scrape website content via a local API.

🚀 Quick Start

✨ Features

• Connects Claude for Desktop with the eGet web crawler.
• Allows Claude to scrape website content through a local API.

📦 Installation

Prerequisites

• Claude for Desktop
• Python 3.7+
• eGet web crawler (from https://github.com/vishwajeetdabholkar/eGet-Crawler-for-ai)

Step 1: Set up the eGet Web Crawler

First, make sure the eGet web crawler is running:

# Clone the eGet repository
git clone https://github.com/vishwajeetdabholkar/eGet-Crawler-for-ai
cd eGet-Crawler-for-ai

# Set up and run eGet according to its instructions (usually using Docker or a local Python installation)

# Verify that the API is running (default: http://localhost:8000/api/v1/scrape)

Step 2: Set up the MCP Server

# Create the project directory
mkdir claude-scraper-mcp
cd claude-scraper-mcp

# Set up UV and the virtual environment
uv venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install dependencies
uv add "mcp[cli]" httpx

# Create the MCP server script
touch scrape_mcp_server.py

Copy the code for scrape_mcp_server.py into the file.

Step 3: Configure Claude for Desktop

1. Create or edit the Claude desktop configuration:

# On macOS
mkdir -p ~/Library/Application\ Support/Claude/

2. Add the following configuration to the ~/Library/Application Support/Claude/claude_desktop_config.json file:

{
    "mcpServers": {
        "scrape-service": {
            "command": "/absolute/path/to/claude-scraper-mcp/.venv/bin/python",
            "args": [
                "/absolute/path/to/claude-scraper-mcp/scrape_mcp_server.py"
            ]
        }
    }
}

Replace the paths with the actual absolute paths. 3. Restart Claude for Desktop

💻 Usage Examples

Basic Usage

After setting up, you can use Claude for web scraping, for example:

• "Scrape the content of https://example.com and summarize it"
• "Get information about the https://news.ycombinator.com website"

🔧 Troubleshooting

If you encounter problems:

1. Check if the eGet crawler is running.
2. Verify that the API endpoint in the script matches your eGet configuration.
3. Ensure that Claude for Desktop uses the correct Python interpreter.
4. Restart Claude for Desktop after changing the configuration.