Website Scraper And Analyzer

🚀 Build an Authentication-Free Remote MCP Server on Cloudflare

This example allows you to deploy an authentication-free remote MCP server on Cloudflare Workers. The server integrates website analysis and content extraction tools, leveraging Cloudflare's browser rendering and artificial intelligence capabilities.

🚀 Quick Start

This will deploy your MCP server to a URL like remote-mcp-server-authless.<your-account>.workers.dev/sse.

Alternatively, you can also use the following command line to create a remote MCP server on your local machine:

npm create cloudflare@latest -- my-mcp-server --template=cloudflare/ai/demos/remote-mcp-authless

✨ Features

Available Tools

Website Analysis Tools

1. analyze_website

   • Crawl and analyze websites using Cloudflare's browser rendering technology.
   • Provide summaries of website content using Cloudflare AI.
   • Input: url (string) - The URL of the website to be analyzed.
   • Example: [tool] analyze_website(url: "https://example.com")

2. ask_q_about_website

   • Crawl websites and answer specific questions about their content.
   • Use Cloudflare AI to understand and respond to questions about website content.
   • Input:
     • url (string) - The URL of the website to be analyzed.
     • question (string) - A question about the website content.
   • Example: [tool] ask_q_about_website(url: "https://example.com", question: "What services does this company offer?")

🔧 Customize Your MCP Server

To add your own tools to the MCP server, define each tool using this.server.tool(...) in the init() method of src/index.ts.

📚 Connect to the Cloudflare AI Playground

You can connect to your MCP server in the following ways:

1. Visit https://playground.ai.cloudflare.com/.
2. Enter the URL of your deployed MCP server (remote-mcp-server-authless.<your-account>.workers.dev/sse).
3. Now, you can use your MCP tools directly from the playground!

📚 Connect Claude Desktop to Your MCP Server

You can also connect to the remote MCP server via the mcp-remote proxy.

To connect Claude Desktop to your remote MCP server, follow Anthropic's Quick Start Guide and make the following changes in Settings > Developers > Edit Configuration in Claude Desktop:

Update to the following configuration:

{
  "mcpServers": {
    "calculator": {
      "command": "npx",
      "args": [
        "mcp-remote",
        "http://localhost:8787/sse"  // or remote-mcp-server-authless.your-account.workers.dev/sse
      ]
    }
  }
}

After restarting Claude, you should see the tools become available.

📦 Prerequisites

To use the website analysis tools, you need:

1. Enable Cloudflare Workers with browser bindings.
2. Configure Cloudflare AI bindings.
3. Configure appropriate permissions for both bindings.

Ensure your wrangler.jsonc contains the following:

{
  "browser": {
    "binding": "BROWSER"
  },
  "ai": {
    "binding": "AI"
  }
}