Site Cloner

🚀 Site Cloner MCP Server

A server designed based on MCP (Model Context Protocol) to assist large language models (such as Claude) in website cloning. This server offers various tools for scraping, analyzing, and downloading website resources.

🚀 Quick Start

The Site Cloner MCP Server is designed to facilitate website cloning for large language models. Here's how you can get started:

Prerequisites

• Docker must be installed on your system.

Running via Docker

1. Build the Docker image:

docker build -t site-cloner-mcp .

2. Start the container:

docker run -i --rm site-cloner-mcp

To achieve persistent storage of downloaded files, you can mount a volume:

docker run -i --rm -v $(pwd)/downloaded_sites:/app/downloaded_site site-cloner-mcp

Connecting with Cursor

There are two ways to set up this MCP server in Cursor:

Method 1: Project-Specific Configuration

Create a .cursor/mcp.json file in the project root directory with the following content:

{
  "mcpServers": {
    "site-cloner": {
      "command": "docker",
      "args": [
        "run",
        "-i",
        "--rm",
        "site-cloner-mcp"
      ]
    }
  }
}

Method 2: Global Configuration

To use the MCP server globally in Cursor, go to Cursor Settings → MCP → Add New Global MCP Server and add the following content:

{
  "mcpServers": {
    "site-cloner": {
      "command": "docker",
      "args": [
        "run",
        "-i",
        "--rm",
        "site-cloner-mcp"
      ]
    }
  }
}

✨ Features

• Scrape the HTML content of any URL.
• Extract resources (CSS, JavaScript, images, fonts, etc.) from HTML.
• Download individual resources to a local directory.
• Parse CSS files to extract linked resources (fonts, images).
• Create a sitemap.
• Analyze page structure and layout.

📦 Installation

This server requires Docker to be installed. Please ensure Docker is installed on your system before proceeding.

💻 Usage Examples

Available Tools

1. fetch_page

Scrape the HTML content of a web page.

Parameters:
    url: The URL of the web page to scrape

2. extract_assets

Extract resource links from HTML.

Parameters:
    url: The URL of the web page (used to resolve relative URLs)
    html_content: The HTML content to parse

3. download_asset

Download a resource based on its URL and save it to a specified directory.

Parameters:
    url: The URL of the resource
    directory: The directory to save the resource

4. parse_css

Parse a CSS file to extract linked resources.

Parameters:
    css_content: The CSS content to parse

5. create_sitemap

Create a sitemap for a website.

Parameters:
    url: The URL of the website

6. analyze_page_structure

Analyze the structure and layout of a web page.

Parameters:
    html_content: The HTML content to analyze

📚 Documentation

Troubleshooting

• If you encounter issues while cloning a website, check if the target website allows automated requests. You may need to adjust the user-agent string.
• If some resources cannot be downloaded, ensure your network connection is stable and check if the target website has access restrictions.

📄 License

Please note:

• Downloaded resources will be stored in different subdirectories according to their types (html, css, js, images, fonts, videos, others).
• When cloning websites, please comply with relevant laws, regulations, and the website's terms of service, and respect copyright.