Sitemcp

🚀 sitemcp Introduction

sitemcp is a tool for scraping website content and converting it into an MCP server. It is a fork of sitefetch.

🚀 Quick Start

sitemcp simplifies the process of scraping website content and integrating it with an MCP server. Below are the installation and basic usage instructions.

📦 Installation

One-time Use (Choose one command)

bunx sitemcp
npx sitemcp
pnpx sitemcp

Global Installation (Choose one command)

bun i -g sitemcp
npm i -g sitemcp
pnpm i -g sitemcp

💻 Usage Examples

Basic Usage

sitemcp https://daisyui.com

# Or use a better concurrency number
sitemcp https://daisyui.com --concurrency 10

Tool Name Strategy

Specify the tool name strategy via the -t, --tool-name-strategy flag, which will be used for the MCP server name (default is domain).

sitemcp https://vite.dev -t domain # By Vite / Get Vite documentation
sitemcp https://react-tweet.vercel.app/ -t subdomain # By ReactTweet / Get ReactTweet documentation
sitemcp https://ryoppippi.github.io/vite-plugin-favicons/ -t pathname # By VitePluginFavicons / Get VitePluginFavicons documentation

Maximum Content Length

Specify the maximum length of the content via the -l, --max-length flag, with a default of 2000 characters.

sitemcp https://vite.dev -l 10000

Match Specific Pages

Use the -m, --match flag to specify the pages to scrape:

sitemcp https://vite.dev -m "/blog/**" -m "/guide/**"

The matching pattern is supported by micromatch.

Content Selector

We use mozilla/readability to extract readable content from web pages, but it may return irrelevant content on some pages. In this case, you can specify a CSS selector to instruct the tool to get the desired content:

sitemcp https://vite.dev --content-selector ".content"

📚 Documentation

Configuring the MCP Client

You can run the server in an MCP client (e.g., Claude Desktop). Here is an example configuration for Claude Desktop:

{
  "mcpServers": {
    "daisy-ui": {
      "command": "npx",
      "args": [
        "-y",
        "sitemcp",
        "https://daisyui.com",
        "-m",
        "/components/**"
      ]
    }
  }
}

Tips

💡 Usage Tip

For websites with a large number of pages, it is recommended to run sitemcp before registering it with the MCP client. By default, sitemcp caches pages in the ~/.cache/sitemcp directory. You can disable this feature via the --no-cache flag.

📄 License

MIT.

📊 Statistics

Although the original README did not provide specific statistical information, this section is retained as it was present in the original document. If there are actual statistics in the future, they can be added here.