MCP Smart Crawler

🚀 MCP Smart Crawler

MCP Smart Crawler is a Model Context Protocol (MCP) server that uses Playwright to scrape web page content, extract metadata, and download resources such as videos and images.

🚀 Quick Start

✨ Features

• Extract metadata (title, description, images) from Xiaohongshu posts.
• Download videos and images from Xiaohongshu sharing links.
• Perform browser automation using Playwright.

💻 Usage Examples

MCP Client Configuration

To use this server in your MCP client, add the following configuration to the MCP server settings of the client. You need to adjust the values of command and args according to how the server script runs.

Example Configuration:

{
  "mcpServers": {
    "mcp-smart-crawler": {
      "command": "npx",
      "args": [
        "-y",
        "mcp-smart-crawler",
        "--download-folder", // Optional parameter
        "c:\\downloads" // Optional parameter value
      ]
    }
  }
}

Example Conversation:

Help me check the content and pictures of this Xiaohongshu post, and tell me what's in the pictures.
59 XXX posted a Xiaohongshu note, come and have a look! 😆 OfTOBst2PsxctaX 😆 http://xhslink.com/a/xxaabbcczz, copy this message and open the [Xiaohongshu] App to view the wonderful content!

⚠️ Important Note

The above content is a complete translation, containing only the translated content without any additional information.