Skrape MCP

🚀 Skrape MCP Server

Use skrape.ai to convert any web page into clean, LLM-friendly Markdown format. It's perfect for feeding web content into LLMs.

This MCP server provides a simple interface to convert web pages into structured, clean Markdown format using the skrape.ai API. It is designed to seamlessly integrate with Claude Desktop, other LLMs, and MCP-compatible applications.

🚀 Quick Start

Why Integrate with Skrape for LLMs

• Clean, Structured Output: Generate well-formatted Markdown suitable for LLM consumption.
• Noise Reduction: Automatically remove ads, navigation menus, and other irrelevant content.
• Consistent Format: Ensure uniform structuring of web content regardless of the source.
• JavaScript Support: Handle dynamic content by rendering JavaScript.
• LLM Optimization: Ideal for feeding web content into Claude, GPT, and other LLM models.

✨ Features

Tools

• get_markdown - Convert any web page into LLM-friendly Markdown.
  • Accepts any input URL and optional parameters.
  • Returns clean, structured Markdown suitable for LLM consumption.
  • Supports JavaScript rendering to handle dynamic content.
  • Optional JSON response format for advanced integration.

📦 Installation

Automatic Installation with Smithery

To automatically install the Skrape MCP server for Claude Desktop via Smithery:

npx -y @smithery/cli install @skrapeai/skrape-mcp --client claude

Manual Installation

1. Obtain your API key from skrape.ai.

2. Install dependencies:

npm install

3. Build the server:

npm run build

4. Add the server configuration to Claude Desktop:

On MacOS:

nano ~/Library/Application\ Support/Claude/claude_desktop_config.json

On Windows:

notepad %APPDATA%/Claude/claude_desktop_config.json

Add the following configuration (replace with your path and API key):

{
  "mcpServers": {
    "skrape": {
      "command": "node",
      "args": ["path/to/skrape-mcp/build/index.js"],
      "env": {
        "SKRAPE_API_KEY": "your-key-here"
      }
    }
  }
}

💻 Usage Examples

Basic Usage with LLMs

Here are the steps to use the server with Claude or other LLM models:

1. First, ensure the server is correctly configured in your LLM application.
2. Then, you can instruct ALLMI to scrape and process any web page:

Convert this web page to Markdown: https://example.com

Claude will use the MCP tool as follows:
<use_mcp_tool>
<server_name>skrape</server_name>
<tool_name>get_markdown</tool_name>
<arguments>
{
  "url": "https://example.com",
  "options": {
    "renderJs": true
  }
}
</arguments>
</use_mcp_tool>

The generated Markdown will be clean, structured, and ready for LLM processing.

Advanced Usage

The get_markdown tool accepts the following parameters:

• url (required): The URL of any web page to convert.
• returnJson (optional): Set to true to get a full JSON response instead of just Markdown.
• options (optional): Additional scraping options.
  • renderJs: Whether to render JavaScript before scraping (default: true).

Example with all options:

<use_mcp_tool>
<server_name>skrape</server_name>
<tool_name>get_markdown</tool_name>
<arguments>
{
  "url": "https://example.com",
  "returnJson": true,
  "options": {
    "renderJs": true
  }
}
</arguments>
</use_mcp_tool>

📚 Documentation

Project Links

• GitHub
• npm

Function Overview

1. Server Installation and Configuration

• Step 1: Clone the repository

  git clone https://github.com/skrape-ai/skrape-mcp-server.git
  cd skrape-mcp-server
  

• Step 2: Install dependencies

  npm install
  

• Step 3: Configure the server

  • Create a config.json file:

    {
      "port": 3000,
      "api_key": "your_api_key_here"
    }
    

• Step 4: Start the server

  npm start
  

2. Client Usage Examples

Example 1: Basic Usage

const { Client } = require('@skrape/mcp-client');

async function main() {
    const client = new Client('localhost', 3000);
    const result = await client.getMarkdown('https://example.com');
    console.log(result.markdown); // Output Markdown content
}

main().catch(console.error);

Example 2: Using Advanced Options

const { Client } = require('@skrape/mcp-client');

async function main() {
    const options = {
        renderJs: true,
        returnJson: true
    };
    
    const client = new Client('localhost', 3000);
    const result = await client.getMarkdown('https://example.com', options);
    console.log(result); // Output JSON response
}

main().catch(console.error);

Official Documentation

• Official Docs: Skrape MCP Server Documentation
• Client Library: @skrape/mcp-client

Support and Contact

• Issue Reporting: Submit issues on GitHub Issues.
• Contact Information: Support Email
• Community Discussion: Join the Skrape Discord Group

Other Resources

• Video Tutorials: How to Use the Skrape MCP Server
• Blog Posts: Boost Web Scraping Efficiency with Skrape