Scraper.is

🚀 Scraper.is MCP Integration Package

A Model Context Protocol (MCP) integration package for Scraper.is, a powerful web scraping tool designed for AI assistants.

This package integrates with Scraper.is via the MCP protocol, allowing AI assistants to access real-time information from the web.

🚀 Quick Start

To get started with the Scraper.is MCP integration package, you first need to install it and obtain an API key. Then you can configure it according to your needs.

✨ Features

• 🌐 Web Scraping: Extract content from any website.
• 📸 Screenshots: Capture visual representations of web pages.
• 📄 Multiple Formats: Get content in Markdown, HTML, or JSON.
• 🔄 Progress Updates: Report the progress of scraping operations in real-time.
• 🔌 MCP Integration: Seamlessly integrate with AI assistants that support MCP.

📦 Installation

You can install the package using npm or yarn:

npm install -g scraperis-mcp

Or use yarn:

yarn global add scraperis-mcp

Prerequisites

You need a Scraper.is API key to use this package.

Get Your API Key

1. Sign up or log in at scraper.is.
2. Visit the API Keys section in the dashboard: https://www.scraper.is/dashboard/apikeys.
3. Create a new key or copy an existing one.
4. Securely save this key as you'll need it to use this package.

💻 Usage Examples

Basic Usage

Environment Setup

Create a .env file and add your Scraper.is API key:

SCRAPERIS_API_KEY=your_api_key_here

Integration with Claude Desktop

To use this package with Claude Desktop:

1. Install the package globally:

   npm install -g scraperis-mcp
   

2. Add the following configuration to the claude_desktop_config.json file:

   {
     "mcpServers": {
       "scraperis_scraper": {
         "command": "scraperis-mcp",
         "args": [],
         "env": {
           "SCRAPERIS_API_KEY": "your-api-key-here",
           "DEBUG": "*"
         }
       }
     }
   }
   

3. Replace your-api-key-here with your actual Scraper.is API key.
4. Restart Claude Desktop to apply the changes.

Development and Testing with MCP Inspector

npx @modelcontextprotocol/inspector scraperis-mcp

Integration with AI Assistants

This package is designed for AI assistants that support the Model Context Protocol (MCP). After configuration, AI assistants can use the following tools:

scrape Tool

Scrape content from web pages.

Advanced Usage

The package can be integrated with various AI assistants supporting MCP. After proper configuration, AI assistants can use the scrape tool to extract content from web pages.

📚 Documentation

API Reference

Parameters

• url: The URL to scrape.
• output_format: The output format, supporting text, html, or json.
• proxy: Optional proxy server address.

Example Response

{
  "content": "...",
  "status": "success"
}

🔧 Technical Details

Development

Install Dependencies

npm install

Start the Service

npm start

🤝 Contribution

Contributions are welcome! Please check the contribution guidelines in the GitHub repository.

📄 License

MIT License

This document provides a complete guide for using and configuring the Scraper.is MCP integration package, suitable for developers and technology enthusiasts.