Scraper.is

🚀 Scraper.is MCP Integration

This is an integration of the Model Context Protocol (MCP) on Scraper.is, a powerful web scraping tool for AI assistants. This package enables AI assistants to perform web scraping via the MCP protocol, allowing them to access the latest internet information.

🚀 Quick Start

This package allows AI assistants to perform web scraping through the MCP protocol, enabling them to access the latest internet information.

✨ Features

• 🌐 Web Scraping: Extract content from any website.
• 📸 Screenshots: Capture visual representations of web pages.
• 📄 Multiple Formats: Obtain content in Markdown, HTML, or JSON format.
• 🔄 Real-time Progress Updates: Receive real-time progress reports during scraping operations.
• 🔌 MCP Integration: Seamlessly integrate with MCP-enabled AI assistants.

📦 Installation

npm install -g scraperis-mcp

Or use Yarn:

yarn global add scraperis-mcp

Prerequisites

You need a Scraper.is API Key to use this package.

Get Your API Key

1. Sign up or log in at scraper.is.
2. Visit the API Keys section in the dashboard: https://www.scraper.is/dashboard/apikeys.
3. Create a new API Key or copy an existing one.
4. Add it to the .env file in your project, for example:

SCRAPER_API_KEY=your_api_key_here

Environment Configuration

Using Claude Desktop

Add the following to your config.json file:

{
  "scraperis": {
    "apiKey": "your_scraper_is_api_key"
  }
}

Make sure to replace your_scraper_is_api_key with your own API Key.

Running the MCP Inspector

Run the MCP inspector for development and testing using the following command:

npx @model-context/protocol-inspect

💻 Usage Examples

Basic Usage

const result = await scraperis.scrape('https://example.com', {
  depth: 2,
  fields: ['title', 'description']
});

📚 Documentation

API Reference

scraperis.scrape()

Parameters

• url: The URL address to scrape.
• options (optional): Configuration options, including:
  • depth: The maximum scraping depth, default is 2.
  • fields: A list of fields to extract.

Return Value

• result.data: The scraped data.
• result.error: If there is an error, it will contain the error information.

🔧 Technical Details

Development

1. Clone the repository:

   git clone https://github.com/your-repository.git
   

2. Install dependencies:

   npm install
   

3. Build the project:

   npm run build
   

🤝 Contribution

Welcome to submit Pull Requests to improve this project!

📄 License

MIT License

Acknowledgments

Thanks to the contributions of the following projects:

• Model Context Protocol
• Scraper.is