MCP Server Firecrawl

🚀 Firecrawl MCP Server

A Model Context Protocol (MCP) server for web crawling, content search, website scraping, and data extraction, utilizing the Firecrawl API.

✨ Features

• Web Scraping: Extract content from any web page with customizable options

  • Mobile device emulation
  • Ad and pop-up blocking
  • Content filtering
  • Structured data extraction
  • Multiple output formats

• Content Search: Intelligent search functionality

  • Multilingual support
  • Location-based results
  • Customizable result limits
  • Structured output format

• Website Crawling: Advanced web crawling capabilities

  • Depth control
  • Path filtering
  • Rate limiting
  • Progress tracking
  • Sitemap integration

• Website Mapping: Generate website structure diagrams

  • Subdomain support
  • Search filtering
  • Link analysis
  • Visual hierarchy

• Data Extraction: Extract structured data from multiple URLs

  • Schema validation
  • Batch processing
  • Enhanced website search
  • Custom extraction prompts

📦 Installation

# Global installation
npm install -g @modelcontextprotocol/mcp-server-firecrawl

# Local installation in the project
npm install @modelcontextprotocol/mcp-server-firecrawl

🚀 Quick Start

1. Get an API Key: Visit the Firecrawl official website and register to obtain your API key.
2. Set up environment variables:

   export FIRECRAWL_API_KEY='your_api_key_here'
   

3. Install and run the server:

   npm install @modelcontextprotocol/mcp-server-firecrawl
   node index.js
   

📚 Documentation

Integration

Claude Desktop App Integration

Add the following content to the config.json file to integrate Firecrawl:

{
  "firecrawl": {
    "apiKey": "your_api_key_here",
    "baseUrl": "https://api.firecrawl.com"
  }
}

VSCode Extension Integration

Add the following code to the extension's configuration file:

import { fireCrawlSettings } from '@modelcontextprotocol/mcp-server-firecrawl';
fireCrawlSettings.apiKey = 'your_api_key_here';

💻 Usage Examples

Basic Usage

const fireCrawl = require('@modelcontextprotocol/mcp-server-firecrawl');

async function basicScrape() {
  try {
    const result = await fireCrawl.scrape('https://example.com');
    console.log(result);
  } catch (error) {
    console.error(error.message);
  }
}

basicScrape();

Advanced Usage

const fireCrawl = require('@modelcontextprotocol/mcp-server-firecrawl');

async function crawlAndMap() {
  try {
    const crawlResult = await fireCrawl.crawl('https://example.com');
    const mapResult = await fireCrawl.map(crawlResult.urls);
    console.log(mapResult);
  } catch (error) {
    console.error(error.message);
  }
}

crawlAndMap();

Error Handling

The server implements robust error handling:

• Rate limiting with exponential backoff
• Automatic retry
• Detailed error messages
• Debug logging

Security

• API key protection
• Request validation
• Allow list
• Rate limiting
• Secure error messages

Development

# Install dependencies
npm install

# Build
npm run build

# Run tests
npm test

# Start in development mode
npm run dev

More Examples

Check out more usage examples in the examples directory:

• Basic scraping: scrape.ts
• Crawling and mapping: crawl-and-map.ts

Contribution

Please refer to CONTRIBUTING.md for contribution guidelines.

📄 License

MIT License - Please check LICENSE for detailed information.