Msparihar MCP Server Firecrawl

🚀 Firecrawl MCP Server

A Model Context Protocol (MCP) server for web crawling, content search, website scraping, and data extraction, based on the Firecrawl API.

✨ Features

• Web Crawling: Extract content from any web page with customizable options

  • Mobile device simulation
  • Ad and popup blocking
  • Content filtering
  • Structured data extraction
  • Multiple output formats

• Content Search: Intelligent search functionality

  • Multilingual support
  • Location-based results
  • Customizable result limits
  • Structured output format

• Website Scraping: Advanced web scraping capabilities

  • Depth control
  • Path filtering
  • Rate limiting
  • Progress tracking
  • Sitemap integration

• Website Mapping: Generate website structure diagrams

  • Subdomain support
  • Search filtering
  • Link analysis
  • Visual hierarchy analysis

• Data Extraction: Extract structured data from multiple URLs

  • Validation schemes
  • Batch processing
  • Enhanced website search
  • Custom extraction prompts

📦 Installation

# Global installation
npm install -g @modelcontextprotocol/mcp-server-firecrawl

# Local installation in the project
npm install @modelcontextprotocol/mcp-server-firecrawl

🚀 Quick Start

1. Obtain your Firecrawl API key from the Developer Portal.

2. Set the API key in your environment variables:

   export FIRECRAWL_API_KEY=your_api_key_here
   

3. Start with the following examples:

   • Basic Scraper
   • Crawl and Map

📚 Documentation

Installation of Dependencies

npm install @modelcontextprotocol/mcp-server-firecrawl

Create an MCP Server Instance

import { FireCrawler } from '@modelcontextprotocol/mcp-server-firecrawl';

const crawler = new FireCrawler({
  apiKey: 'your_api_key_here',
});

// Start the server
crawler.listen(3000, () => {
  console.log('Server is running on http://localhost:3000');
});

💻 Usage Examples

Basic Usage

import { FireCrawler } from '@modelcontextprotocol/mcp-server-firecrawl';

async function basicScrape() {
  const crawler = new FireCrawler({
    apiKey: 'your_api_key_here',
  });

  const result = await crawler.scrape('https://example.com');
  console.log(result);
}

basicScrape();

Advanced Usage

import { FireCrawler } from '@modelcontextprotocol/mcp-server-firecrawl';

async function crawlAndMap() {
  const crawler = new FireCrawler({
    apiKey: 'your_api_key_here',
  });

  // Crawl the website
  const crawlResult = await crawler.crawl('https://example.com');
  console.log('Crawl result:', crawlResult);

  // Generate the website map
  const mapResult = await crawler.createMap('example', ['https://example.com']);
  console.log('Website map creation result:', mapResult);
}

crawlAndMap();

🔧 Technical Details

Error Handling

The server implements robust error handling:

• Rate limiting with exponential backoff
• Automatic retries
• Detailed error messages
• Debug logging

Security

• API key protection
• Request validation
• Whitelisting
• Rate limiting
• Secure error messages

Development

# Install dependencies
npm install

# Build
npm run build

# Run tests
npm test

# Start in development mode
npm run dev

More Examples

Check out the examples directory for more usage examples:

• Basic Scraper: scrape.ts
• Crawl and Map: crawl-and-map.ts

Contribution

Refer to CONTRIBUTING.md for contribution guidelines.

📄 License

This project is under the MIT License. See LICENSE for details.