Firecrawl 8mz

🚀 Firecrawl MCP Server

FireCrawl MCP (Management Control Panel) Server is an integrated solution that combines web crawling, logging, and large language model (LLM) generation. It offers features through a command - line interface, supporting intelligent crawler operations, in - depth research, generation of standardized files (such as llms.txt), credit monitoring, and error handling.

🚀 Quick Start

Configuration

Environment Variables

Configure the server using the following environment variables:

Example Configuration

# Set environment variables
export FIRECRAWL_API_KEY="your_api_key_here"

Function Modules

Crawler Framework (firecrawl_crawler)

• Tool Name: firecrawl_crawler
• Description: Used for deep web crawling starting from a specified URL.
• Example Usage

{
  "name": "firecrawl_crawler",
  "arguments": {
    "url": "https://example.com",
    "maxDepth": 3,
    "outputFormat": "json"
  }
}

• Parameters
  • url (string, required): The starting URL for crawling.
  • maxDepth (number, optional): The maximum depth for crawling, defaulting to 2.
  • outputFormat (string, optional): The output format, with optional values of "json" or "xml".
• Example Response

{
  "content": [
    {
      "type": "text",
      "text": "Successfully crawled URL: https://example.com"
    }
  ],
  "isError": false
}

Logging System

• Description: Provides comprehensive logging capabilities, including operation status, performance metrics, credit usage, rate limits, and error conditions.
• Example Log Messages

[INFO] FireCrawl MCP Server has been successfully initialized
[INFO] Crawling URL: https://example.com
[WARNING] Credit usage has reached the warning threshold
[ERROR] Rate limit exceeded, retrying in 2 seconds...

Error Handling

• Description: Offers powerful error - handling features, including automatic retries, rate - limit handling, detailed error messages, credit - usage warnings, and network resilience.
• Example Error Response

{
  "content": [
    {
      "type": "text",
      "text": "The requested parameters are invalid"
    }
  ],
  "isError": true,
  "errorDetails": "Incorrect parameter format"
}

Development Guide

Install Dependencies

Use the following command to install the required dependencies:

pip install -r requirements.txt

Run the Server

Before starting the server, ensure that the FIRECRAWL_API_KEY environment variable is set. Then run the following command:

python server.py

Contribution Guide

Submit Code

Please follow these steps to submit code:

1. Create a feature branch.
2. Commit your changes.
3. Submit a pull request to the main branch.

Follow the Guidelines

• Follow the PEP8 code - style guide.
• Ensure that unit tests cover all new features.
• Check the code quality before submission.

Acknowledgments

Thanks to the following contributors:

• [Contributor Name] - Initial development
• [Contributor Name] - Feature enhancement

📄 License

MIT License

Copyright (c) Year Author Name

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

1. The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
2. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.