Jcrawl4ai MCP Server

🚀 jcrawl Project Introduction

jcrawl is a Java-based crawler project that utilizes dependencies such as Spring AI MCP Server and Hutool. It provides two main interfaces: crawl and task.

🚀 Quick Start

Prerequisites

• Download the JAR file. The download link can be found in the official documentation.

Installation Steps

1. Install the project using Maven:

mvn clean install

2. Run the JAR package:

java -jar target/jcrawl-1.0.0.jar

✨ Features

• Core Crawling: Responsible for page fetching and parsing.
• Task Management: Supports batch crawling and task scheduling.
• Data Storage: Offers multiple data persistence solutions.

📦 Installation

The project's configuration file is located in application.properties:

cawl4ai.base-url=
cawl4ai.api-token=

Dependencies

• Spring AI MCP Server
• Spring Boot
• Hutool

💻 Usage Examples

Example Configuration for MCP Server

{
  "mcpServers": {
    "jcrawl": {
      "autoApprove": ["crawl", "task"],
      "disabled": false,
      "timeout": 60,
      "command": "java",
      "args": [
        "-jar",
        "/path/to/jcrawl.jar"
      ],
      "transportType": "stdio"
    }
  }
}

📚 Documentation

Interface Explanation

Crawl4aiApi Interface

crawl Method

• Function: Crawl the specified URLs.
• Parameters:
  • urls: A list of target website addresses.
  • strategy: Crawling strategy (e.g., breadth-first, depth-first).
  • max_depth: Maximum crawling depth.
  • output_format: Output format (e.g., JSON, XML).
• Return: A JSON string of the crawling results.

task Method

• Function: Get the crawling results of a specified task.
• Parameters:
  • taskId: The unique identifier of the task.
• Return: A JSON string of the crawling results.

📄 Contact

If you need feedback or technical support, please contact Ken Ye.

⚠️ Important Note

The download link for the JAR file can be found in the official documentation.