Mcpdocserver

🚀 MCP Server Project Documentation

This project provides a server for managing and searching documentation, with built - in crawlers and useful MCP tools.

🚀 Quick Start

📦 Installation

1. Clone the repository to your local machine:

   git clone https://github.com/your-username/mcp-server.git
   cd mcp-server
   

2. Install the dependencies:

   npm install
   

3. Start the server:

   node server.js
   

📁 Project Structure

mcp-server/
├── server.js          # Main server file
├── docs/             # Directory for storing documents
│   └── doc.json      # Example document file
├── crawl/            # Modules related to the crawler
│   ├── crawler.js    # Main logic of the crawler
│   └── parsers/      # Data parsing modules
├── tools/            # Implementations of MCP tools
│   ├── search_docs.js  # Tool for searching documents
│   └── get_doc_detail.js  # Tool for getting document details
└── test/             # Test cases
    └── test_server.js     # Test cases for the server

📚 Documentation

📄 Document Structure

The document files should be in JSON format, containing the following fields:

{
  "source": {
    "name": "taro",
    "url": "https://docs.taro.zone/docs"
  },
  "lastUpdated": "2024-03-27T12:00:00Z",
  "pages": {
    "https://docs.taro.zone/docs/components-desc": {
      "title": "Component Library Description | Taro Documentation",
      "content": "Page content..."
    },
    // More pages...
  }
}

📈 Loading Process

1. When the server starts, it will automatically detect and load JSON files in the docs directory.
2. If no documents are found in the project directory, it will try to load from the current working directory.
3. The page ID uses the URL as the key by default, and there is no need to specify an additional url field.
4. All source names will be automatically converted to lowercase to ensure consistency.

🕷️ Crawler Functionality

The system has a built - in crawler that can fetch content from official documentation sites of various frameworks and convert it into a locally usable document format. The crawler features include:

1. Multi - site Support: Supports documentation websites of any framework and library, fully configurable.
2. Selective Crawling: You can configure inclusion and exclusion patterns to precisely control the content to be crawled.
3. Intelligent Content Extraction: Automatically identifies the title, body content, and structure of the document page.
4. Multi - thread Crawling: Supports high - concurrency crawling to improve efficiency.
5. Automatic Conversion: Converts the crawled content into a standard document JSON format.
6. Fault - Tolerant Mechanism: Provides timeout handling and retry mechanisms to enhance stability.

💻 MCP Tools

The server provides the following MCP tools:

1. search_docs - Search Documents

• Parameters:
  • query: Search keyword (string, required)
  • source: Document source name (string, optional)
  • limit: Maximum number of results (number, optional, default is 10)
• Special Function:
  • When query is "reload", it will trigger a reload of the documents.

2. get_doc_detail - Get Document Details

• Parameters:
  • id: Document ID (string, required)
  • source: Document source name (string, optional)

💻 Usage Examples

⚙️ Basic Usage

// Example of searching documents
{
  "command": "search_docs",
  "params": {
    "query": "Component Development Guide",
    "source": "taro"
  }
}

// Example of getting document details
{
  "command": "get_doc_detail",
  "params": {
    "id": "https://docs.taro.zone/docs/components-desc"
  }
}

⚙️ Configuration in Cursor

⚙️ Configuration Steps

1. After starting the server, get its IP address and port number.
2. Configure the remote server connection in Cursor:

   {
     "name": "mcp-server",
     "type": "http",
     "url": "http://localhost:3000"
   }
   

3. Use MCP tools for document management and search.

🧪 Testing

🧪 Testing Steps

1. Start the server:

   node server.js
   

2. Run the test cases:

   npm test
   

🧪 Example Test Code

test('search_docs should return results', async () => {
  const response = await fetch('/api/search_docs?query=test');
  expect(response.status).toBe(200);
});

test('get_doc_detail should return details', async () => {
  const response = await fetch('/api/get_doc_detail?id=document-id');
  expect(response.status).toBe(200);
});

❓ Frequently Asked Questions

1. How to handle anti - crawling mechanisms of crawlers?

• Use request intervals and random User - Agents to simulate real user behavior.
• Handle captchas and dynamically loaded content.

2. How to improve search efficiency?

• Use full - text indexing technology (such as Elasticsearch).
• Optimize query logic and result sorting algorithms.

🤝 Project Contribution

Welcome to fork the project and submit PRs!

• Submit an issue to report problems or propose suggestions.
• Contribute code to improve features or fix bugs.