MCP Rss Crawler

🚀 MCP-RSS-Crawler Server

A server based on the Message Chain Protocol (MCP) for crawling RSS feeds and sharing them with Large Language Models (LLMs).

🚀 Quick Start

Prerequisites

• Bun environment
• Firecrawl API key
• Claude Desktop or other MCP clients

Setup as an MCP Server

1. Clone this repository.
2. Create a claude_desktop_config.json file based on claude_desktop_config.json.example with your configuration:

{
  "mcpServers": {
    "rss-crawler": {
      "command": "/path/to/bun",
      "args": ["run", "/path/to/mcp-rss-crawler/apps/mcp-server/src/mcp-cli.ts"],
      "cwd": "/path/to/mcp-rss-crawler",
      "env": {
        "PORT": "5556",
        "DB_DIR": "/path/to/mcp-rss-crawler",
        "FIRECRAWL_API_KEY": "fc-<YOUR_FIRECRAWL_API_KEY>"
      }
    }
  }
}

3. Install dependencies:

bun install

4. Start Claude Desktop.

✨ Features

• Crawl and cache RSS feeds into an SQLite database.
• Implement the MCP protocol for seamless integration with LLMs.
• Support filtering RSS feeds by category, source, or keyword.
• Provide comprehensive API endpoints for feed management:
  • Add, update, and delete RSS feeds.
• Support fetching article content from Firecrawl.

💻 Usage Examples

MCP Protocol

This server implements the Message Chain Protocol (MCP), allowing LLMs to access your latest RSS feeds. The MCP endpoint accepts POST requests containing a messages array and returns a response with the latest feed items.

Basic Usage

{
  "messages": [
    {
      "role": "user",
      "content": "What's new in my RSS feeds?"
    }
  ]
}

Advanced Usage

{
  "messages": [
    {
      "role": "assistant",
      "content": "Here are the latest articles from your RSS feeds:",
      "name": "rss-mcp"
    },
    {
      "role": "tool",
      "content": "[{\"title\":\"Article Title\",\"summary\":\"Article summary...\",\"published\":\"2025-03-16T04:30:00.000Z\",\"origin\":\"Source Name\",\"link\":\"https://example.com/article\"}]",
      "name": "rss-feeds"
    }
  ]
}

🔧 Technical Details

Configuration Options

The server can be configured via environment variables or a .env file:

Troubleshooting

• When encountering connection issues, check your network settings and firewall configuration.
• Logs can be viewed in the console for problem diagnosis.
• For more detailed logging, set DEBUG=mcp-rss:* in the environment variables.