Rag Anything MCP

🚀 RAG Anything MCP Server

An MCP (Model Context Protocol) server that provides comprehensive RAG (Retrieval-Augmented Generation) capabilities for processing and querying directories of documents using the raganything library with full multimodal support.

🚀 Quick Start

To start using the RAG Anything MCP Server, follow these steps:

1. Environment Variables

export OPENAI_API_KEY="your-openai-api-key-here"

2. Install Dependencies

uv sync

3. Run the MCP Server

python main.py

✨ Features

• End-to-End Document Processing: Complete document parsing with multimodal content extraction
• Multimodal RAG: Support for images, tables, equations, and text processing
• Batch Processing: Process entire directories with multiple file types
• Advanced Querying: Both pure text and multimodal-enhanced queries
• Multiple Query Modes: hybrid, local, global, naive, mix, and bypass modes
• Vision Processing: Advanced image analysis using GPT-4V
• Persistent Storage: RAG instances maintained per directory for efficient querying

📦 Available Tools

process_directory

Process all files in a directory for comprehensive RAG indexing with multimodal support.

Required Parameters:

• directory_path: Path to the directory containing files to process
• api_key: OpenAI API key for LLM and embedding functions

Optional Parameters:

• working_dir: Custom working directory for RAG storage
• base_url: OpenAI API base URL (for custom endpoints)
• file_extensions: List of file extensions to process (default: ['.pdf', '.docx', '.pptx', '.txt', '.md'])
• recursive: Process subdirectories (default: True)
• enable_image_processing: Enable image analysis (default: True)
• enable_table_processing: Enable table extraction (default: True)
• enable_equation_processing: Enable equation processing (default: True)
• max_workers: Concurrent processing workers (default: 4)

process_single_document

Process a single document with full multimodal analysis.

Required Parameters:

• file_path: Path to the document to process
• api_key: OpenAI API key

Optional Parameters:

• working_dir: Custom working directory for RAG storage
• base_url: OpenAI API base URL
• output_dir: Output directory for parsed content
• parse_method: Document parsing method (default: "auto")
• enable_image_processing: Enable image analysis (default: True)
• enable_table_processing: Enable table extraction (default: True)
• enable_equation_processing: Enable equation processing (default: True)

query_directory

Pure text query against processed documents using LightRAG.

Parameters:

• directory_path: Path to the processed directory
• query: The question to ask about the documents
• mode: Query mode - "hybrid", "local", "global", "naive", "mix", or "bypass" (default: "hybrid")

query_with_multimodal_content

Enhanced query with additional multimodal content (tables, equations, etc.).

Parameters:

• directory_path: Path to the processed directory
• query: The question to ask
• multimodal_content: List of multimodal content dictionaries
• mode: Query mode (default: "hybrid")

Example multimodal_content:

[
  {
    "type": "table",
    "table_data": "Method,Accuracy\\nRAGAnything,95.2%\\nBaseline,87.3%",
    "table_caption": "Performance comparison"
  },
  {
    "type": "equation",
    "latex": "P(d|q) = \\frac{P(q|d) \\cdot P(d)}{P(q)}",
    "equation_caption": "Document relevance probability"
  }
]

list_processed_directories

List all directories that have been processed and are available for querying.

get_rag_info

Get detailed information about the RAG configuration and status for a directory.

💻 Usage Examples

Basic Usage

# Basic Directory Processing
process_directory(
  directory_path="/path/to/documents",
  api_key="your-openai-api-key"
)

# Pure Text Query
query_directory(
  directory_path="/path/to/documents",
  query="What are the main findings in these research papers?",
  mode="hybrid"
)

# Single Document Processing
process_single_document(
  file_path="/path/to/important_paper.pdf",
  api_key="your-openai-api-key",
  enable_image_processing=true
)

Advanced Usage

# Advanced Directory Processing
process_directory(
  directory_path="/path/to/research_papers",
  api_key="your-openai-api-key",
  file_extensions=[".pdf", ".docx"],
  enable_image_processing=true,
  enable_table_processing=true,
  max_workers=6
)

# Multimodal Query with Table Data
query_with_multimodal_content(
  directory_path="/path/to/documents",
  query="Compare these results with the document findings",
  multimodal_content=[{
    "type": "table",
    "table_data": "Method,Accuracy,Speed\\nRAGAnything,95.2%,120ms\\nBaseline,87.3%,180ms",
    "table_caption": "Performance comparison"
  }],
  mode="hybrid"
)

📚 Query Modes Explained

• hybrid: Combines local and global search (recommended for most use cases)
• local: Focuses on local context and entity relationships
• global: Provides broader, document-level insights and summaries
• naive: Simple keyword-based search without graph reasoning
• mix: Combines multiple approaches for comprehensive results
• bypass: Direct access without RAG processing

📚 Multimodal Content Types

The server supports processing and querying with:

• Images: Automatic caption generation and visual analysis
• Tables: Structure extraction and content analysis
• Equations: LaTeX parsing and mathematical reasoning
• Charts/Graphs: Visual data interpretation
• Mixed Content: Combined analysis of multiple content types

📚 API Configuration

The server uses OpenAI's APIs by default:

You can customize the base_url parameter to use:

• Azure OpenAI
• OpenAI-compatible APIs
• Custom model endpoints

📚 File Support

Supported file formats include:

• PDF documents
• Microsoft Word (.docx)
• PowerPoint presentations (.pptx)
• Text files (.txt)
• Markdown files (.md)
• And more via the raganything library

📚 Performance Notes

⚠️ Important Note

• Concurrent Processing: Use max_workers to control parallel document processing
• Memory Usage: Large documents with many images may require significant memory
• API Costs: Vision processing (GPT-4o) is more expensive than text processing
• Storage: Processed data is stored locally for efficient re-querying

💡 Usage Tip

Consider the performance notes when using the server to optimize resource usage and cost.