MCP Jina Supabase Rag

🚀 MCP Jina Supabase RAG

A lean, focused MCP server for crawling documentation websites and indexing them to Supabase for RAG (Retrieval-Augmented Generation).

🚀 Quick Start

To quickly get started with the MCP Jina Supabase RAG, follow the installation steps and usage instructions provided below.

✨ Features

• Smart URL Discovery: Tries sitemap.xml first, falls back to Crawl4AI recursive discovery.
• Hybrid Content Extraction: Uses Jina AI for fast content extraction, Crawl4AI as fallback.
• Multi-Project Support: Index multiple documentation sites to separate Supabase projects.
• Efficient Chunking: Intelligent text chunking with configurable size and overlap.
• Vector Embeddings: OpenAI embeddings stored in Supabase pgvector.

🔧 Technical Details

Architecture

┌─────────────────────────────────────────────────────────────┐
│                    MCP Server Tools                         │
├─────────────────────────────────────────────────────────────┤
│  1. crawl_and_index(url_pattern, project_name)             │
│  2. list_projects()                                         │
│  3. search_documents(query, project_name, limit)           │
└─────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌─────────────────────────────────────────────────────────────┐
│                   Discovery Layer                           │
├─────────────────────────────────────────────────────────────┤
│  • Try sitemap.xml (fast)                                   │
│  • Try common doc patterns                                  │
│  • Crawl4AI recursive discovery (fallback)                  │
└─────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌─────────────────────────────────────────────────────────────┐
│                  Extraction Layer                           │
├─────────────────────────────────────────────────────────────┤
│  • Jina AI Reader API (primary, fast)                       │
│  • Crawl4AI (fallback for complex pages)                    │
└─────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌─────────────────────────────────────────────────────────────┐
│              Chunking & Embedding Layer                     │
├─────────────────────────────────────────────────────────────┤
│  • Smart text chunking                                      │
│  • OpenAI embeddings (text-embedding-3-small)               │
└─────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌─────────────────────────────────────────────────────────────┐
│                   Supabase Storage                          │
├─────────────────────────────────────────────────────────────┤
│  • pgvector for similarity search                           │
│  • Project isolation via source column                      │
└─────────────────────────────────────────────────────────────┘

📦 Installation

Prerequisites

• Python 3.12+
• Supabase account
• OpenAI API key
• Jina AI API key (optional, recommended)

Setup

1. Clone the repository:

git clone https://github.com/yourusername/mcp-jina-supabase-rag.git
cd mcp-jina-supabase-rag

2. Install dependencies:

# Using uv (recommended)
uv venv
source .venv/bin/activate  # or .venv\Scripts\activate on Windows
uv pip install -e .

# Or using pip
pip install -e .

3. Set up Supabase database:

# Run the SQL in supabase_schema.sql in your Supabase SQL Editor

4. Configure environment:

cp .env.example .env
# Edit .env with your credentials

💻 Usage Examples

Basic Usage

Running the MCP Server

# SSE transport (recommended for remote connections)
python src/main.py

# The server will start on http://localhost:8052/sse

Configure MCP Client

Claude Code

claude mcp add --transport sse jina-supabase http://localhost:8052/sse

Cursor / Claude Desktop

{
  "mcpServers": {
    "jina-supabase": {
      "transport": "sse",
      "url": "http://localhost:8052/sse"
    }
  }
}

Slash Command

Create /home/marty/.claude/commands/jina.md:

---
allowed-tools: mcp__jina-supabase
argument-hint: <url_pattern> <project_name>
description: Crawl documentation and index to Supabase RAG
---

# Index Documentation to Supabase

Use the jina-supabase MCP server to crawl and index documentation.

Arguments:
- $1: URL pattern (e.g., https://docs.example.com/*)
- $2: Project name for isolation

Example:
/jina https://docs.anthropic.com/claude/* anthropic-docs

Advanced Usage

Tools

crawl_and_index

Crawl a documentation site and index to Supabase. Parameters:

• url_pattern (string): URL or pattern to crawl
• project_name (string): Project identifier for isolation
• discovery_method (string, optional): auto, sitemap, or crawl
• extraction_method (string, optional): auto, jina, or crawl4ai

Example:

await crawl_and_index(
    url_pattern="https://docs.supabase.com/docs/*",
    project_name="supabase-docs",
    discovery_method="auto",
    extraction_method="jina"
)

list_projects

List all indexed projects. Returns: List of project names with document counts

search_documents

Search indexed documents using vector similarity. Parameters:

• query (string): Search query
• project_name (string, optional): Filter by project
• limit (int, optional): Max results (default: 5)

Example:

results = await search_documents(
    query="How do I set up authentication?",
    project_name="supabase-docs",
    limit=10
)

📚 Documentation

Configuration

See .env.example for all configuration options.

Discovery Methods

• auto: Try sitemap first, fallback to crawl.
• sitemap: Only use sitemap.xml (fast, fails if no sitemap).
• crawl: Only use Crawl4AI recursive discovery (slow, comprehensive).

Extraction Methods

• auto: Use Jina for bulk extraction (>10 URLs), Crawl4AI otherwise.
• jina: Use Jina AI Reader API (fast, requires API key).
• crawl4ai: Use Crawl4AI browser automation (slow, no API key needed).

Development

# Install dev dependencies
uv pip install -e ".[dev]"

# Run tests
pytest

# Format code
black src/

# Lint
ruff check src/

Differences from mcp-crawl4ai-rag

📄 License

This project is licensed under the MIT license.

Contributing

Contributions welcome! Please open an issue first to discuss changes.