Codebasemcp

🚀 Python Codebase Analysis RAG System

This system analyzes Python code by parsing the Abstract Syntax Tree (AST) in Python files. It stores the extracted information (functions, classes, calls, variables, etc.) in the Weaviate vector database and provides tools to query and understand the codebase through the Model Context Protocol (MCP) server. The system leverages Google's Gemini model to generate embedded representations and natural language descriptions/answers.

✨ Features

• Code Scanning: Parse Python files to identify code elements (functions, classes, imports, calls, assignments) and their relationships. Extract:

  • Basic information: Name, type, file path, line number, code snippet, docstring.
  • Function/method details: Parameters, return type, signature, decorators.
  • Scope information: Parent scope (class/function) UUID, readable ID (e.g., file:type,name), location information (start and end line/column).
  • Metadata: Hash value of the code file for version control, scan timestamp to record the analysis time.

• Vector Database Integration: Use Weaviate's multi - tenant feature to store code files (CodeFile) and code elements (CodeElement). Each codebase uses a user - defined codebase_name as the tenant ID, and there is an independent non - multi - tenant collection CodebaseRegistry to record codebase metadata.

• RAG Implementation: Use Retrieval - Augmented Generation (RAG) to answer questions about the codebase. Find relevant code elements through semantic search and then use a large language model (LLM) for answer synthesis.

• MCP Server: Set up a server through the FastMCP framework, manage codebase registration (CodebaseRegistry), maintain the active codebase context (ACTIVE_CODEBASE_NAME), integrate file monitoring logic (including automatic start/stop), manage codebase dependencies, and expose analysis functions as MCP tools.

• Visualization Support: Generate MermaidJS call graphs through visualization.py for a visual representation of the code structure based on the stored relationship data.

📦 Installation

Install Dependencies

pip install weaviate-client python-multipart fastapi uvicorn

Initialize Configuration

Create a .env file:

WEAVIATE_URL=http://localhost:5000
WEAVIATE_API_KEY=your_api_key
GENERATE_LLM_DESCRIPTIONS=true  # Optional, default is false

Start the Server

uvicorn mcp_server:app --reload

📚 Documentation

The main modules of the system include:

• code_scanner.py: Find Python files, parse them using AST, extract structural elements (functions, classes, imports, calls, variables, etc.), and prepare data for Weaviate.
• weaviate_client.py: Manage the connection with Weaviate, define data schemas (CodeFile, CodeElement, CodebaseRegistry), and provide functions for batch uploading, querying, updating, and deleting data.
• rag.py: Implement Retrieval - Augmented Generation (RAG) to answer questions about the codebase. Combine semantic search and LLM for answer generation.
• mcp_server.py: Set up a FastMCP server, manage codebase registration (CodebaseRegistry), maintain the active codebase context (ACTIVE_CODEBASE_NAME), integrate file monitoring logic (automatic start/stop), manage codebase dependencies, and expose analysis functions as MCP tools.
• visualization.py: Generate MermaidJS call graphs based on the stored relationships for code structure visualization.

The system uses Weaviate's multi - tenant feature to store CodeFile and CodeElement, where the tenant ID is the user - defined codebase_name. An independent non - multi - tenant collection CodebaseRegistry records codebase metadata (name, directory, status, summary, monitor status, dependencies). The global variable ACTIVE_CODEBASE_NAME in the MCP server determines the primary codebase tenant for queries. Tools (find_element, ask_question) optionally search within the active codebase and its declared dependencies.

Background LLM processing is used to generate semantic descriptions and embeddings of code elements. This feature can be optionally enabled/disabled through the .env file.