Rag MCP Server

🚀 RAG Architecture Overview

This project is based on the Retrieval-Augmented Generation (RAG) architecture. It mainly uses API Gateway and Lambda functions to handle requests, and combines OpenAI, OpenSearch Serverless, and S3 buckets to complete text retrieval and generation tasks. This document details the project's deployment process, functional modules, and solutions to common problems.

🚀 Quick Start

✨ Features

• API Gateway: Provides an HTTP API interface, listening on the /mcp path. Supports GET requests for service discovery and POST requests for task execution.
• Lambda Function: Parses MCP protocol requests, retrieves OpenAI keys from Secrets Manager, performs vector retrieval (based on OpenSearch Serverless) and content generation (calls the OpenAI API), and manages documents in the S3 bucket.
• Data Storage: Uses OpenSearch Serverless for efficient vector retrieval. Stores original text files in S3, supporting version control and lifecycle management.
• Security Mechanisms: IAM policies strictly control the permissions of Lambda functions. Secrets Manager ensures the secure transmission and storage of sensitive information. API Gateway integrates authentication functions to ensure that only authorized users can access the service.

📦 Installation

1. Environment Preparation

• Ensure that the AWS CLI tool is installed and the corresponding IAM permissions are configured.
• Install Python 3.8 or a higher version, and make sure that dependency libraries such as boto3, openai, and pytest are installed.

2. Code Structure

project/
├── src/
│   └── lambda_function.py          # Lambda function implementation logic
├── stack/
│   └── rag_mpc_stack.py           # CDK stack definition file
└── example.py                     # Example script demonstrating API call method

3. Dependency Management

• Add Python packages to pyproject.toml and run make deps for installation.
• Ensure that all dependency versions are compatible with the project.

4. Deployment Steps

# Install dependencies
pip install -r requirements.txt

# Build the project
make build

# Deploy to AWS
make deploy AWS_PROFILE=your-profile AWS_REGION=us-east-1

💻 Usage Examples

1. Environment Variable Configuration

export API_KEY=your-api-key
export API_URL=$(aws cloudformation describe-stacks --stack-name rag-stack | jq -r '.Stacks[0].Outputs[].OutputValue')

2. Service Discovery Request

import requests

response = requests.get(f"{API_URL}/mcp/health")
print(response.json())

3. Text Generation Task

payload = {
    "prompt": "请简述RAG技术的核心优势。",
    "temperature": 0.7,
    "max_tokens": 500
}

response = requests.post(f"{API_URL}/mcp/generate", json=payload, headers={'Authorization': f'Bearer {API_KEY}'})
print(response.json())

📚 Documentation

Development Notes

• Test Environment: Use make test to run unit tests, and simulate external dependencies (such as AWS, OpenAI, etc.) through the mock module.
• Log Monitoring: After deployment, you can use make logs to view the real-time log output of the Lambda function, which is convenient for debugging and performance optimization.
• Code Update: After modifying the business logic code in the src/ directory, you need to rebuild and deploy a new Lambda version.

Solutions to Common Problems

• Deployment Failure: Check whether the IAM permissions are correct. Ensure that the status of all resource stacks is CREATE_COMPLETE. View the event logs in the CloudFormation console to locate specific errors.
• Slow API Response: Optimize the OpenSearch indexing strategy. Increase the memory quota or timeout of the Lambda function. Use AWS performance monitoring tools (such as X-Ray) for analysis.
• Key Leakage: Regularly check the status of credentials in Secrets Manager. Configure an automatic rotation strategy to reduce the risk of manual operations. Enable the SSO (Single Sign-On) function to strengthen authentication.
• Scalability Issues: Configure Auto Scaling groups according to the load. Use the rate limiting function of API Gateway to control concurrent requests. Pre - allocate sufficient resources to avoid performance bottlenecks during peak periods.

Monitoring and Maintenance

• Log Analysis: Conduct in - depth analysis through the ELK Stack or log files stored in S3.
• Performance Optimization: Regularly evaluate the system load and adjust the architecture parameters.
• Version Control: Record the change log of each deployment for easy rollback and traceability.

The above document fully describes the deployment and management process of the RAG architecture project. For further assistance, please refer to the official documentation or contact the technical support team.