Querynest

🚀 QueryNest - MongoDB Multi-Instance Query Service

QueryNest is a MongoDB multi-instance query service based on the MCP (Model Context Protocol). It offers intelligent database structure discovery, semantic analysis, and natural language query generation capabilities.

📍 Project Information

• Version: v1.0.0
• Status: ✅ Production-ready
• License: MIT License
• Language: Python 3.12+
• Architecture: Asynchronous / Based on MCP Protocol

✨ Features

🔍 Intelligent Query

• Natural Language Query: Supports query requirements described in Chinese natural language.
• Native MongoDB Query: Supports standard MongoDB query syntax.
• Aggregation Pipeline: Supports complex data aggregation operations.
• Query Optimization: Automatically optimizes query performance.
• Query Cache: Intelligent caching to improve query speed.

🏢 Multi-Instance Management

• Instance Discovery: Automatically discovers and connects to multiple MongoDB instances.
• Load Balancing: Intelligently distributes query requests.
• Health Check: Monitors instance status in real-time.
• Failover: Automatically handles instance failures.
• Connection Pool Management: Optimizes database connection usage.

🛡️ Security Control

• Read-Only Permission: Ensures data security by only supporting read operations.
• Query Limitation: Limits query complexity and the amount of returned data.
• Data Masking: Automatically identifies and masks sensitive information.
• Access Control: Role-based access permission management.
• Security Audit: Records all query operations.

🧠 Intelligent Analysis

• Structure Discovery: Automatically analyzes database structure and field types.
• Semantic Understanding: Understands the business meaning of fields.
• Query Suggestion: Provides query optimization suggestions.
• Performance Analysis: Analyzes query performance and bottlenecks.
• Index Suggestion: Intelligently recommends index optimization solutions.

📊 Monitoring and Metrics

• Real-Time Monitoring: Monitors system performance and query metrics in real-time.
• Performance Analysis: Detailed query performance statistics.
• Error Tracking: Complete error recording and analysis.
• Health Check: Evaluates system health status.
• Metrics Export: Supports metrics export in multiple formats.

🔧 User Experience

• Error Handling: Friendly error prompts and suggestions.
• User Feedback: A complete feedback collection system.
• Help System: Built-in help documentation and FAQ.
• Configuration Validation: Automatically validates configuration files and the environment.

🔌 MCP Integration

• Standard Protocol: Fully compatible with MCP (Model Context Protocol).
• Rich Tools: Provides a complete set of query and analysis tools.
• Interactive: Supports conversational queries and exploration.
• Scalable: Easy to integrate into various AI applications.
• Feedback Tools: Built-in user feedback and help tools.

📦 Installation

Environment Requirements

• Python 3.8+
• MongoDB 4.0+
• Optional: Redis (for caching)

🚀 Quick Start

Quick Launch (Recommended)

Use uvx to quickly start the service:

# Install the uv tool (if not already installed)
pip install uv

# Start from the project directory (recommended)
cd /path/to/QueryNest
uvx --from . --no-cache querynest-mcp

# Or start from any location
uvx --from /path/to/QueryNest --no-cache querynest-mcp

Advantages of starting with uvx:

• Automatically handles dependencies.
• No need to pre-install packages in the environment.
• Uses an isolated execution environment.
• Automatically caches to accelerate subsequent launches.

Manual Installation

1. Clone the project

git clone https://github.com/niuzaishu/QueryNest.git
cd QueryNest

2. Install dependencies

cd QueryNest
pip install -r requirements.txt

3. Configure the service

# Copy the configuration template
cp config.example.yaml config.yaml

# Edit the configuration file (modify the MongoDB connection string according to the actual environment)
vim config.yaml  # Or use your preferred editor

4. Start the service

# Development mode (run directly)
python mcp_server.py --log-level DEBUG

# Production mode (using uvx, recommended)
uvx --from . --no-cache querynest-mcp

# Set the configuration file path (if needed)
export QUERYNEST_CONFIG_PATH=/path/to/config.yaml

Docker Deployment

# Build and start
docker-compose up -d

# View logs
docker-compose logs -f

# Stop the service
docker-compose down

⚙️ Configuration

🔌 MCP Client Configuration

After the service is started, you can configure the QueryNest service in an AI client that supports the MCP protocol to implement intelligent database query functionality.

1. Project Structure

QueryNest/
├── 📄 Configuration Files
│   ├── config.yaml              # Main configuration file
│   ├── config.example.yaml      # Configuration template
│   └── config.py               # Configuration management
├── 🚀 Core Services
│   ├── mcp_server.py           # MCP server entry point
│   ├── start.py               # Alternative startup script
│   └── database/              # Database connection and management
├── 🔧 MCP Toolset
│   └── mcp_tools/             # MCP protocol tool implementation
├── 🔍 Scanning and Analysis
│   └── scanner/               # Database scanning and semantic analysis
├── 🛠️ Utility Classes
│   └── utils/                 # Validation, error handling, workflow management
├── 🧪 Test Code
│   └── tests/                 # Unit tests and integration tests
├── 📚 Documentation
│   └── docs/                  # Complete project documentation
├── 📦 Deployment
│   └── deployment/            # Docker and service configuration
└── 📜 Scripts
    └── scripts/               # Database checking and testing tools

📖 For detailed structure instructions, please refer to

QueryNest is configured as a package that can be run via uvx. The project contains the following key files:

setup.py - Package configuration file:

setup(
    name="querynest",
    version="1.0.0",
    description="QueryNest MCP MongoDB query service",
    py_modules=["mcp_server", "config"],
    packages=["database", "scanner", "mcp_tools", "utils"],
    entry_points={
        "console_scripts": [
            "querynest-mcp=mcp_server:cli_main",
        ]
    },
)

Entry point configuration - The CLI entry point is defined in mcp_server.py:

def cli_main():
    """Command-line entry point"""
    # Automatically find the configuration file and set the environment
    # Supports starting from different directories
    asyncio.run(main())

if __name__ == "__main__":
    cli_main()

2. Local Running Steps

Step 1: Install the uv tool

If uv is not installed, you can install it as follows:

# Install using pip (recommended)
pip install uv

# Or use the official installation script (Linux/macOS)
curl -LsSf https://astral.sh/uv/install.sh | sh

# Windows PowerShell
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"

# Verify the installation
uvx --version

Step 2: Start the service

Run in the project root directory:

# Recommended way: Run from the project directory
cd /path/to/QueryNest
uvx --from . --no-cache querynest-mcp

# Or set an environment variable to specify the configuration file
export QUERYNEST_CONFIG_PATH=/path/to/QueryNest/config.yaml
uvx --from /path/to/QueryNest --no-cache querynest-mcp

Step 3: Verify service startup

After the service starts successfully, you should see log output similar to the following:

{"event": "Starting QueryNest MCP server initialization", "config_path": "/path/to/config.yaml"}
{"event": "Configuration loaded successfully", "instances_count": 2}
{"event": "MCP tools initialized successfully", "tools_count": 13}
{"event": "Starting stdio MCP server"}

3. MCP Client Integration

How uvx works:

uvx is a modern Python package execution tool that can:

• Automatically install packages from the current directory (.).
• Manage temporary virtual environments.
• Execute package entry point commands.

Key points for MCP client configuration:

For an AI client that supports the MCP protocol, here is an example configuration for QueryNest:

{
  "mcpServers": {
    "QueryNest": {
      "command": "uvx",
      "args": ["--from", "/path/to/QueryNest", "--no-cache", "querynest-mcp"],
      "cwd": "/path/to/QueryNest",
      "env": {
        "QUERYNEST_CONFIG_PATH": "/path/to/QueryNest/config.yaml",
        "QUERYNEST_LOG_LEVEL": "INFO"
      }
    }
  }
}

Windows configuration example:

{
  "mcpServers": {
    "QueryNest": {
      "command": "uvx",
      "args": ["--from", "C:\\path\\to\\QueryNest", "--no-cache", "querynest-mcp"],
      "cwd": "C:\\path\\to\\QueryNest",
      "env": {
        "QUERYNEST_CONFIG_PATH": "C:\\path\\to\\QueryNest\\config.yaml"
      }
    }
  }
}

Key configuration explanations:

• --from /path/to/QueryNest: Specifies the absolute path of the project.
• --no-cache: Ensures the use of the latest code.
• cwd: Sets the working directory to the project root directory.
• querynest-mcp: The entry point command defined in setup.py.

Advantages:

1. Clear project path: Using an absolute path ensures the correct project is found.
2. Automatic dependency management: uvx automatically handles all dependency packages.
3. Isolated environment: Each run is in an independent temporary environment.
4. Automatic configuration file discovery: The server automatically finds the configuration file.

4. Troubleshooting

Common issues and solutions:

Issue 1: The uvx command does not exist

# Solution: Install the uv tool
pip install uv

# Or use the official installation script
curl -LsSf https://astral.sh/uv/install.sh | sh  # Linux/macOS
# powershell -c "irm https://astral.sh/uv/install.ps1 | iex"  # Windows

# Verify the installation
uvx --version

Issue 2: The configuration file was not found

# Check if the configuration file exists
ls -la config.yaml

# Create a configuration file from the example
cp config.example.yaml config.yaml

# Set an environment variable
export QUERYNEST_CONFIG_PATH=/path/to/QueryNest/config.yaml

Issue 3: MCP service connection failed

• Check the format of the MCP client configuration file.
• Confirm that the project path is correct (use an absolute path).
• Verify that the MongoDB service is running.
• Check if the configuration file config.yaml exists.

Issue 4: MongoDB connection failed

# Check the MongoDB service status
python scripts/check_db.py

# Manually test the MongoDB connection
python -c "
from pymongo import MongoClient
client = MongoClient('mongodb://localhost:27017/')
print('MongoDB connection successful')
"

# Check if the MongoDB service is running
# Linux/macOS
sudo systemctl status mongod
# Windows
net start | findstr -i mongo

Verify successful configuration:

# Test local running
cd /path/to/QueryNest
uvx --from . --no-cache querynest-mcp --help

# Check the project structure
ls -la setup.py mcp_server.py config.yaml

# Verify the entry point
python -c "
from mcp_server import cli_main
print('Entry point OK')
"

# Test the complete startup process
uvx --from . --no-cache querynest-mcp --log-level INFO

MongoDB Instance Configuration

QueryNest supports flexible environment configuration. You can configure different types of instances according to your actual needs:

1. Traditional environment configuration (dev, test, uat, sit, staging, prod)
2. Business system configuration (crm-prod, order-system, user-center)
3. Regional cluster configuration (beijing, shanghai, guangzhou)
4. Custom environment configuration (any name)

mongodb:
  instances:
    prod-main:
      name: "Production Main Database"
      environment: "prod"
      connection_string: "mongodb://admin:password@localhost:27017/admin"
      database: "prod_database"
      description: "Main database for the production environment"
      status: "active"
      tags: ["production", "primary"]
    
    crm-prod:
      name: "CRM Production Database"
      environment: "crm-prod"
      connection_string: "mongodb://crm_user:${CRM_DB_PASSWORD}@crm-db.company.com:27017/admin"
      database: "crm_database"
      description: "Production database for the CRM system"
      status: "active"
      tags: ["crm", "production"]
    
    beijing-cluster:
      name: "Beijing Cluster"
      environment: "beijing"
      connection_string: "mongodb://readonly:${BEIJING_DB_PASSWORD}@beijing-mongo.company.com:27017/admin"
      database: "beijing_database"
      description: "MongoDB cluster in the Beijing region"
      status: "active"
      tags: ["beijing", "cluster"]

Security Configuration

security:
  permissions:
    allowed_operations:
      - "find"
      - "count"
      - "aggregate"
      - "distinct"
    forbidden_operations:
      - "insert"
      - "update"
      - "delete"
  limits:
    max_documents: 1000
    query_timeout: 30
  data_masking:
    enabled: true
    sensitive_field_patterns:
      - "password"
      - "email"
      - "phone"

Environment Variable Configuration

Supports independent environment variable management for multiple instances:

# .env file example
# Traditional environment passwords
PROD_DB_PASSWORD=your_prod_password
TEST_DB_PASSWORD=your_test_password
DEV_DB_PASSWORD=your_dev_password

# Business system passwords
CRM_DB_PASSWORD=your_crm_password
ORDER_DB_PASSWORD=your_order_password
USER_CENTER_DB_PASSWORD=your_user_center_password

# Regional cluster passwords
BEIJING_DB_PASSWORD=your_beijing_password
SHANGHAI_DB_PASSWORD=your_shanghai_password
GUANGZHOU_DB_PASSWORD=your_guangzhou_password

# Custom instance password
CUSTOM_INSTANCE_PASSWORD=your_custom_password

Port Configuration

• MCP service: By default, uses stdio communication and does not require a port; the port can be configured in HTTP mode (default is 8000).
• MongoDB: 27017 (inside the Docker container).
• Prometheus: 9090 (monitoring panel).
• Application monitoring: 8000 (optional, for health checks).

Port explanations:

• stdio mode: Communicates through standard input and output and does not require a network port.
• HTTP mode: Configures the port through the environment variable QUERYNEST_MCP_PORT.

Metadata Configuration

metadata:
  instance_id: "dev-local"  # Can be any environment identifier
  database_name: "querynest_metadata"
  collections:
    instances: "instances"
    databases: "databases"
    collections: "collections"
    fields: "fields"
    query_history: "query_history"

🛠️ Using MCP Tools

1. Instance Discovery (discover_instances)

Discover and list all available MongoDB instances.

{
  "name": "discover_instances",
  "arguments": {
    "include_health": true,
    "include_stats": true
  }
}

2. Database Discovery (discover_databases)

List all databases in a specified instance.

{
  "name": "discover_databases",
  "arguments": {
    "instance_id": "prod-main",
    "include_collections": true,
    "exclude_system": true
  }
}

3. Collection Analysis (analyze_collection)

Analyze the structure and field information of a specified collection.

{
  "name": "analyze_collection",
  "arguments": {
    "instance_id": "prod-main",
    "database_name": "ecommerce",
    "collection_name": "users",
    "include_semantics": true,
    "include_examples": true,
    "rescan": false
  }
}

4. Semantic Management (manage_semantics)

Manage the business semantic information of fields.

{
  "name": "manage_semantics",
  "arguments": {
    "action": "batch_analyze",
    "instance_id": "prod-main",
    "database_name": "ecommerce",
    "collection_name": "users"
  }
}

5. Query Generation (generate_query)

Generate a MongoDB query based on a natural language description.

{
  "name": "generate_query",
  "arguments": {
    "instance_id": "prod-main",
    "database_name": "ecommerce",
    "collection_name": "orders",
    "query_description": "Find orders created today, sorted by amount in descending order",
    "query_type": "auto",
    "limit": 50
  }
}

6. Query Confirmation (confirm_query)

Execute the generated query and return the results.

{
  "name": "confirm_query",
  "arguments": {
    "instance_id": "prod-main",
    "database_name": "ecommerce",
    "collection_name": "orders",
    "query_type": "find",
    "mongodb_query": {
      "filter": {"created_at": {"$gte": "2024-01-01T00:00:00Z"}},
      "sort": {"amount": -1},
      "limit": 50
    },
    "explain": true
  }
}

💻 Usage Examples

Scenario 1: E-commerce Data Analysis

1. Discover instances and databases

User: "Help me check which database instances are available"
Assistant: Use the `discover_instances` tool

2. Analyze the user collection

User: "Analyze the structure of the user table in the e-commerce database"
Assistant: Use the `analyze_collection` tool to analyze the `users` collection

3. Natural language query

User: "Find active users who registered in the last week, sorted by registration time"
Assistant: Use `generate_query` to generate a query, then use `confirm_query` to execute it

Scenario 2: Log Data Query

1. Semantic analysis

User: "Help me understand the meaning of each field in the log collection"
Assistant: Use `manage_semantics` for batch semantic analysis

2. Complex aggregation query

User: "Count the number of error logs per hour, grouped by time"
Assistant: Generate and execute an aggregation query

🔧 Development Guide

Project Structure

QueryNest/
├── src/
│   ├── __init__.py
│   ├── config.py              # Configuration management
│   ├── mcp_server.py          # Main MCP server file
│   ├── database/              # Database module
│   │   ├── __init__.py
│   │   ├── connection_manager.py
│   │   ├── metadata_manager.py
│   │   └── query_engine.py
│   ├── scanner/               # Scanning module
│   │   ├── __init__.py
│   │   ├── structure_scanner.py
│   │   └── semantic_analyzer.py
│   └── mcp_tools/             # MCP tools
│       ├── __init__.py
│       ├── instance_discovery.py
│       ├── database_discovery.py
│       ├── collection_analysis.py
│       ├── semantic_management.py
│       ├── query_generation.py
│       └── query_confirmation.py
├── config.yaml                # Configuration file
├── requirements.txt           # Dependency list
└── README.md                  # Project documentation

Adding New Tools

1. Create a tool class

class NewTool:
    def get_tool_definition(self) -> Tool:
        # Define the tool interface
        pass
    
    async def execute(self, arguments: Dict[str, Any]) -> List[TextContent]:
        # Implement the tool logic
        pass

2. Register the tool

# Register in mcp_server.py
new_tool = NewTool(...)
self.tools["new_tool"] = new_tool

Extending Semantic Analysis

1. Add semantic rules

# Add in semantic_analyzer.py
self.semantic_patterns.update({
    "custom_field": {
        "patterns": [r"custom_.*"],
        "meaning": "Custom field",
        "confidence": 0.8
    }
})

2. Custom analysis logic

def analyze_custom_semantics(self, field_info):
    # Implement custom semantic analysis logic
    pass

🚨 Important Notes

Security Considerations

1. Permission Control

   • Ensure only read operations are allowed.
   • Configure appropriate query limitations.
   • Enable data masking.

2. Network Security

   • Use SSL/TLS connections.
   • Configure firewall rules.
   • Update passwords regularly.

3. Data Protection

   • Avoid recording sensitive information.
   • Regularly clean up query history.
   • Monitor abnormal access.

Performance Optimization

1. Connection Management

   • Reasonably configure the connection pool size.
   • Enable connection reuse.
   • Monitor connection health status.

2. Query Optimization

   • Use appropriate indexes.
   • Limit the number of query results.
   • Avoid complex aggregation operations.

3. Caching Strategy

   • Enable metadata caching.
   • Cache commonly used query results.
   • Regularly clean up expired caches.

📝 Changelog

v1.0.0 (2024-01-01)

• Initial version released.
• Supports multi-instance MongoDB connections.
• Implements basic structure scanning and semantic analysis.
• Provides a complete set of MCP tools.
• Supports natural language query generation.

🧪 Testing

Run All Tests

python -m pytest tests/ -v

Run Unit Tests

# Test the connection manager
python -m pytest tests/unit/test_connection_manager.py -v

# Test the query engine
python -m pytest tests/unit/test_query_engine.py -v

# Test the metadata manager
python -m pytest tests/unit/test_metadata_manager.py -v

# Test the database scanner
python -m pytest tests/unit/test_database_scanner.py -v

# Test the MCP tools
python -m pytest tests/unit/test_mcp_tools.py -v

Test Coverage

pip install pytest-cov
python -m pytest tests/ --cov=src --cov-report=html

Environment Validation

# Validate the startup environment
python -c "
from utils.startup_validator import validate_startup_environment
print(validate_startup_environment())
"

📚 Documentation

Core Documentation

• Technical Architecture - Detailed technical architecture description.
• Configuration Guide - Configuration file instructions.
• Environment Variables - Environment variable configuration instructions.

Deployment Documentation

• Deployment Script - Automatic deployment tool.
• Docker Deployment - Containerized deployment.
• Service Configuration - System service configuration.

Development Documentation

• Unit Tests - Complete unit test suite.
• Error Handling - Error handling mechanism.
• Monitoring Metrics - Performance monitoring system.
• Configuration Validation - Configuration validation tool.

User Guide

• Quick Start - Quick deployment and usage.
• Features - Detailed feature description.
• Troubleshooting - Solutions to common problems.

📚 More Resources

• Docker Deployment Guide
• Contribution Guide
• Changelog
• MCP Protocol Documentation
• MongoDB Official Documentation
• Python Asynchronous Programming Guide

🔧 Troubleshooting

Common Issues

3. Failed to connect to MongoDB

# Check the MongoDB service status
sudo systemctl status mongod

# Test the network connection
telnet <mongodb_host> <mongodb_port>

# Verify authentication information
mongo --host <host> --port <port> -u <username> -p

4. Configuration file error

# Validate the configuration file
python -c "
from utils.config_validator import ConfigValidator
validator = ConfigValidator()
print(validator.validate_config_file('config.yaml'))
"

Environment Variable Configuration

QueryNest supports the following environment variables:

Linux/macOS example:

# Set the configuration file path
export QUERYNEST_CONFIG_PATH=/path/to/QueryNest/config.yaml

# Set the log level
export QUERYNEST_LOG_LEVEL=DEBUG

# MCP transport mode (currently only supports stdio)
export QUERYNEST_MCP_TRANSPORT=stdio

Windows example:

# CMD
set QUERYNEST_CONFIG_PATH=C:\path\to\QueryNest\config.yaml
set QUERYNEST_LOG_LEVEL=DEBUG

# PowerShell
$env:QUERYNEST_CONFIG_PATH="C:\path\to\QueryNest\config.yaml"
$env:QUERYNEST_LOG_LEVEL="DEBUG"

5. Dependency package issues

cd /path/to/QueryNest

# Reinstall dependencies
pip install -r requirements.txt --force-reinstall

# Check the Python version
python --version

# Check the installation status of key packages
pip list | grep -E "(mcp|pymongo|motor)"

6. Permission and path issues

# Check if the files exist
ls -la config.yaml mcp_server.py

# Check directory permissions
ls -ld . logs/

# Fix permissions (if needed)
chmod 755 .
chmod 644 config.yaml
chmod +x mcp_server.py

# Create the log directory (if it doesn't exist)
mkdir -p logs/

Log Analysis

View detailed logs:

# View application logs
tail -f logs/querynest.log

# View error logs
tail -f logs/error.log

# View system logs
journalctl -u querynest -f

Performance Tuning

# View system resource usage
top
htop

# View MongoDB performance
mongotop
mongostat

# View network connections
netstat -an | grep :27017

🤝 Contribution Guide

We welcome contributions in all forms! Please check CONTRIBUTING.md to learn how to participate in project development.

Quick Contribution Guide

1. Fork the project.
2. Create a feature branch (git checkout -b feature/AmazingFeature).
3. Add test cases.
4. Run tests to ensure they pass (python -m pytest tests/ -v).
5. Commit changes (git commit -m 'Add some AmazingFeature').
6. Push to the branch (git push origin feature/AmazingFeature).
7. Open a Pull Request.

Development Environment

# Clone the project
git clone <repository_url>
cd QueryNest

# Create a virtual environment
python -m venv venv
source venv/bin/activate  # Linux/Mac
venv\Scripts\activate     # Windows

# Install development dependencies
pip install -r requirements.txt
pip install pytest pytest-cov black flake8

# Run code formatting
black src/ tests/

# Run code checks
flake8 src/ tests/

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

Thanks to all contributors and the following open-source projects:

• PyMongo - MongoDB Python driver
• PyYAML - YAML parser
• psutil - System monitoring library
• pytest - Testing framework

📞 Support

If you encounter problems or have suggestions, please:

1. Check the FAQ.
2. Search Issues.
3. Create a new Issue.
4. Check GitHub Discussions.
5. Contact the maintainers.

🙏 Acknowledgments

Thanks to the following projects and contributors:

• MCP (Model Context Protocol)
• PyMongo
• Motor
• Pydantic
• StructLog

QueryNest - Making MongoDB queries simple and intelligent 🚀