Atla MCP Server U1z

🚀 Atla MCP Server

An MCP (Machine Learning Platform) server implementation that provides standardized interfaces, enabling LLMs to interact with the Atla SDK and utilize our latest evaluation model Selene 1.

✨ Features

• Evaluate a single response using Selene 1.
• Run Selene 1 evaluations in batches.
• List available evaluation metrics, create new ones, or retrieve them by name.

📦 Installation

1. Fork the repository and clone it locally in a directory. This will define your path/to/atla-mcp-server.

git clone https://github.com/yourusername/atla-mcp-server.git
cd atla-mcp-server
pwd
# /path/to/atla-mcp-server

2. If you haven't installed uv yet, install it at /path/to/uv. You can find installation instructions at this link. For example, on MacOS:

brew install uv
which uv
# /path/to/uv

3. Install the requirements into a virtual environment at /path/to/atla-mcp-server.

uv venv
uv sync

4. Add your ATLA_API_KEY to the environment variables. You can find it here. For example, on MacOS:

export ATLA_API_KEY=<your-atla-api-key> >> ~/.zshrc
source ~/.zshrc
echo $ATLA_API_KEY
# <your-atla-api-key>

• If you're building with the OpenAI proxy, you also need to provide an OPENAI_API_KEY environment variable.

💻 Usage Examples

Basic Usage

Use with OpenAI Agents SDK

The Atla MCP server can be used in conjunction with the OpenAI Agents SDK as follows:

from agents import Agent
from agents.mcp import MCPServerStdio
import os
atla_api_key = os.environ.get("ATLA_API_KEY", "<your_atla_api_key>") # You can also manually set your ATLA_API_KEY here

async with MCPServerStdio(
        params={
            "command": "python",
            "args": ["/path/to/atla-mcp-server/atla-mcp-server.py"],
            "env": {"ATLA_API_KEY": atla_api_key}
        }
    ) as server:
        # Add your code here

Advanced Usage

Use with Claude Desktop

In the config.yaml file, add the following to the agents section:

  atla_mcp_server:
    type: mcp_server
    params:
      command: uv run --env ATLA_API_KEY=<your-atla-api-key> /path/to/atla-mcp-server ./atla-mcp-server.py

Use with Cursor

In the config.yaml file, add the following to the agents section:

  atla_mcp_server:
    type: mcp_server
    params:
      command: uv run --env ATLA_API_KEY=<your-atla-api-key> /path/to/atla-mcp-server ./atla-mcp-server.py