MCP Telemetry

🚀 MCP Telemetry

An MCP (Model Context Protocol) server for telemetry in chat systems, using Weights & Biases Weave.

🚀 Quick Start

MCP Telemetry provides a simple interface for recording and tracking conversations between users and Large Language Models (LLMs). It exposes telemetry tools using the Model Context Protocol, which can be used to trace and analyze conversations.

✨ Features

• Initiate session tracking with custom identifiers.
• Record comprehensive conversation data, including:
  • User inputs
  • LLM responses
  • LLM actions
  • Tool calls and their results
• Seamlessly integrate with Weights & Biases Weave for visualization and analysis.
• Monitor conversation flow in real-time.
• Export and share conversation analytics.

📦 Installation

First, obtain an API Key from Weights & Biases.

This server can be installed by adding the following JSON to your Claude desktop configuration:

{
  "mcpServers": {
    "MCP Telemetry": {
      "command": "uv", -- Check with 'which uv' to find the location of uv you can use
      "args": [
        "run",
        "--with",
        "mcp[cli]",
        "--with",
        "weave",
        "mcp",
        "run",
        "~/mcp-telemetry/server.py"
      ],
      "env": {
        "WANDB_API_KEY": "..." -- Get one from wandb.com
      }
    }
  }
}

💻 Usage Examples

Basic Usage

Once installed, the MCP Telemetry server will automatically start running when you launch Claude. It will begin collecting telemetry data for all conversations. You can view the telemetry data in the Weights & Biases dashboard.

1. Start a conversation with Claude.
2. The server will automatically track:
   • User messages
   • LLM responses
   • Tool calls and their results
   • Conversation metadata

Advanced Usage

Start a Tracking Session

Prompt Claude to track the conversation. Example: Record this conversation with MCP Telemetry, topic: Cats

View Telemetry Data

1. Log in to your Weights & Biases account.
2. Navigate to your project.
3. You will see various visualizations, including:
   • Conversation flow
   • Tool usage patterns
   • Response times
   • Error rates

📚 Documentation

Configuration

The server can be configured via environment variables:

• WANDB_API_KEY - Your Weights & Biases API key (required)

📄 License

This project is released under the MIT License. For more details, see the LICENSE file.

🤝 Contribution

Contributions are welcome! Feel free to submit pull requests.