Blacksmith MCP

🚀 Blacksmith MCP

An MCP server that connects Claude to your Blacksmith CI data. Query workflow runs, analyze test failures, detect flaky tests, and monitor usage—all through natural conversation.

🚀 Quick Start

Zero-config if you're logged into Blacksmith in Chrome:

# Add to Claude Code
claude mcp add blacksmith -- npx blacksmith-mcp

# Set your org (run once)
export BLACKSMITH_ORG="your-org-name"

The MCP automatically extracts your session from Chrome cookies. No manual token copying needed.

✨ Features

Debugging CI failures usually involves clicking through dashboards, copying run IDs, and piecing together information across multiple pages. With this MCP, you can simply ask questions like:

• "Why did the last CI run fail?"
• "Which tests are flaky this week?"
• "Compare test failures between main and my PR"
• "What's using the most cache storage?"

Claude handles the API calls and provides actionable insights.

📦 Installation

Option 1: Claude Code CLI

claude mcp add blacksmith -- npx blacksmith-mcp

Option 2: Project Configuration

Add to your .mcp.json:

{
  "mcpServers": {
    "blacksmith": {
      "type": "stdio",
      "command": "npx",
      "args": ["blacksmith-mcp"],
      "env": {
        "BLACKSMITH_ORG": "your-org-name"
      }
    }
  }
}

Option 3: Global Install

npm install -g blacksmith-mcp

📚 Documentation

Configuration

Authentication

Automatic (recommended): Log into app.blacksmith.sh in Chrome. The MCP extracts your session cookie automatically.

Manual: Set BLACKSMITH_SESSION_COOKIE environment variable with your session cookie value.

Environment Variables

Available Tools

Workflow Runs

Test Analytics

Usage & Billing

Organization

💻 Usage Examples

Debug a Failed Run

You: Why did the last CI run fail?

Claude: Uses list_runs(status="failure", limit=1) then get_failed_tests

The last run failed with 34 test failures. The main issues are:

• Connection timeout (12 tests) - Tests in api/ are timing out connecting to the test database
• Undefined property (8 tests) - user.settings.theme is undefined in the new dark mode code
• Assertion failures (14 tests) - Snapshot mismatches in UI components

Find Flaky Tests

You: Which tests are flaky?

Claude: Uses get_flaky_tests(days=7, min_runs=5)

Found 3 flaky tests:

Test	Flake Rate	Pattern
should handle concurrent requests	40% (4/10 runs)	Fails on CI, passes locally
renders loading state	20% (2/10 runs)	Race condition with setTimeout
validates auth token	30% (3/10 runs)	Intermittent network timeout

Compare Branches

You: Did my PR introduce new test failures?

Claude: Uses compare_test_runs with PR run vs base branch

Your PR introduced 2 new failures:

• UserProfile.test.tsx: "Cannot read property 'avatar' of undefined"
• api/users.test.ts: Expected 200, got 404

5 previously failing tests now pass (nice!).

🔧 Technical Details

Development

# Install dependencies
pnpm install

# Build
pnpm build

# Development mode (watch)
pnpm dev

# Test with MCP Inspector
npx @modelcontextprotocol/inspector node dist/index.js

Troubleshooting

Session Expired

If you see SESSION_EXPIRED, your Blacksmith session has expired. Simply log back into app.blacksmith.sh in Chrome and retry.

Cookie Extraction Failed

The automatic cookie extraction requires:

• macOS with Chrome installed
• Being logged into Blacksmith in Chrome
• Chrome not running with a locked profile

If it fails, set BLACKSMITH_SESSION_COOKIE manually.

No Organization Set

Run list_orgs to see available organizations, then set BLACKSMITH_ORG to your org name.

API Notes

This MCP uses Blacksmith's internal web API, which is undocumented. The API was reverse-engineered from the Blacksmith web app and may change without notice.

📄 License

MIT

Contributing

Contributions welcome! Please open an issue first to discuss proposed changes.