Peekaboo

🚀 Peekaboo MCP: Screenshots so fast they're paranormal.

A macOS utility that enables AI assistants to see screen content through rapid screenshots and advanced analysis

🚀 Quick Start

Peekaboo MCP is a ghostly macOS utility that allows AI assistants to "see" your screen. It captures screenshots and uses AI to analyze them, helping you with bug hunting, design reviews, and more.

Prerequisites

• macOS 14.0+ (Sonoma or later)
• Node.js 20.0+

Installation

Summon Peekaboo into your Agent realm by adding the following configuration to your Claude Desktop:

{
  "mcpServers": {
    "peekaboo": {
      "command": "npx",
      "args": [
        "-y",
        "@steipete/peekaboo-mcp@beta"
      ],
      "env": {
        "PEEKABOO_AI_PROVIDERS": "ollama/llava:latest"
      }
    }
  }
}

Then, restart Claude Desktop.

Verification

Verify that Peekaboo has successfully crossed over:

# Commune with the Swift spirit directly
./peekaboo --help

# Check the spectral server's pulse
./peekaboo list server_status --json-output

# Capture a soul (requires permission wards)
./peekaboo image --mode screen --format png

# Open the portal for testing
peekaboo-mcp

✨ Features

👁️‍🗨️ AI Vision for Assistants

Peekaboo gives your AI assistants the power of actual vision, making it easier to communicate UI issues and get accurate responses.

📸 Fast Screenshots

Capture screenshots of any window, app, or screen instantly.

🤖 AI Analysis

Analyze screenshots using AI models like Ollama and OpenAI, getting insights and answers to your questions.

🎨 Design Reviews

Let AI provide feedback on your designs, helping you improve your CSS and UI.

📊 Data Analysis

Ask AI to analyze charts and graphs in your screenshots, extracting valuable information.

🖼️ UI Testing

Verify your app's appearance across different screens and devices, avoiding the "works on my machine" problem.

📱 Multi-Screen Support

Capture screenshots from multiple screens simultaneously.

🤖 Automation

Integrate Peekaboo with your automation workflows, letting AI see what you see and fix what you break.

📦 Installation

Ritual Requirements

• macOS 14.0+ (Sonoma or later)
• Node.js 20.0+

🕯️ Quick Summoning Ritual

Add the following configuration to your Claude Desktop:

{
  "mcpServers": {
    "peekaboo": {
      "command": "npx",
      "args": [
        "-y",
        "@steipete/peekaboo-mcp@beta"
      ],
      "env": {
        "PEEKABOO_AI_PROVIDERS": "ollama/llava:latest"
      }
    }
  }
}

Then, restart Claude Desktop.

🔮 Mystical Configuration

Enchantment Variables

Configure Peekaboo using the following environment variables:

{
  "PEEKABOO_AI_PROVIDERS": "ollama/llava:latest,openai/gpt-4o",
  "PEEKABOO_LOG_LEVEL": "debug",
  "PEEKABOO_LOG_FILE": "~/Library/Logs/peekaboo-mcp-debug.log",
  "PEEKABOO_DEFAULT_SAVE_PATH": "~/Pictures/PeekabooCaptures",
  "PEEKABOO_CONSOLE_LOGGING": "true",
  "PEEKABOO_CLI_PATH": "/opt/custom/peekaboo"
}

🎭 Available Enchantments

🧙 AI Spirit Guide Configuration (PEEKABOO_AI_PROVIDERS In-Depth)

The PEEKABOO_AI_PROVIDERS environment variable is used to configure the AI providers and models for the analyze and image tools. It should be a comma-separated string in the format provider_name/model_identifier.

• provider_name: Currently supported values are ollama and openai. Support for anthropic is planned.
• model_identifier: The specific model to use for that provider (e.g., llava:latest, gpt-4o).

If the provider_config argument in the analyze or image tools is set to "auto" (the default for analyze, and an option for image), Peekaboo will try the providers from PEEKABOO_AI_PROVIDERS in the order they are listed, checking for necessary API keys or service availability.

🦙 Summoning Ollama - The Local Vision Oracle

Ollama provides a powerful local AI that can analyze your screenshots without sending data to the cloud.

📦 Installing Ollama

macOS (via Homebrew):

brew install ollama

Visit ollama.ai and download the macOS app.

Start the Ollama daemon:

ollama serve

The daemon will run at http://localhost:11434 by default.

🎭 Downloading Vision Models

For powerful machines, LLaVA is the recommended model:

# Download the latest LLaVA model (recommended for best quality)
ollama pull llava:latest

# Alternative LLaVA versions
ollama pull llava:7b-v1.6
ollama pull llava:13b-v1.6  # Larger, more capable
ollama pull llava:34b-v1.6  # Largest, most powerful (requires significant RAM)

For less beefy machines, Qwen2-VL provides excellent performance with lower resource requirements:

# Download Qwen2-VL 7B model (great balance of quality and performance)
ollama pull qwen2-vl:7b

Model Size Guide:

• qwen2-vl:7b - ~4GB download, ~6GB RAM required
• llava:7b - ~4.5GB download, ~8GB RAM required
• llava:13b - ~8GB download, ~16GB RAM required
• llava:34b - ~20GB download, ~40GB RAM required

🔮 Configuring Peekaboo with Ollama

Add Ollama to your Claude Desktop configuration:

{
  "mcpServers": {
    "peekaboo": {
      "command": "npx",
      "args": [
        "-y",
        "@steipete/peekaboo-mcp@beta"
      ],
      "env": {
        "PEEKABOO_AI_PROVIDERS": "ollama/llava:latest"
      }
    }
  }
}

For less powerful machines (using Qwen2-VL):

{
  "mcpServers": {
    "peekaboo": {
      "command": "npx",
      "args": [
        "-y",
        "@steipete/peekaboo-mcp@beta"
      ],
      "env": {
        "PEEKABOO_AI_PROVIDERS": "ollama/qwen2-vl:7b"
      }
    }
  }
}

Multiple AI Providers (Ollama + OpenAI):

{
  "env": {
    "PEEKABOO_AI_PROVIDERS": "ollama/llava:latest,openai/gpt-4o",
    "OPENAI_API_KEY": "your-api-key-here"
  }
}

🧪 Testing Ollama Integration

Verify Ollama is running and accessible:

# Check Ollama is running
curl http://localhost:11434/api/tags

# Test with Peekaboo directly (image capture only)
./peekaboo image --app Finder --path ~/Desktop/finder.png

# Test with Peekaboo directly (image capture and analysis - requires PEEKABOO_AI_PROVIDERS to be set for the environment Peekaboo runs in)
# Note: The CLI itself doesn't take a question, this is an MCP server feature.
# The MCP server would call: ./peekaboo image ... (to get the image)
# And then internally call the AI provider if a question was part of the MCP 'image' tool input.

🕰️ Granting Mystical Permissions

Peekaboo requires certain macOS permissions to function properly:

1. 👁️ The All-Seeing Eye Permission

1. Open System Preferences → Security & Privacy → Privacy
2. Select Screen Recording from the left sidebar
3. Click the lock icon and enter your password
4. Click + and add your terminal application or MCP client
5. Restart the application

Known vessels that can channel Peekaboo:

• Terminal.app: /Applications/Utilities/Terminal.app
• Claude Desktop: /Applications/Claude.app
• VS Code: /Applications/Visual Studio Code.app

2. 🪄 Window Whisperer Permission (Optional)

To control windows during capture, add your terminal/MCP client application to the Accessibility permission in System Preferences → Security & Privacy → Privacy.

🕯️ Séance Verification

Verify that Peekaboo has successfully crossed over:

# Commune with the Swift spirit directly
./peekaboo --help

# Check the spectral server's pulse
./peekaboo list server_status --json-output

# Capture a soul (requires permission wards)
./peekaboo image --mode screen --format png

# Open the portal for testing
peekaboo-mcp

Expected ghostly whispers:

{
  "success": true,
  "data": {
    "swift_cli_available": true,
    "permissions": {
      "screen_recording": true
    },
    "system_info": {
      "macos_version": "14.0"
    }
  }
}

💻 Usage Examples

🖼️ image: Capture Ghostly Visions

Capture the entire main screen and save it:

{
  "tool_name": "image",
  "arguments": {
    "mode": "screen",
    "path": "~/Desktop/myscreen.png",
    "format": "png"
  }
}

Capture the active window of Finder and return its data as Base64:

{
  "tool_name": "image",
  "arguments": {
    "app": "Finder",
    "mode": "window",
    "return_data": true,
    "format": "jpg"
  }
}

Capture all windows of "Google Chrome" and bring it to the foreground first:

{
  "tool_name": "image",
  "arguments": {
    "app": "Google Chrome",
    "mode": "multi",
    "capture_focus": "foreground",
    "path": "~/Desktop/ChromeWindows/"
  }
}

👁️ list: Reveal Hidden Spirits

List all running applications:

{
  "tool_name": "list",
  "arguments": {
    "item_type": "running_applications"
  }
}

List all windows of the "Preview" app, including their bounds and IDs:

{
  "tool_name": "list",
  "arguments": {
    "item_type": "application_windows",
    "app": "Preview",
    "include_window_details": ["bounds", "ids"]
  }
}

Get the server's current status:

{
  "tool_name": "list",
  "arguments": {
    "item_type": "server_status"
  }
}

🔮 analyze: Divine the Captured Essence

Ask a question about an image using the auto-configured AI provider:

{
  "tool_name": "analyze",
  "arguments": {
    "image_path": "~/Desktop/myscreen.png",
    "question": "What is the main color visible in the top-left quadrant?"
  }
}

Force using Ollama with a specific model for analysis:

{
  "tool_name": "analyze",
  "arguments": {
    "image_path": "~/Desktop/some_diagram.jpg",
    "question": "Explain this diagram.",
    "provider_config": {
      "type": "ollama",
      "model": "llava:13b-v1.6"
    }
  }
}

📚 Documentation

🎭 Spectral Powers

Once summoned, Peekaboo grants you three supernatural abilities:

🖼️ image - Soul Capture

Captures macOS screen content and optionally analyzes it.

Parameters:

• app_target (string, optional): Specifies the capture target.
• path (string, optional): Base absolute path for saving the captured image(s).
• question (string, optional): If provided, the captured image will be analyzed.
• format (string, optional, default: "png"): Specifies the output image format or data return type.
• capture_focus (string, optional, default: "background"): Controls window focus behavior during capture.

Behavior with question (AI Analysis):

• If a question is provided, the tool will capture the image and send it to an AI model for analysis.
• The analysis result is returned as analysis_text in the response.

Output Structure (Simplified):

• content: Can contain ImageContentItem and/or TextContentItem.
• saved_files: Array of objects detailing saved files.
• analysis_text: Text from AI (if question was asked).
• model_used: AI model identifier (if question was asked).

👻 list - Spirit Detection

Lists system items like running applications, windows of a specific app, or server status.

Parameters:

• item_type: "running_applications" | "application_windows" | "server_status"
• app: Application identifier (required for application_windows)

🔮 analyze - Vision Divination

Analyzes a pre-existing image file using a configured AI model.

Parameters:

• image_path: Absolute path to image file
• question: Question/prompt for AI analysis

🌙 Supernatural Abilities

🖼️ Ethereal Vision Capture

• Multi-realm vision: Captures each display separately
• Soul targeting: App/window divination with matching
• Essence preservation: Supports PNG, JPEG, WebP, HEIF
• Mystical naming: Temporal runes and descriptive names
• Ward detection: Automatic permission verification

👻 Spirit Management

• Spirit census: Complete digital ghost registry
• Portal detection: Window scrying with metadata
• Spectral matching: Divine apps by partial essence, ID, or number
• Life force monitoring: Active/slumbering status, portal counts

🧿 Oracle Integration

• Oracle agnostic: Supports Ollama and OpenAI, with more to come
• Image analysis: Natural language querying of captured content
• Configurable: Environment-based provider selection

🏩 Haunted Architecture

Peekaboo/
├── src/                      # Node.js MCP Server (TypeScript)
│   ├── index.ts             # Main MCP server entry point
│   ├── tools/               # Individual tool implementations
│   │   ├── image.ts         # Screen capture tool
│   │   ├── analyze.ts       # AI analysis tool  
│   │   └── list.ts          # Application/window listing
│   ├── utils/               # Utility modules
│   │   ├── peekaboo-cli.ts   # Swift CLI integration
│   │   ├── ai-providers.ts  # AI provider management
│   │   └── server-status.ts # Server status utilities
│   └── types/               # Shared type definitions
├── peekaboo-cli/            # Native Swift CLI
│   └── Sources/peekaboo/    # Swift source files
│       ├── main.swift       # CLI entry point
│       ├── ImageCommand.swift    # Image capture implementation
│       ├── ListCommand.swift     # Application listing
│       ├── Models.swift          # Data structures
│       ├── ApplicationFinder.swift   # App discovery logic
│       ├── WindowManager.swift      # Window management
│       ├── PermissionsChecker.swift # macOS permissions
│       └── JSONOutput.swift        # JSON response formatting
├── package.json             # Node.js dependencies
├── tsconfig.json           # TypeScript configuration
└── README.md               # This file

🔬 Arcane Knowledge

📜 Ancient Runes (JSON Output)

The Swift CLI outputs structured JSON when called with --json-output:

{
  "success": true,
  "data": {
    "applications": [
      {
        "app_name": "Safari",
        "bundle_id": "com.apple.Safari", 
        "pid": 1234,
        "is_active": true,
        "window_count": 2
      }
    ]
  },
  "debug_logs": ["Found 50 applications"]
}

🌌 Portal Integration

The Node.js server translates between MCP's JSON-RPC protocol and the Swift CLI's JSON output, providing schema validation, error handling, logging, and type safety.

🚪 Permission Wards

Peekaboo respects macOS security by checking screen recording permissions, degrading gracefully when permissions are missing, and providing clear error messages.

🔧 Technical Details

🕸️ Known Curses

• FileHandle warning: Non-critical Swift warning about TextOutputStream conformance
• AI Provider Config: Requires PEEKABOO_AI_PROVIDERS environment variable for analysis features

🌌 Future Hauntings

• [ ] OCR Integration: Built-in text extraction from screenshots
• [ ] Video Capture: Screen recording capabilities
• [ ] Annotation Tools: Drawing/markup on captured images
• [ ] Cloud Storage: Direct upload to cloud providers
• [ ] Hotkey Support: System-wide keyboard shortcuts

📄 License

This project is licensed under the MIT License. See the LICENSE file for details.

🧛 Join the Coven

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

📜 Available Tools (via MCP Server)

Peekaboo exposes its powers through the following tools when run as an MCP server:

• image: Captures macOS screen content. Can target entire screens, specific application windows, or all windows of an app. Supports various formats and capture modes. Can optionally analyze the captured image immediately.
• analyze: Analyzes a pre-existing image file using a configured AI model.
• list: Lists system items like running applications, windows of a specific app, or server status.

For full input/output schemas, see docs/spec.md.

🎃 Peekaboo awaits your command! Happy haunting! 👻