MCP Server Client Computer Use Ai Sdk

🚀 Computer Use AI SDK

• We've built an MCP server to control computers.
• You've heard of OpenAI's operators and Claude's computer use feature. Now, an open - source alternative is here: the Computer Use SDK from screenpipe.
• It runs natively on macOS—no virtual machine hassle and no restrictions. You can freely combine it with any application or website.
• There's no pixel - level hassle—it relies on underlying desktop rendering elements, which is much faster and more reliable than pixel - based vision models.
• Now you can start building your own agents from our simple Hello World template, using our MCP server and client.
• Our MCP server offers some out - of - the - box tools:
  • Launch applications
  • Read content
  • Click
  • Input text
  • Press keys

These will be the computational primitives that allow AI to control your computer and complete tasks for you. What will you build? Check out our https://screenpi.pe right away.

✨ Features

Agent Sending Messages

https://github.com/user-attachments/assets/f8687500-9a8c-4a96-81b6-77562feff093

Getting the Latest WhatsApp Messages

Opening the Arc Browser

🚀 Quick Start

Clone the Repository

git clone https://github.com/m13v/computer-use-ai-sdk.git
cd MCP-server-client-computer-use-ai-sdk

Install Dependencies

# Install Rust (if not already installed)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# Install Node.js and npm (if not already installed)
# Visit https://nodejs.org/ or use nvm

Run the Back - end Server

# Run the backend server
cd mcp-server-os-level
cargo run --bin server
# Keep it running

Option 1: Command - Line Interface

# Run the CLI interface client in a new terminal (suitable for debugging)
cd mcp-client-cli-interface
npm install  # Install dependencies first

# Set the Anthropic API key as an environment variable
export ANTHROPIC_API_KEY=sk-ant-xxxx  # Replace xxxx with your actual Anthropic API key
# For Windows, use: set ANTHROPIC_API_KEY=sk-ant-xxxx
# To set it permanently, add it to your shell configuration file (.bashrc, .zshrc, etc.)
npx tsx main.ts

Option 2: Web Application Interface

# Run the CLI interface client in a new terminal (suitable for debugging)
cd mcp-client-nextjs
npm install  # Install dependencies first

# Set the API key via the command line
echo "ANTHROPIC_API_KEY=sk-ant-XXXXXXXX" > .env  # Replace XXXXXXXX with your actual key
# Or append to preserve other environment variables
# echo "ANTHROPIC_API_KEY=sk-ant-XXXXXXXX" >> .env

npm run dev
# Open the provided localhost webpage

💡 What Can I Do with It?

• Build custom workflow agents to perform various operations.
• Build user - friendly computer automation interfaces for users.
• Save workflows and set cron jobs to run them.
• Combine with other MCP servers to complete cool tasks, such as populating Google Sheets based on the history of the people you've talked to throughout the day.

🐛 Submit Issues or Feature Requests on GitHub

If you have any questions or feature requests, please submit them in our GitHub repository.