Screenpilot

🚀 Screen Control Expert

The MCP server enables large language models to fully control your device, offering a screen automation toolkit for controlling and interacting with graphical user interfaces. It's suitable for automation, education, and entertainment.

🚀 Quick Start

The MCP server allows large language models to have comprehensive control over your device, providing a screen automation toolkit to control and interact with the graphical user interface. It is applicable to automation, education, and entertainment scenarios.

✨ Features

• 📷 Screen capture and analysis
• 🖱️ Mouse control (clicking, positioning)
• ⌨️ Keyboard input (typing, key presses, hotkeys)

📦 Installation

0. Install Python 3.12.
1. Clone the repository:

git clone https://github.com/Mtehabsim/ScreenPilot.git

2. Create a virtual environment:

python -m venv venv

3. Activate the virtual environment:

venv\Scripts\activate

4. Install the required packages:

pip install -r requirements.txt

5. Open the Claude AI desktop.
6. Go to File -> Settings -> Developer -> Edit Configuration.
7. Open the configuration file and paste the following content:

{
    "mcpServers": {
        "device-controll": {
            "command": "pathToEnv\\venv\\Scripts\\python.exe",
            "args": [
                "pathToProject\\ScreenPilot\\main.py"
            ]
        }
    }
}

8. Replace:

   • “pathToEnv\\venv\\Scripts\\python.exe” with the full path to your python.exe.
   • “pathToProject\\ScreenPilot\\main.py” with the full path to your main.py file.

9. Save the configuration file.

10. Open the Claude AI desktop.

11. Go to File -> Exit.

12. Now you can open the Claude AI desktop and enjoy ScreenPilot.

Available Tools

• Screen Capture: Capture the screen and obtain screen information.
• Mouse Control: Move the mouse and perform clicks.
• Keyboard Operations: Input text, press keys, and use hotkey combinations.
• Scrolling: Scroll in different directions and at specific positions.
• Element Detection: Check if an element exists on the screen and wait for its appearance.
• Action Sequences: Execute multiple actions in sequence.

🤝 Contribution

Contributions are welcome! Please feel free to submit a Pull Request.