Mspaint MCP Server

This project uses the Model Context Protocol (MCP) framework to achieve automated control of the Windows Paint program (MSPaint) by an AI agent. It uses pywinauto to operate the Paint tool and combines with the Gemini model to implement drawing functions driven by natural language instructions.

Operating system automation Image and video processing #AI automation #MCP protocol #Paint control .Python

rating : 2 points

downloads : 12

update time : 2025-04-29

What is the MSPaint MCP service?

This is a service that connects the Windows Paint program (MSPaint) with an AI intelligent agent. Through the Model Context Protocol (MCP), users can use natural language instructions to let the AI automatically perform operations such as opening Paint, drawing shapes, and adding text.

How to use the MSPaint MCP service?

Simply install the Python environment and run the client program, then send instructions like 'Draw a blue rectangle in Paint and add text' to the AI.

Use cases

Suitable for scenarios that require automating simple drawing tasks, such as quickly creating schematic diagrams, teaching demonstrations, and automated testing.

Main features

Paint program controlAutomatically open/close the Windows Paint program and control the window size and position

Basic drawing functionsSupport basic drawing operations such as drawing rectangles and selecting colors

Text additionAdd custom text content at a specified position on the canvas

Advantages and limitations

Advantages

Automate drawing operations without programming knowledge

Control through natural language instructions, simple and intuitive to use

Combine with other AI functions to implement complex automation processes

Limitations

Only support the Windows system

The drawing functions are relatively basic and do not support complex shapes

Coordinate positioning may need to be adjusted due to different screen resolutions

How to use

Install the environment

Install Python 3.11+ and the Conda environment management tool

Install dependencies

Use pip to install the required Python packages

Configure the API key

Set your Gemini API key in the.env file

Run the client

Start the MCP client program to start using

Usage examples

Create a simple schematic diagramQuickly draw simple shapes with explanatory text

Teaching demonstrationAutomatically demonstrate basic drawing operations

Frequently Asked Questions

Why doesn't my Paint program respond correctly?

What drawing operations are supported?

What kind of hardware configuration is required?

Related resources

pywinauto documentation

Documentation for the Windows GUI automation library

Google Gemini API

Documentation for the Gemini AI model API

Project code repository

Source code for this project

🚀 Model Context Protocol (MCP) Drawing Application Automation

This project demonstrates how to use the Model Context Protocol (MCP) to automate interactions with legacy Windows applications (MSPaint). It leverages pywinauto to control the drawing application and fastmcp to define tools that can be invoked by an AI agent. The AI agent, powered by Google's Gemini model, can execute tasks via natural language instructions, such as drawing rectangles and adding text on the Paint canvas.

🚀 Quick Start

This project showcases the automation of MSPaint through an AI agent. The agent can open Paint, draw rectangles, and add text, all driven by natural language instructions.

✨ Features

MCP Framework: Enables AI models to interact with external tools and resources.
Automation: Automates interactions with MSPaint using natural language instructions.
AI - Powered: Utilizes Google's Gemini model for intelligent task execution.

📦 Installation

Requirements

Python Version: 3.6 or higher
Dependencies:
- python - dotenv
- http.server
- argparse

Setup Steps

Install the required Python packages:

pip install python - dotenv

Download and install the latest version of the [MCP protocol](https://github.com/your - repository).
Install all dependencies from requirements.txt into your environment.
Configure environment variables by storing the Google Gemini API key in the .env file:

GEMINI_API_KEY=your_api_key_here

💻 Usage Examples

Basic Usage

Run the MCP server:

python mcp_server.py --port 8000

Start the MCP client:

python mcp_client.py --mode automation

📚 Documentation

Project Structure

├── MSPaint - MCP - Server/
│ ├── mcp_server.py # Defines the MCP server with Paint automation functions
│ ├── mcp_client.py # Defines the MCP client that interacts with the server and AI model
│ ├── requirements.txt # Lists project dependencies
│ └── .env # Stores Google Gemini API key

How It Works

MCP Protocol: Serves as a communication bridge between the AI agent and the Paint application.
Natural Language Processing: The AI agent parses user instructions and selects appropriate tools to perform operations.
Drawing Automation: Uses the pywinauto library to control the Paint window for graphic drawing.

Key Components

mcp_server.py: Responsible for receiving and processing requests from the client and managing interactions with the Paint application.
mcp_client.py: Serves as the user interface, receiving input instructions and invoking corresponding MCP services.
requirements.txt: Lists all Python packages required by the project.
.env: Stores sensitive information such as API keys.

🔧 Technical Details

The Model Context Protocol (MCP) is a framework that allows AI models to interact with external tools and resources. It provides a standard way for models to call functions, retrieve data, and perform real - world operations. In this project, MCP is used to expose Paint automation functions as tools that can be invoked by an AI agent.