Omniparser Autogui MCP

🚀 omniparser-autogui-mcp

A server based on the MCP protocol that uses OmniParser to analyze the screen and automate GUI operations. It has been verified on Windows.

🚀 Quick Start

This is a server based on the MCP protocol. It utilizes OmniParser to analyze the screen and perform automatic operations on the graphical user interface (GUI). It has been verified on Windows.

📄 License

This project is licensed under the MIT license, excluding sub - modules and sub - packages. The OmniParser repository uses the CC - BY - 4.0 license. Each OmniParser model has a different license agreement (Reference).

📦 Installation

Step 1

Please follow these steps:

git clone --recursive https://github.com/NON906/omniparser-autogui-mcp.git
cd omniparser-autogui-mcp
uv sync
set OCR_LANG=en
uv run download_models.py

(For non - Windows systems, replace set with export.) (If you want langchain_example.py to run properly, use uv sync --extra langchain.)

Step 2

Add the following content to your claude_desktop_config.json file:

{
  "mcpServers": {
    "omniparser_autogui_mcp": {
      "command": "uv",
      "args": [
        "--directory",
        "D:\\CLONED_PATH\\omniparser-autogui-mcp",
        "run",
        "omniparser-autogui-mcp"
      ],
      "env": {
        "PYTHONIOENCODING": "utf-8",
        "OCR_LANG": "en"
      }
    }
  }
}

(Please replace D:\\CLONED_PATH\\omniparser-autogui-mcp with the actual path where you cloned the project.)

The following additional configurations are allowed:
- ``OMNI_PARSER_BACKEND_LOAD``
If there are conflicts with other clients (such as [LibreChat](https://github.com/danny-avila/LibreChat)), specify the value as `1`.

- ``TARGET_WINDOW_NAME``
If you want to specify the window to operate on, specify the window name.
If not specified, the entire screen will be operated on by default.

- ``OMNI_PARSER_SERVER``
If you want the OmniParser processing to be completed on another device, specify the server address and port, for example, `127.0.0.1:8000`.
The server can be started via `uv run omniparserserver`.

- ``SSE_HOST``, ``SSE_PORT``
If specified, the communication will be via SSE (Server - Sent Events) instead of standard input and output.

- ``SOM_MODEL_PATH``, ``CAPTION_MODEL_NAME``, ``CAPTION_MODEL_PATH``, ``OMNI_PARSER_DEVICE``, ``BOX_TRESHOLD``
These options are for OmniParser configuration.
Normally, you may not need to adjust these parameters.