Omniparser AutoGUI

🚀 omniparser-autogui-mcp

This is an MCP server that uses OmniParser to analyze the screen and perform automatic GUI operations. It is confirmed to be available on Windows.

🚀 Quick Start

This is an MCP server that uses OmniParser to analyze the screen and perform automatic GUI operations. It has been confirmed to work on Windows. (Japanese version here)

📄 License

This project is licensed under the MIT license, excluding sub - modules and sub - packages. The OmniParser repository uses the CC - BY - 4.0 license. Each OmniParser model has a different license (Reference).

📦 Installation

1. Please follow these steps:

git clone --recursive https://github.com/NON906/omniparser-autogui-mcp.git
cd omniparser-autogui-mcp
uv sync
set OCR_LANG=en
uv run download_models.py

(On non - Windows systems, use export instead of set.) (If you want langchain_example.py to run correctly, use uv sync --extra langchain.)

2. Add the following content to your claude_desktop_config.json file:

{
  "mcpServers": {
    "omniparser_autogui_mcp": {
      "command": "uv",
      "args": [
        "--directory",
        "D:\\CLONED_PATH\\omniparser-autogui-mcp",
        "run",
        "omniparser-autogui-mcp"
      ],
      "env": {
        "PYTHONIOENCODING": "utf-8",
        "OCR_LANG": "en"
      }
    }
  }
}

(Replace D:\\CLONED_PATH\\omniparser-autogui-mcp with the actual path where you cloned the repository.)

The env allows the following additional configurations:

• OMNI_PARSER_BACKEND_LOAD Specify 1 if there is a conflict with other clients (e.g., LibreChat).

• TARGET_WINDOW_NAME Specify the window name if you want to target a specific window. If not specified, the entire screen will be processed by default.

• OMNI_PARSER_SERVER Specify the server address and port (e.g., 127.0.0.1:8000) if you want OmniParser processing to be performed on another device. The server can be started with uv run omniparserserver.

• SSE_HOST, SSE_PORT If specified, communication will be via SSE instead of standard input/output.

• SOM_MODEL_PATH, CAPTION_MODEL_NAME, CAPTION_MODEL_PATH, OMNI_PARSER_DEVICE, BOX_TRESHOLD These are configurations for OmniParser. Normally, manual configuration is not required.

💻 Usage Examples

• Search for "MCP server" in the browser on the screen. And so on.