Fire in da houseTop Tip:Paying $100+ per month for Perplexity, MidJourney, Runway, ChatGPT and other tools is crazy - get all your AI tools in one site starting at $15 per month with Galaxy AI Fire in da houseCheck it out free

computer-control-mcp

MCP.Pizza Chef: AB498

The computer-control-mcp is an MCP server that provides comprehensive computer control capabilities including mouse and keyboard automation, optical character recognition (OCR), and more. It leverages PyAutoGUI for input control, RapidOCR and ONNXRuntime for OCR tasks, and operates with zero external dependencies. Primarily tested on Windows, it aims to offer robust automation and interaction features similar to Anthropic's 'computer-use' MCP, facilitating seamless programmatic control of computer interfaces.

Use This MCP server To

Automate mouse and keyboard actions for UI testing Extract text from screen regions using OCR Control desktop applications programmatically Simulate user input for repetitive tasks Integrate OCR-based data extraction in workflows Enable AI agents to interact with the OS GUI Perform screen reading and text recognition Automate form filling and navigation on desktop

README

Computer Control MCP

MCP server that provides computer control capabilities, like mouse, keyboard, OCR, etc. using PyAutoGUI, RapidOCR, ONNXRuntime. Similar to 'computer-use' by Anthropic. With Zero External Dependencies.

  • Only tested on Windows. Should work on other platforms.
Discord License: MIT

MCP Computer Control Demo

Quick Usage (MCP Setup Using uvx)

Note: Running uvx computer-control-mcp@latest for the first time will download python dependencies (around 70MB) which may take some time. Recommended to run this in a terminal before using it as MCP. Subsequent runs will be instant.

{
  "mcpServers": {
    "computer-control-mcp": {
      "command": "uvx",
      "args": ["computer-control-mcp@latest"]
    }
  }
}

OR install globally with pip:

pip install computer-control-mcp

Then run the server with:

computer-control-mcp # instead of uvx computer-control-mcp, so you can use the latest version, also you can `uv cache clean` to clear the cache and `uvx` again to use latest version.

Features

  • Control mouse movements and clicks
  • Type text at the current cursor position
  • Take screenshots of the entire screen or specific windows with optional saving to downloads directory
  • Extract text from screenshots using OCR (Optical Character Recognition)
  • List and activate windows
  • Press keyboard keys
  • Drag and drop operations

Available Tools

Mouse Control

  • click_screen(x: int, y: int): Click at specified screen coordinates
  • move_mouse(x: int, y: int): Move mouse cursor to specified coordinates
  • drag_mouse(from_x: int, from_y: int, to_x: int, to_y: int, duration: float = 0.5): Drag mouse from one position to another

Keyboard Control

  • type_text(text: str): Type the specified text at current cursor position
  • press_key(key: str): Press a specified keyboard key

Screen and Window Management

  • take_screenshot(title_pattern: str = None, use_regex: bool = False, threshold: int = 60, with_ocr_text_and_coords: bool = False, scale_percent_for_ocr: int = 100, save_to_downloads: bool = False): Capture screen or window with optional OCR
  • get_screen_size(): Get current screen resolution
  • list_windows(): List all open windows
  • activate_window(title_pattern: str, use_regex: bool = False, threshold: int = 60): Bring specified window to foreground

Development

Setting up the Development Environment

# Clone the repository
git clone https://github.com/AB498/computer-control-mcp.git
cd computer-control-mcp

# Install in development mode
pip install -e .

# Start server
python -m computer_control_mcp.core

Running Tests

python -m pytest

API Reference

See the API Reference for detailed information about the available functions and classes.

License

MIT

For more information or help

computer-control-mcp FAQ

How do I install the computer-control-mcp server?
You can install it easily using the uvx tool with the command `uvx computer-control-mcp@latest`, which handles all Python dependencies automatically.
Is computer-control-mcp cross-platform?
It is primarily tested on Windows but should work on other platforms with compatible Python environments.
Does computer-control-mcp require external dependencies?
No, it operates with zero external dependencies, bundling all necessary libraries like PyAutoGUI, RapidOCR, and ONNXRuntime.
Can computer-control-mcp perform OCR on screen content?
Yes, it uses RapidOCR and ONNXRuntime to perform optical character recognition on screen regions.
How does computer-control-mcp compare to Anthropic's 'computer-use'?
It offers similar computer control capabilities but is designed to run without external dependencies and is open source.
What programming languages or environments does computer-control-mcp support?
It is a Python-based MCP server but can be integrated with any MCP client that supports the protocol.
Can I use computer-control-mcp to automate complex workflows?
Yes, it supports mouse, keyboard automation, and OCR, enabling complex desktop automation workflows.
Is there community support available for computer-control-mcp?
Yes, there is a Discord community linked in the GitHub repository for support and discussion.