Happy slice of mcp pizzaMCP.pizza
Fire in da houseTop Tip:Paying $100+ per month for Perplexity, MidJourney, Runway, ChatGPT and other tools is crazy - get all your AI tools in one site starting at $15 per month with Galaxy AI Fire in da houseCheck it out free

swift-mcp-gui

MCP.Pizza Chef: NakaokaRei

swift-mcp-gui is an MCP server enabling programmatic control of macOS mouse and keyboard using SwiftAutoGUI. It supports macOS 15.0+ and Swift 6.0+, allowing MCP clients to execute commands like mouse movement and clicks, and keyboard input. This server facilitates automation and interaction with the macOS GUI environment through standardized MCP protocol commands.

Use This MCP server To

Automate mouse movements and clicks on macOS Simulate keyboard input for macOS applications Integrate macOS GUI control into MCP workflows Enable remote control of macOS GUI via MCP clients Create automated testing scripts for macOS UI Trigger macOS GUI actions from AI-driven agents

README

Swift MCP GUI Server

A Model Context Protocol (MCP) server that allows controlling Mac OS X through SwiftAutoGUI. This server provides tools for programmatically controlling the mouse and keyboard through MCP clients.

Requirements

  • macOS 15.0 or later
  • Swift 6.0 or later
  • Xcode 16.0 or later

Installation

  1. Clone this repository:
git clone https://github.com/NakaokaRei/swift-mcp-gui.git
cd swift-mcp-gui
  1. Install
swift package experimental-install
  1. Add command to your MCP client.
{
  "mcpServers" : {
    "swift-mcp-gui" : {
      "command" : "/Users/USERNAME/.swiftpm/bin/swift-mcp-gui"
    }
  }
}

Available Tools

The server provides the following tools for controlling macOS:

1. Mouse Movement

  • Tool name: moveMouse
  • Input:
    • x: double (x-coordinate)
    • y: double (y-coordinate)
  • Moves the mouse cursor to the specified coordinates

2. Mouse Clicks

  • Tool name: mouseClick
  • Input:
    • button: String ("left" or "right")
  • Performs a mouse click at the current cursor position

3. Keyboard Input

  • Tool name: sendKeys
  • Input:
    • keys: Array of strings (key names)
  • Sends keyboard shortcuts or key combinations
  • Example keys: "command", "control", "option", "shift", "return", "space", "a", "1", etc.

4. Scrolling

  • Tool name: scroll
  • Input:
    • direction: String ("up", "down", "left", "right")
    • clicks: Integer (number of scroll clicks)
  • Performs scrolling in the specified direction

Security Considerations

This server has full control over your mouse and keyboard. Be careful when running it and only connect trusted MCP clients.

License

MIT License

swift-mcp-gui FAQ

What macOS versions does swift-mcp-gui support?
It supports macOS 15.0 or later.
What are the system requirements for swift-mcp-gui?
Requires macOS 15.0+, Swift 6.0+, and Xcode 16.0+.
How do I install swift-mcp-gui?
Clone the repo and run 'swift package experimental-install' as per instructions.
How does swift-mcp-gui integrate with MCP clients?
MCP clients add the server command to their config to send control commands.
What kind of controls does swift-mcp-gui provide?
It provides mouse movement, mouse clicks, and keyboard input control tools.
Can swift-mcp-gui be used for automated UI testing?
Yes, it enables scripting of mouse and keyboard actions for testing.
Is swift-mcp-gui limited to any specific programming language?
No, it communicates via MCP protocol, usable by any MCP client.
Does swift-mcp-gui support secure or scoped control?
It follows MCP principles for scoped and observable model interaction.