macOS Control MCP Server
A Model Context Protocol (MCP) server for macOS desktop automation using FastMCP and PyAutoGUI. This MCP gives full control to your MacOS desktop. It will work with Linux, and windows but these are not tested. Needless to day this is very dangerous. Run in a container, VM, etc.
Features
Mouse Control
- Move mouse to coordinates
- Click (left, right, middle)
- Double-click
- Scroll
- Drag and drop
- Get mouse position
Keyboard Control
- Type text
- Press individual keys
- Execute hotkey combinations (Cmd+C, Cmd+V, etc.)
- Hold and release keys
Screen Capture & Analysis
- Take screenshots (full screen or regions)
- Get screen resolution
- Locate images on screen
- Get pixel colors at coordinates
Utilities
- Sleep/pause execution
- Configure failsafe and pause settings
Installation
Prerequisites
- macOS
- Python 3.10 or later
- uv package manager
Setup
-
Clone or navigate to the project directory:
cd ~/macos-control-mcp -
Install dependencies:
uv sync -
Grant necessary permissions:
- Accessibility Access: System Settings → Privacy & Security → Accessibility
- Screen Recording: System Settings → Privacy & Security → Screen Recording
Add Terminal (or your terminal app) to both of these permissions.
Configure Claude Desktop For development
Add this to your Claude Desktop configuration file (~/Library/Application Support/Claude/claude_desktop_config.json):
{
"mcpServers": {
"macos-control": {
"command": "uv",
"args": [
"--directory",
"<path_to_folder>/macos-control-mcp",
"run",
"macos-control-mcp"
]
}
}
}
Usage Examples
Once configured in Claude Desktop, you can use natural language to control your Mac:
Mouse Control
- "Move the mouse to position 500, 300"
- "Click at coordinates 100, 200"
- "Double-click at the center of the screen"
- "Right-click at 400, 500"
- "Scroll down 5 clicks"
Keyboard Control
- "Type 'Hello World'"
- "Press the Enter key"
- "Press Cmd+C to copy"
- "Press Cmd+Shift+3 to take a screenshot"
Screen Capture
- "Take a screenshot"
- "What's the screen resolution?"
- "Get the color of the pixel at 100, 100"
Combined Actions
- "Move mouse to 500, 300 and click"
- "Type 'test@example.com' and press Tab"
- "Press Cmd+Space, type 'Safari', and press Enter"
Available Tools
Mouse Tools
move_mouse(x, y, duration)- Move cursor to coordinatesclick_mouse(x, y, button, clicks, interval)- Click at positiondouble_click(x, y)- Double-clickright_click(x, y)- Right-clickscroll_mouse(clicks, x, y)- Scroll wheeldrag_mouse(x, y, duration, button)- Drag to positionget_mouse_position()- Get current cursor position
Keyboard Tools
type_text(text, interval)- Type textpress_key(key, presses, interval)- Press a keyhotkey(*keys)- Press key combinationkey_down(key)- Hold down a keykey_up(key)- Release a key
Screen Tools
get_screen_size()- Get screen resolutiontake_screenshot(region)- Capture screenlocate_on_screen(image_path, confidence)- Find imageget_pixel_color(x, y)- Get RGB color
Utility Tools
sleep(seconds)- Pause executionset_failsafe(enabled)- Configure failsafeset_pause(duration)- Set action pause
Safety Features
Failsafe
By default, moving the mouse to any screen corner will abort PyAutoGUI operations. This can be disabled with set_failsafe(False).
Pause
A 0.1 second pause is inserted between actions by default. Adjust with set_pause(duration).
Development
Running Tests
uv run pytest
Running the Server Manually
uv run macos-control-mcp
Troubleshooting
"Permission denied" errors
Make sure you've granted Accessibility and Screen Recording permissions to your terminal application.
"Image not found" errors
When using locate_on_screen, try lowering the confidence parameter (default is 0.9).
Server not responding
- Check Claude Desktop logs:
~/Library/Logs/Claude/ - Verify the path in your configuration matches your installation
- Restart Claude Desktop
License
MIT
Credits
Built with: