Qt Pilot
An MCP server for headless Qt/PySide6 GUI testing. Enables AI assistants like Claude to visually test and interact with Qt desktop applications.
Repository: github.com/neatobandit0/qt-pilot
Features
- Launch Qt apps headlessly via Xvfb virtual display
- Capture screenshots for visual verification
- Simulate interactions: clicks, hovers, keyboard input
- Widget discovery by object name
- App health monitoring with stderr capture
- Full Qt introspection via QTest and Qt APIs
Installation
From GitHub
git clone https://github.com/neatobandit0/qt-pilot.git ~/.claude/plugins/qt-pilot
pip install -r ~/.claude/plugins/qt-pilot/requirements.txt
Manual Installation
Copy the plugin to your Claude plugins directory:
cp -r qt-pilot ~/.claude/plugins/
Then add to your ~/.claude.json:
{
"mcpServers": {
"qt-pilot": {
"type": "stdio",
"command": "python3",
"args": ["/path/to/qt-pilot/server/main.py"]
}
}
}
Dependencies
pip install mcp PySide6
Also requires Xvfb for headless display:
# Debian/Ubuntu
sudo apt install xvfb
# RHEL/CentOS/Fedora
sudo yum install xorg-x11-server-Xvfb
# macOS (via Homebrew)
brew install xquartz
MCP Tools
launch_app
Launch a Qt application headlessly.
# Script mode
launch_app(script_path="/path/to/test_gui.py")
# Module mode
launch_app(module="myapp.main", working_dir="/path/to/project")
capture_screenshot
Capture the current window.
capture_screenshot(output_path="/tmp/screenshot.png")
click_widget
Click a widget by its object name.
click_widget(widget_name="submit_button", button="left")
hover_widget
Hover over a widget.
hover_widget(widget_name="menu_item")
type_text
Type text into a widget or focused widget.
type_text(text="hello world", widget_name="search_input")
type_text(text="hello") # Types into currently focused widget
press_key
Simulate a key press with optional modifiers.
press_key(key="Enter")
press_key(key="S", modifiers=["Ctrl"]) # Ctrl+S
press_key(key="Tab")
find_widgets
List widgets matching a name pattern.
find_widgets(name_pattern="*") # All named widgets
find_widgets(name_pattern="btn_*") # Widgets starting with "btn_"
get_widget_info
Get detailed widget information.
get_widget_info(widget_name="submit_button")
# Returns: type, visible, enabled, size, position, text, checked state, etc.
get_app_status
Check if the application is still running and get diagnostics.
get_app_status()
# Returns: {"running": true, "exit_code": null, "stderr": "", "display": ":99"}
wait_for_idle
Wait for the Qt event queue to settle after actions.
click_widget(widget_name="load_button")
wait_for_idle(timeout=5.0) # Wait for async operations to complete
capture_screenshot()
close_app
Close the running application.
close_app()
Requirements for Target Applications
For widget interactions to work, your Qt application must:
-
Set object names on interactive widgets:
button = QPushButton("Click Me") button.setObjectName("my_button") # Required for widget discovery -
Use QApplication (not QCoreApplication)
-
Show at least one window
Architecture
┌─────────────────────────────┐
│ AI Assistant (Claude) │
└─────────────┬───────────────┘
│ MCP Protocol (stdio)
▼
┌─────────────────────────────┐
│ MCP Server (main.py) │
│ - Tool definitions │
│ - Process management │
└─────────────┬───────────────┘
│ Unix Socket (IPC)
▼
┌─────────────────────────────┐
│ Test Harness (harness.py) │
│ - Runs inside Xvfb │
│ - QTest interactions │
│ - Widget introspection │
├─────────────────────────────┤
│ Your Qt Application │
└─────────────────────────────┘
Example Workflow
# 1. Launch a test app
launch_app(module="myapp.main", working_dir="/path/to/project")
# 2. List available widgets
find_widgets()
# 3. Interact with the UI
click_widget(widget_name="login_button")
wait_for_idle()
# 4. Type into a field
type_text(text="user@example.com", widget_name="email_input")
press_key(key="Tab")
type_text(text="password123", widget_name="password_input")
# 5. Submit and capture result
click_widget(widget_name="submit_button")
wait_for_idle(timeout=3.0)
capture_screenshot(output_path="/tmp/result.png")
# 6. Clean up
close_app()
Troubleshooting
"Widget not found"
- Ensure the widget has
setObjectName()called - Use
find_widgets()to list available widget names
"No app is running"
- Call
launch_app()first - Check that the script/module path is correct
App crashes silently
- Use
get_app_status()to check for errors - The
stderrfield contains crash information
Screenshots are blank
- Ensure the application creates and shows a window
- Use
wait_for_idle()after launch for window to render
License
MIT License - see LICENSE file.