Layout Detector MCP
An MCP (Model Context Protocol) server that analyzes webpage screenshots to extract layout information. Given a screenshot and image assets, it finds where each asset appears and calculates spatial relationships - enabling AI assistants to rebuild layouts with proper semantic structure.
Quick Start
Install from GitHub
pip install git+https://github.com/katlis/layout-detector-mcp.git
Or clone and install locally
git clone https://github.com/katlis/layout-detector-mcp.git
cd layout-detector-mcp
pip install .
Verify installation:
python3 -c "from layout_detector import server; print('OK')"
Configuration
Add to your Claude Code MCP settings (~/.claude.json or project .claude/settings.json):
{
"mcpServers": {
"layout-detector": {
"command": "layout-detector-mcp"
}
}
}
After adding the configuration, restart Claude Code and run /mcp to verify the server is connected.
The Problem
When an AI assistant looks at a screenshot, it can describe what it sees but cannot extract precise pixel measurements. This makes it difficult to accurately recreate layouts without human intervention or extensive trial-and-error.
The Solution
This MCP server uses computer vision (OpenCV template matching) to:
- Find known assets - Locate images within a screenshot with pixel-perfect coordinates
- Analyze relationships - Calculate angles, distances, and relative positions
- Detect patterns - Identify radial, grid, stacked, sidebar, or freeform layouts
- Enable semantic rebuilds - Provide structured data for modern CSS implementation
Tools
analyze_layout
Performs full layout analysis including pattern detection. This is the main tool you'll use.
Parameters:
screenshot_path(string, required): Absolute path to the screenshot imageasset_paths(array of strings, required): Absolute paths to asset images to findthreshold(number, optional): Match confidence 0-1, default 0.8
Returns:
{
"viewport": { "width": 900, "height": 650 },
"pattern": {
"type": "radial",
"confidence": 0.90
},
"radial": {
"center_x": 450,
"center_y": 250,
"center_element": "logo.gif",
"average_radius": 196
},
"elements": [
{
"asset_name": "planet1.gif",
"x": 628,
"y": 89,
"width": 62,
"height": 62,
"angle_degrees": 45.0,
"distance_from_center": 240
}
]
}
find_assets_in_screenshot
Locates image assets within a screenshot without layout analysis.
Parameters:
screenshot_path(string, required): Path to the screenshot imageasset_paths(array of strings, required): Paths to asset images to findthreshold(number, optional): Match confidence 0-1, default 0.8
Returns:
{
"found": 5,
"total_assets": 6,
"matches": [
{
"asset_path": "/path/to/logo.png",
"asset_name": "logo.png",
"x": 350,
"y": 200,
"width": 200,
"height": 100,
"center_x": 450,
"center_y": 250,
"confidence": 0.95
}
]
}
get_screenshot_info
Get basic screenshot dimensions.
Parameters:
screenshot_path(string, required): Path to the screenshot image
Returns:
{
"path": "/path/to/screenshot.png",
"width": 900,
"height": 650
}
Supported Layout Patterns
| Pattern | Description | Key Data Returned |
|---|---|---|
| Radial | Elements arranged around a center point | Center element, angles, distances |
| Grid | Elements in rows and columns | Row/column positions, gaps |
| Stacked | Vertical sections (header/main/footer) | Section names, Y positions |
| Sidebar | Two-column with narrow sidebar | Sidebar side, widths |
| Freeform | No clear pattern | Raw X/Y coordinates |
Example Usage
Once configured, Claude Code can use these tools:
User: Rebuild this webpage screenshot using the images in /assets
Claude: I'll analyze the layout first using the layout detector.
[Calls analyze_layout tool]
The analysis shows:
- Viewport: 900x650px
- Pattern: Radial (90% confidence)
- Center element: logo.gif at (450, 250)
- 8 elements arranged around the center
- Average distance from center: 196px
I'll implement this using CSS with the logo centered and
other elements positioned using absolute positioning...
Supported Image Formats
- PNG
- JPEG
- GIF (including animated - uses first frame)
- WebP
- BMP
Troubleshooting
"No module named 'cv2'"
OpenCV isn't installed. Run:
pip install opencv-python-headless
MCP server not showing in /mcp
- Check your settings file path is correct
- Ensure the command path is absolute (for source installs)
- Restart Claude Code after changing settings
- Run
python3 test_install.pyto verify the package works
Low confidence matches
Try lowering the threshold parameter (default 0.8). Values between 0.6-0.7 may help with compressed or scaled images.
Development
# Install in editable mode with dev dependencies
pip install -e ".[dev]"
# Run tests
pytest
# Test installation
python3 test_install.py
Requirements
- Python 3.11+
- OpenCV (opencv-python-headless)
- NumPy
- Pillow
- MCP SDK
License
MIT