MCP Synth Controller
This project implements a Python-based Model Context Protocol (MCP) server designed to allow large language models (LLMs) to control parameters of a JUCE synthesizer in real time.
The MCP server exposes structured tools that an LLM can call, and these tool calls are translated into OSC messages sent to a running JUCE plugin or application.
This repository contains the Python side only. The corresponding JUCE synthesizer must implement an OSC receiver capable of handling messages such as:
/setParameter <paramName> <value>
The JUCE implementation is being developed separately in: https://github.com/TYLERSFOSTER/MIDIControl001
Project Overview
This project provides:
- A working MCP server (
mcp_server.py) - Tool schemas that define how the LLM may interact (
schemas.py) - Python functions implementing those tools (
tools.py) - An OSC bridge for communicating with the JUCE synth (
juce_bridge.py) - A test suite ensuring correctness (
tests/)
The goal is to enable a workflow such as:
- A speech-to-text system turns spoken commands into text.
- The text is fed to an LLM.
- The LLM responds by invoking MCP tools such as
setParameter. - The MCP server receives the tool call and executes the corresponding Python function.
- The function sends an OSC message to the JUCE synth, modifying parameters in real time.
This architecture allows expressive, natural-language control of a synthesizer using speech or text.
Directory Structure
mcp_synth_controller/
├── pyproject.toml
├── README.md
├── src/
│ └── server/
│ ├── config.py
│ ├── juce_bridge.py
│ ├── mcp_server.py
│ ├── schemas.py
│ └── tools.py
├── tests/
│ ├── test_osc_client_init.py
│ ├── test_list_parameters.py
│ ├── test_set_parameter_message_format.py
│ ├── test_mcp_initialize.py
│ └── test_mcp_tool_call.py
└── examples/
└── test_send_osc.py
Dependency Management
This project uses uv for Python dependency management.
To install dependencies:
uv sync
To run the MCP server:
uv run python src/server/mcp_server.py
To run the test suite:
uv run pytest
MCP Server
The MCP server:
- Handles the MCP initialization handshake.
- Advertises available tools to the LLM.
- Receives tool calls from the LLM.
- Dispatches these calls to the correct Python functions.
- Returns structured tool results.
The server communicates via STDIN/STDOUT using JSON, following the MCP protocol.
Tools
Each tool corresponds to a callable action available to the LLM.
Tools currently implemented:
setParameter(param: str, value: float)
Sets a synthesizer parameter via OSC.
getParameter(param: str)
Placeholder implementation (returns a dummy value).
Real bidirectional communication may be added later.
listParameters()
Returns a list of known parameters.
This can be later expanded to query the JUCE synth dynamically.
OSC Bridge
juce_bridge.py uses python-osc to send messages to JUCE.
By default, messages follow this format:
/setParameter <paramName> <value>
The corresponding JUCE OSCReceiver must be implemented.
See the next section.
Required JUCE Implementation
To complete this system, the JUCE synth must:
- Create an
OSCReceiver - Bind to the same port specified in
config.py - Add a listener for
/setParameter - Parse incoming messages and map parameter names to actual JUCE parameters
Example responsibilities on the JUCE side:
- Initialize an OSCReceiver (
connect(9001)) - Add a listener for
/setParameter - Extract parameter name and float value from the OSCMessage
- Apply the value using
setValueNotifyingHost
The JUCE implementation belongs in the separate repository:
https://github.com/TYLERSFOSTER/MIDIControl001
Example OSC Test
To manually verify OSC transmission:
uv run python examples/test_send_osc.py
This sends:
/setParameter testParam 0.42
If your JUCE OSCReceiver is active, it should appear in your debug output.
Tests
This project includes a complete pytest suite validating:
- OSC client initialization
- OSC message format
- MCP initialization handshake
- Tool call dispatch logic
- Parameter listing behavior
Run tests with:
uv run pytest
All tests should pass.
Future Work
- Implement bidirectional OSC or TCP communication with JUCE
- Add dynamic parameter discovery from JUCE
- Add ramping, smoothing, and modulation utilities
- Integrate with speech-to-text pipeline
- Provide real-time LLM agent control in Claude Desktop or similar
License
MIT License.