SceneCraft MCP — Text → Storyboard → Video (Test Version)

Author: snippetWizard

SceneCraft MCP turns plain text into a visual plan (storyboard) and—optionally—an assembled vertical video. It exposes the pipeline both as Python modules and as an MCP tool so agentic clients can orchestrate it.

Text-to-Scene breakdown using an LLM (OpenAI, Ollama, or an offline Mock).
Pydantic-backed models for Shots, Scenes, and Storyboards.
Optional Stable Diffusion frame generation + MoviePy assembly into a vertical MP4.
File-based storage of generated storyboards.
MCP server (optional) so tools-savvy LLMs can call it directly.

This repository is a test version. More features are on the way, including cloud uploads and cross-platform social scheduling.

What’s MCP? The Model Context Protocol is a lightweight protocol that lets LLMs talk to local/remote tools securely and consistently. Here, we provide a simple MCP server exposing a “create_storyboard” tool.

Why this project? Planning shots is time-consuming. The idea came from wanting a zero-friction loop for solo creators: describe a scene → get a structured shot plan → (optionally) generate visuals → assemble a draft video. It’s a rapid prototyping lane for script-to-screen experiments.

Pipeline Overview (Files & Flow)

Input (your text/script)
- Entry points:
  - examples/test_scene_local.py:1 — generates a storyboard only.
  - examples/script_to_video.py:1 — full storyboard → frames → video demo.
Parse Script → Scenes
- scenecraft_mcp/engine/script_parser.py:1
  - parse_script(script) — minimal parser that treats the whole text as one scene (replaceable later with a real slugline parser).
Plan Shots (LLM)
- scenecraft_mcp/engine/shot_planner.py:1
  - plan_shots(scene_text, scene_number, style_preset) → list[Shot]
  - _build_system_prompt(...), _build_user_prompt(...) craft prompt instructions.
  - Uses scenecraft_mcp/llm/factory.py:1 → get_llm_client() to select provider.
- Providers (implement LLMClient):
  - scenecraft_mcp/llm/base.py:1 — abstract interface (complete_json, complete_text).
  - scenecraft_mcp/llm/openai_client.py:1 — calls OpenAI Chat Completions and parses strict JSON.
  - scenecraft_mcp/llm/ollama_client.py:1 — calls local Ollama /api/chat and parses JSON.
  - scenecraft_mcp/llm/mock_client.py:1 — offline, deterministic mock for local demos (default in .env).
- Config:
  - scenecraft_mcp/config.py:1 — LLMProvider enum (openai, ollama, mock) and env-driven settings.
Data Models & Storage
- scenecraft_mcp/models.py:1 — Pydantic models: Shot, Scene, Storyboard, enums (ShotType, CameraAngle, etc.).
- scenecraft_mcp/storage/repository.py:1 — repo interface.
- scenecraft_mcp/storage/file_repository.py:1 — JSON-based persistence under ~/.scenecraft_mcp/projects/.
- scenecraft_mcp/utils/ids.py:1 — generate_project_id() like proj_ab12cd34.
Optional: Frame Generation (Stable Diffusion)
- scenecraft_mcp/video/framegen_sd.py:1 — SDFrameGenerator.generate_frame(...)
  - Loads a Stable Diffusion pipeline (e.g., runwayml/stable-diffusion-v1-5) on CPU/GPU.
  - Produces stylized vertical 9:16 PNGs per shot.
Optional: Video Assembly (MoviePy)
- scenecraft_mcp/video/assembler.py:1 — assemble_video(frames, output_path, fps=24)
  - Resizes and concatenates the per-shot images into a vertical MP4 (requires ffmpeg).
MCP Server (Optional Integration)
- scenecraft_mcp/mcp_server.py:1 — build_server() with a create_storyboard(script, title=None) tool.
- Expose storyboard creation to MCP-compatible hosts (e.g., Claude Desktop) once mcp is installed.

Installation

Core (storyboard + MCP):

pip install -r requirements.txt

Optional (frames + video):

ffmpeg (required by MoviePy)
Python packages:

pip install moviepy diffusers
# Then install PyTorch appropriate for your system:
# CPU-only example:
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu

Tip: For NVIDIA GPUs, use the official PyTorch site to get the correct CUDA wheels: https://pytorch.org/get-started/locally/

Configuration

Copy .env (already included) and set a provider:

# LLM Provider (openai / ollama / mock)
LLM_PROVIDER=mock

# OpenAI
OPENAI_API_KEY=sk-your-key
OPENAI_MODEL=gpt-4.1-mini

# Ollama
OLLAMA_MODEL=llama3
OLLAMA_BASE_URL=http://127.0.0.1:11434

# Performance
LOW_LATENCY_MODE=true

mock runs fully offline for predictable local demos.
ollama requires ollama serve and a pulled model like llama3.
openai requires a valid OPENAI_API_KEY.

Quickstart

Storyboard-only (offline by default):

python examples/test_scene_local.py

Storyboard → Frames → Video (requires optional deps):

python examples/script_to_video.py

Outputs:

Storyboard JSON: ~/.scenecraft_mcp/projects/<project_id>.json
Frames: outputs/frames/*.png
Video: outputs/videos/<project_id>.mp4

Using the MCP Server (optional)

pip install mcp
python -m scenecraft_mcp.mcp_server

Connect this server to an MCP-compatible host (e.g., Claude Desktop). The tool create_storyboard accepts script and returns { project_id, scene_count, shot_count }.

What This Solves

Fast ideation: turn a draft description into a structured shot plan.
Consistent visual language: shot types, angles, durations captured as data, not prose.
Extensible pipeline: swap LLMs, models, or frame generators without changing the flow.
Automation-ready: MCP tool interface and file-based outputs are easy to orchestrate.

Roadmap (Test Version — Coming Soon)

Cloud uploads after video generation (S3/GCS/Azure).
Automation script to fetch by id/date and auto-post to YouTube and Instagram.
Schedule entire months of content in one pass.
Better text-to-scene parser for multi-scene scripts.
Transitions, captions/overlays, TTS/VO integration, music bed.
More MCP tools and richer schemas.

Follow along — more MCP projects are coming soon. Star and follow on GitHub: snippetWizard. Stay tuned!

File-by-File Quick Reference

scenecraft_mcp/engine/script_parser.py:1 — parse_script(script) → minimal single-scene dict.
scenecraft_mcp/engine/shot_planner.py:1 — plan_shots(...) calls LLM to output ShotPlan.
scenecraft_mcp/llm/factory.py:1 — get_llm_client() from env-configured provider.
scenecraft_mcp/llm/openai_client.py:1 — OpenAI JSON completions.
scenecraft_mcp/llm/ollama_client.py:1 — Ollama JSON completions.
scenecraft_mcp/llm/mock_client.py:1 — offline JSON synthesis for demos.
scenecraft_mcp/models.py:1 — core Pydantic models and enums.
scenecraft_mcp/storage/file_repository.py:1 — save/load Storyboard to ~/.scenecraft_mcp/projects/.
scenecraft_mcp/video/framegen_sd.py:1 — Stable Diffusion image generator per shot.
scenecraft_mcp/video/assembler.py:1 — MoviePy image sequence → MP4.
scenecraft_mcp/mcp_server.py:1 — MCP tool create_storyboard.
examples/test_scene_local.py:1 — storyboard demo.
examples/script_to_video.py:1 — full pipeline demo.

License

This project is licensed under the MIT License. See LICENSE for details.