LLM Council Core

A multi-LLM deliberation system where multiple LLMs collaboratively answer questions through peer review and synthesis. Available as a Python library, MCP server, or HTTP API.

What is This?

Instead of asking a single LLM for answers, this MCP server:

1. Stage 1: Sends your question to multiple LLMs in parallel (GPT, Claude, Gemini, Grok, etc.)
2. Stage 2: Each LLM reviews and ranks the other responses (anonymized to prevent bias)
3. Stage 3: A Chairman LLM synthesizes all responses into a final, high-quality answer

Quick Deploy

Deploy your own LLM Council instance:

Platform	Deploy	Best For
Railway		Production, webhooks
Render		Evaluation, free tier

Required Environment Variables:

• OPENROUTER_API_KEY - Your OpenRouter API key
• LLM_COUNCIL_API_TOKEN - A secure token for API authentication (generate with openssl rand -hex 16)

Note: Railway is recommended for n8n integration (no cold-start). Render Free tier spins down after 15 minutes which may cause webhook timeouts.

For detailed deployment instructions, see the Deployment Guide.

Credits & Attribution

This project is a derivative work based on the original llm-council by Andrej Karpathy.

Karpathy's original README stated:

"I'm not going to support it in any way, it's provided here as is for other people's inspiration and I don't intend to improve it. Code is ephemeral now and libraries are over, ask your LLM to change it in whatever way you like."

...the irony of producing a derivative work that packages the core concept for broader use via the Model Context Protocol!

Installation

pip install "llm-council-core[mcp]"

For core library only (no MCP server):

pip install llm-council-core

Setup

1. Choose Your Gateway

The council supports three gateway options for accessing LLMs:

Quick Start (OpenRouter):

# Sign up at openrouter.ai and get your API key
export OPENROUTER_API_KEY="sk-or-v1-..."

Direct Provider Access:

# Use your existing provider API keys directly
export ANTHROPIC_API_KEY="sk-ant-..."
export OPENAI_API_KEY="sk-..."
export GOOGLE_API_KEY="..."
export LLM_COUNCIL_DEFAULT_GATEWAY=direct

Requesty with BYOK:

export REQUESTY_API_KEY="..."
export ANTHROPIC_API_KEY="sk-ant-..."  # Your own key, routed through Requesty
export LLM_COUNCIL_DEFAULT_GATEWAY=requesty

2. Store Your API Keys Securely

Choose one of these options (in order of recommendation):

Option A: System Keychain (Most Secure)

Store keys encrypted in your OS keychain:

# Install with keychain support
pip install "llm-council-core[mcp,secure]"

# Store key securely (prompts for key, no echo)
llm-council setup-key

# For CI/CD automation, pipe from stdin:
echo "$OPENROUTER_API_KEY" | llm-council setup-key --stdin

Option B: Environment Variables

Set in your shell profile (~/.zshrc, ~/.bashrc):

# OpenRouter (default gateway)
export OPENROUTER_API_KEY="sk-or-v1-..."

# Or use direct provider APIs
export ANTHROPIC_API_KEY="sk-ant-..."
export OPENAI_API_KEY="sk-..."
export GOOGLE_API_KEY="..."

Option C: Environment File

Create a .env file (ensure it's in .gitignore):

# For OpenRouter
echo "OPENROUTER_API_KEY=sk-or-v1-..." > .env

# Or for direct APIs
cat > .env << 'EOF'
ANTHROPIC_API_KEY=sk-ant-...
OPENAI_API_KEY=sk-...
GOOGLE_API_KEY=...
LLM_COUNCIL_DEFAULT_GATEWAY=direct
EOF

Security Note: Never put API keys in command-line arguments or JSON config files that might be committed to version control.

3. Customize Models (Optional)

You can customize which models participate in the council using three methods (in priority order):

Option 1: Environment Variables (Recommended)

# Comma-separated list of council models
export LLM_COUNCIL_MODELS="openai/gpt-4,anthropic/claude-3-opus,google/gemini-pro"

# Chairman model (synthesizes final response)
export LLM_COUNCIL_CHAIRMAN="anthropic/claude-3-opus"

Option 2: YAML Configuration (Recommended)

Create llm_council.yaml in your project root or ~/.config/llm-council/llm_council.yaml:

council:
  # Tier configuration (ADR-022)
  tiers:
    default: high
    pools:
      quick:
        models:
          - openai/gpt-4o-mini
          - anthropic/claude-3-5-haiku-20241022
        timeout_seconds: 30
      balanced:
        models:
          - openai/gpt-4o
          - anthropic/claude-3-5-sonnet-20241022
        timeout_seconds: 90
      high:
        models:
          - openai/gpt-4o
          - anthropic/claude-opus-4-5-20250514
          - google/gemini-3-pro
        timeout_seconds: 180

  # Triage configuration (ADR-020)
  triage:
    enabled: false
    wildcard:
      enabled: true
    prompt_optimization:
      enabled: true

  # Gateway configuration (ADR-023, ADR-025a)
  gateways:
    default: openrouter
    fallback:
      enabled: true
      chain: [openrouter, ollama]  # Can use Ollama as fallback

    # Provider-specific configuration
    providers:
      ollama:
        enabled: true
        base_url: http://localhost:11434
        timeout_seconds: 120.0
        hardware_profile: recommended  # minimum|recommended|professional|enterprise

      openrouter:
        enabled: true
        base_url: https://openrouter.ai/api/v1/chat/completions

  # Webhook notifications (ADR-025a)
  webhooks:
    enabled: false  # Opt-in
    timeout_seconds: 5.0
    max_retries: 3
    https_only: true
    default_events:
      - council.complete
      - council.error

  observability:
    log_escalations: true

Priority: YAML config > Environment variables > Defaults

Option 3: JSON Configuration (Legacy)

Create ~/.config/llm-council/config.json:

{
  "council_models": [
    "openai/gpt-4-turbo",
    "anthropic/claude-3-opus",
    "google/gemini-pro",
    "meta-llama/llama-3-70b-instruct"
  ],
  "chairman_model": "anthropic/claude-3-opus",
  "synthesis_mode": "consensus",
  "exclude_self_votes": true,
  "style_normalization": false,
  "max_reviewers": null
}

Option 4: Use Defaults

If you don't configure anything, these defaults are used:

• Council: GPT-5.1, Gemini 3 Pro, Claude Sonnet 4.5, Grok 4
• Chairman: Gemini 3 Pro
• Mode: consensus
• Self-vote exclusion: enabled

Finding Models:

• OpenRouter: openrouter.ai/models
• Anthropic: docs.anthropic.com/models
• OpenAI: platform.openai.com/docs/models
• Google: ai.google.dev/gemini-api/docs/models

Usage

With Claude Code

# First, store your API key securely (one-time setup)
llm-council setup-key

# Then add the MCP server (key is read from keychain or environment)
claude mcp add --transport stdio llm-council --scope user -- llm-council

Then in Claude Code:

Consult the LLM council about best practices for error handling

With Claude Desktop

First ensure your API key is available (via keychain, environment variable, or .env file).

Add to ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "llm-council": {
      "command": "llm-council"
    }
  }
}

Note: No env block needed—the key is resolved from your system keychain or environment automatically.

With Other MCP Clients

Any MCP client can use the server by running:

llm-council

Available Tools

consult_council

Ask the LLM council a question and get synthesized guidance.

Arguments:

• query (string, required): The question to ask the council
• confidence (string, optional): Response quality level (default: "high")
  • "quick": Fast models (mini/flash/haiku), ~30 seconds - fast responses for simple questions
  • "balanced": Mid-tier models (GPT-4o, Sonnet), ~90 seconds - good balance of speed and quality
  • "high": Full council (Opus, GPT-4o), ~180 seconds - comprehensive deliberation
  • "reasoning": Deep thinking models (GPT-5.2, o1, DeepSeek-R1), ~600 seconds - complex reasoning
• include_details (boolean, optional): Include individual model responses and rankings (default: false)
• verdict_type (string, optional): Type of verdict to render (default: "synthesis")
  • "synthesis": Free-form natural language synthesis
  • "binary": Go/no-go decision (approved/rejected) with confidence score
  • "tie_breaker": Chairman resolves deadlocked decisions
• include_dissent (boolean, optional): Extract minority opinions from Stage 2 (default: false)

Example:

Use consult_council to ask: "What are the trade-offs between microservices and monolithic architecture?"

Example with confidence level:

Use consult_council with confidence="quick" to ask: "What's the syntax for a Python list comprehension?"

Example with Jury Mode (binary verdict):

Use consult_council with verdict_type="binary" and include_dissent=true to ask: "Should we approve this PR that adds caching?"

council_health_check

Verify the council is working before expensive operations. Returns API connectivity status, configured models, and estimated response times.

Arguments: None

Returns:

• api_key_configured: Whether an API key was found
• key_source: Where the key came from ("environment", "keychain", or "config_file")
• council_size: Number of models in the council
• estimated_duration: Expected response times for each confidence level
• ready: Whether the council is ready to accept queries

Example:

Run council_health_check to verify the LLM council is working

How It Works

The council uses a multi-stage process inspired by ensemble methods and peer review:

User Query
    ↓
┌─────────────────────────────────────────────┐
│ STAGE 1: Independent Responses              │
│ • All council models queried in parallel    │
│ • No knowledge of other responses           │
│ • Graceful degradation if some fail         │
└─────────────────────────────────────────────┘
    ↓
┌─────────────────────────────────────────────┐
│ STAGE 1.5: Style Normalization (optional)   │
│ • Rewrites responses in neutral style       │
│ • Removes AI preambles and fingerprints     │
│ • Strengthens anonymization                 │
└─────────────────────────────────────────────┘
    ↓
┌─────────────────────────────────────────────┐
│ STAGE 2: Anonymous Peer Review              │
│ • Responses labeled A, B, C (randomized)    │
│ • XML sandboxing prevents prompt injection  │
│ • JSON-structured rankings with scores      │
│ • Self-votes excluded from aggregation      │
└─────────────────────────────────────────────┘
    ↓
┌─────────────────────────────────────────────┐
│ STAGE 3: Chairman Synthesis                 │
│ • Receives all responses + rankings         │
│ • Consensus mode: single best answer        │
│ • Debate mode: highlights disagreements     │
└─────────────────────────────────────────────┘
    ↓
Final Response + Metadata

This approach helps surface diverse perspectives, identify consensus, and produce more balanced, well-reasoned answers.

Advanced Features

Self-Vote Exclusion

By default, each model's vote for its own response is excluded from the aggregate rankings. This prevents self-preference bias.

export LLM_COUNCIL_EXCLUDE_SELF_VOTES=true  # default

Synthesis Modes

Consensus Mode (default): Chairman synthesizes a single best answer.

Debate Mode: Chairman highlights areas of agreement, key disagreements, and trade-offs between perspectives.

export LLM_COUNCIL_MODE=debate

Style Normalization (Stage 1.5)

Optional preprocessing that rewrites all responses in a neutral style before peer review. This strengthens anonymization by removing stylistic "fingerprints" that might allow models to recognize each other.

export LLM_COUNCIL_STYLE_NORMALIZATION=true
export LLM_COUNCIL_NORMALIZER_MODEL=google/gemini-2.0-flash-001  # fast/cheap

Stratified Sampling (Large Councils)

For councils with more than 5 models, you can limit the number of reviewers per response to reduce API costs (O(N²) → O(N×k)):

export LLM_COUNCIL_MAX_REVIEWERS=3

Reliability Features

The council includes built-in reliability features for long-running operations:

Tiered Timeouts: Graceful degradation under time pressure:

• Per-model soft deadline: 15s (start planning fallback)
• Per-model hard deadline: 25s (abandon slow model)
• Global synthesis trigger: 40s (must start synthesis)
• Response deadline: 50s (must return something)

Partial Results: If some models timeout, the council returns results from the models that responded, with a clear warning indicating which models were excluded.

Confidence Levels: Use the confidence parameter to trade off speed vs. thoroughness:

• quick: ~20-30 seconds (fastest models)
• balanced: ~45-60 seconds (most models)
• high: ~60-90 seconds (full council, default)

Progress Feedback: During deliberation, progress updates show which models have responded and which are still pending:

✓ claude-opus-4.5 (1/4) | waiting: gpt-5.1, gemini-3-pro, grok-4
✓ gemini-3-pro (2/4) | waiting: gpt-5.1, grok-4

Jury Mode (ADR-025b)

Transform the council from a "summary generator" to a "decision engine" with structured verdicts.

Verdict Types:

Binary Verdict Mode:

Returns structured go/no-go decisions with confidence scores:

# MCP Tool
result = await consult_council(
    query="Should we approve this architectural change?",
    verdict_type="binary"
)

# Returns:
{
  "verdict": "approved",      # or "rejected"
  "confidence": 0.75,         # 0.0-1.0 based on council agreement
  "rationale": "Council agreed the change improves modularity..."
}

Tie-Breaker Mode:

When the council is deadlocked (top scores within 0.1 of each other), the chairman casts the deciding vote:

result = await consult_council(
    query="Option A vs Option B for caching strategy?",
    verdict_type="binary"
)

# If deadlocked, returns:
{
  "verdict": "approved",
  "confidence": 0.60,
  "rationale": "Chairman resolved deadlock based on...",
  "deadlocked": true  # Flag indicates tie-breaker was needed
}

Constructive Dissent:

Extract minority opinions from Stage 2 peer reviews:

result = await consult_council(
    query="Should we migrate to microservices?",
    verdict_type="binary",
    include_dissent=True
)

# Returns:
{
  "verdict": "approved",
  "confidence": 0.70,
  "rationale": "Majority supports migration...",
  "dissent": "Minority perspective: One reviewer noted scalability concerns with current team size."
}

Dissent Algorithm:

1. Collect all scores from Stage 2 peer reviews
2. Calculate median and standard deviation per response
3. Identify outliers (score < median - 1.5 × std)
4. Extract evaluation text from outliers
5. Format as minority perspective

Example: CI/CD Gate Integration:

import asyncio
from llm_council.council import run_full_council
from llm_council.verdict import VerdictType

async def review_pull_request(pr_diff: str, pr_description: str):
    """Use LLM Council as a PR review gate."""
    query = f"""
    Review this pull request and determine if it should be approved.

    Description: {pr_description}

    Changes:
    {pr_diff}
    """

    stage1, stage2, stage3, metadata = await run_full_council(
        query,
        verdict_type=VerdictType.BINARY,
        include_dissent=True
    )

    verdict = metadata.get("verdict", {})

    if verdict.get("verdict") == "approved" and verdict.get("confidence", 0) >= 0.7:
        return True, verdict.get("rationale")
    else:
        return False, f"{verdict.get('rationale')}\n\nDissent: {verdict.get('dissent', 'None')}"

Environment Variables:

Structured Rubric Scoring (ADR-016)

By default, reviewers provide a single 1-10 holistic score. With rubric scoring enabled, reviewers score each response on five dimensions:

Accuracy Ceiling: When enabled (default), accuracy acts as a ceiling on the overall score:

• Accuracy < 5: Score caps at 4.0 (40%) — "significant errors or worse"
• Accuracy 5-6: Score caps at 7.0 (70%) — "mixed accuracy"
• Accuracy ≥ 7: No ceiling — "mostly accurate or better"

This prevents well-written hallucinations from ranking well.

Scoring Anchors: Each score level has defined behavioral meaning (see ADR-016):

• 9-10: Excellent (completely accurate, comprehensive, crystal clear)
• 7-8: Good (mostly accurate, covers main points)
• 5-6: Mixed (some errors or gaps)
• 3-4: Poor (significant issues)
• 1-2: Failing (fundamentally flawed)

# Enable rubric scoring
export LLM_COUNCIL_RUBRIC_SCORING=true

# Customize weights (must sum to 1.0)
export LLM_COUNCIL_WEIGHT_ACCURACY=0.40
export LLM_COUNCIL_WEIGHT_RELEVANCE=0.10
export LLM_COUNCIL_WEIGHT_COMPLETENESS=0.20
export LLM_COUNCIL_WEIGHT_CONCISENESS=0.10
export LLM_COUNCIL_WEIGHT_CLARITY=0.20

Safety Gate (ADR-016)

When enabled, a pass/fail safety check runs before rubric scoring to filter harmful content:

Responses that fail safety checks are capped at score 0 regardless of other dimension scores. Educational/defensive content is context-aware and allowed.

# Enable safety gate
export LLM_COUNCIL_SAFETY_GATE=true

# Customize score cap (default: 0)
export LLM_COUNCIL_SAFETY_SCORE_CAP=0.0

Bias Auditing (ADR-015)

When enabled, the council reports per-session bias indicators from peer review scoring:

Output: The metadata includes a bias_audit object with:

• length_score_correlation: Pearson correlation coefficient
• length_bias_detected: Boolean flag
• position_score_variance: Variance of mean scores by presentation position
• position_bias_detected: Boolean flag (derived from anonymization labels A, B, C...)
• harsh_reviewers / generous_reviewers: Lists of biased reviewers
• overall_bias_risk: "low", "medium", or "high"

Important Limitations: With only 4-5 models per session, these metrics have limited statistical power:

• Length correlation with n=5 data points can only detect extreme biases
• Position bias from a single ordering cannot distinguish position effects from quality differences
• Reviewer calibration is relative to the current session only

These are per-session indicators (red flags for extreme anomalies), not statistically robust proof of systematic bias. Interpret with appropriate caution.

# Enable bias auditing
export LLM_COUNCIL_BIAS_AUDIT=true

# Customize thresholds (optional)
export LLM_COUNCIL_LENGTH_CORRELATION_THRESHOLD=0.3
export LLM_COUNCIL_POSITION_VARIANCE_THRESHOLD=0.5

Cross-Session Bias Aggregation (ADR-018)

For statistically meaningful bias detection across multiple sessions, enable bias persistence:

# Enable bias persistence (stores metrics locally)
export LLM_COUNCIL_BIAS_PERSISTENCE=true

# Consent level: 0=off, 1=local only (default), 2=anonymous, 3=enhanced, 4=research
export LLM_COUNCIL_BIAS_CONSENT=1

# Store path (default: ~/.llm-council/bias_metrics.jsonl)
export LLM_COUNCIL_BIAS_STORE=~/.llm-council/bias_metrics.jsonl

Generate cross-session bias reports:

# Text report (default)
llm-council bias-report

# JSON output
llm-council bias-report --format json

# Include detailed reviewer profiles
llm-council bias-report --verbose

# Limit to last 50 sessions
llm-council bias-report --sessions 50

Statistical confidence tiers:

Output Quality Metrics (ADR-036)

Quantify the reliability and quality of council outputs with three core metrics:

CSS Interpretation:

SAS Components:

• winner_alignment: Similarity to top-ranked responses
• max_source_alignment: Best match to any response
• hallucination_risk: 1 - max_source_alignment
• grounded: True if synthesis traces to sources (threshold: 0.6)

Usage:

Quality metrics are automatically included in metadata:

stage1, stage2, stage3, metadata = await run_full_council(query)

quality = metadata.get("quality_metrics", {})
core = quality.get("core", {})

print(f"Consensus: {core['consensus_strength']:.2f}")
print(f"Depth: {core['deliberation_depth']:.2f}")
print(f"Grounded: {core['synthesis_attribution']['grounded']}")

MCP Tool Output:

The consult_council MCP tool displays quality metrics with visual bars:

### Quality Metrics
- **Consensus Strength**: 0.82 [████████░░]
- **Deliberation Depth**: 0.74 [███████░░░]
- **Synthesis Grounded**: ✓ (alignment: 0.89)

Environment Variables:

# Enable/disable quality metrics (default: true)
export LLM_COUNCIL_QUALITY_METRICS=true

# Quality tier: core (OSS), standard, enterprise
export LLM_COUNCIL_QUALITY_TIER=core

Gateway Layer (ADR-023)

The gateway layer provides an abstraction over LLM API requests with multiple gateway options:

Available Gateways:

Core Features:

• Circuit Breaker: Prevents cascading failures (CLOSED → OPEN → HALF_OPEN → CLOSED)
• Fallback Chains: Automatic retry with secondary gateways on failure
• Per-Gateway Metrics: Track failure counts, latency, and health status
• BYOK Support: Bring Your Own Key for Requesty and Direct gateways

Basic Usage:

from llm_council.gateway import (
    GatewayRouter, GatewayRequest, CanonicalMessage, ContentBlock,
    OpenRouterGateway, RequestyGateway, DirectGateway
)

# Single gateway (OpenRouter)
router = GatewayRouter()

# Multi-gateway with fallback
router = GatewayRouter(
    gateways={
        "openrouter": OpenRouterGateway(),
        "requesty": RequestyGateway(byok_enabled=True, byok_keys={"anthropic": "sk-ant-..."}),
        "direct": DirectGateway(provider_keys={"openai": "sk-..."}),
    },
    default_gateway="openrouter",
    fallback_chains={"openrouter": ["requesty", "direct"]}
)

request = GatewayRequest(
    model="openai/gpt-4o",
    messages=[CanonicalMessage(role="user", content=[ContentBlock(type="text", text="Hello")])]
)
response = await router.complete(request)

Enable the gateway layer:

export LLM_COUNCIL_USE_GATEWAY=true

# Optional: Configure specific gateways
export REQUESTY_API_KEY=your-requesty-key    # For Requesty
export ANTHROPIC_API_KEY=sk-ant-...           # For Direct (Anthropic)
export OPENAI_API_KEY=sk-...                  # For Direct (OpenAI)
export GOOGLE_API_KEY=...                     # For Direct (Google)

Circuit Breaker Behavior:

• Default: 5 failures to trip the circuit
• Recovery timeout: 60 seconds
• Half-open state allows test requests to check recovery
• Open circuits are skipped in fallback chain

The gateway layer is currently opt-in (default: disabled) for backward compatibility.

Triage Layer (ADR-020)

The triage layer provides query classification, model selection optimizations, and confidence-gated routing:

• Confidence-Gated Fast Path: Routes simple queries to a single model, escalating to full council when confidence is low
• Shadow Council Sampling: Random 5% sampling validates fast path quality against full council
• Rollback Monitoring: Automatic rollback when disagreement/escalation rates breach thresholds
• Wildcard Selection: Adds domain-specialized models to the council based on query classification
• Prompt Optimization: Per-model prompt adaptation (Claude gets XML, OpenAI gets Markdown)
• Complexity Classification: Heuristic-based with optional Not Diamond API integration

Domain Categories:

Enable wildcard selection:

export LLM_COUNCIL_WILDCARD_ENABLED=true

This automatically adds a domain specialist to the council based on query classification. For example, a Python coding question will add a DeepSeek model alongside the default council.

Enable prompt optimization:

export LLM_COUNCIL_PROMPT_OPTIMIZATION_ENABLED=true

This applies per-model prompt formatting. Claude receives XML-structured prompts, while other providers receive their preferred format.

Enable confidence-gated fast path:

export LLM_COUNCIL_FAST_PATH_ENABLED=true
export LLM_COUNCIL_FAST_PATH_CONFIDENCE_THRESHOLD=0.92  # default

When enabled, simple queries are routed to a single model. If the model's confidence is below the threshold, the query automatically escalates to the full council. This can reduce costs by 45-55% on simple queries while maintaining quality.

Fast Path Quality Monitoring:

The fast path includes built-in quality monitoring:

Configure monitoring:

export LLM_COUNCIL_SHADOW_SAMPLE_RATE=0.05  # 5% shadow sampling
export LLM_COUNCIL_ROLLBACK_ENABLED=true
export LLM_COUNCIL_ROLLBACK_WINDOW=100  # rolling window size

Optional Not Diamond Integration:

For advanced model routing, integrate with Not Diamond:

export NOT_DIAMOND_API_KEY="your-key"
export LLM_COUNCIL_USE_NOT_DIAMOND=true

When Not Diamond is unavailable, the system gracefully falls back to heuristic-based classification.

The triage layer is currently opt-in (default: disabled) for backward compatibility.

Local LLM Support (ADR-025)

Run council deliberations entirely on local hardware using Ollama:

# Install with Ollama support
pip install "llm-council-core[ollama]"

# Start Ollama (if not already running)
ollama serve

# Pull a model
ollama pull llama3.2

Use local models in your council:

# Mix local and cloud models
export LLM_COUNCIL_MODELS="ollama/llama3.2,openai/gpt-4o,anthropic/claude-3-5-sonnet"

# Or use only local models (air-gapped mode)
export LLM_COUNCIL_MODELS="ollama/llama3.2,ollama/mistral,ollama/codellama"
export LLM_COUNCIL_CHAIRMAN="ollama/llama3.2"

Hardware Requirements:

Quality Degradation Notice: Local models typically have reduced capabilities compared to cloud-hosted frontier models. The gateway includes quality notices in responses to inform users when local models are used.

Configuration:

# Ollama endpoint (default: http://localhost:11434)
export LLM_COUNCIL_OLLAMA_BASE_URL=http://localhost:11434

# Timeout for local models (default: 120s - first load can be slow)
export LLM_COUNCIL_OLLAMA_TIMEOUT=120.0

Webhook Notifications (ADR-025)

Receive real-time notifications as the council deliberates:

from llm_council.webhooks import WebhookConfig, WebhookDispatcher, WebhookPayload

# Configure webhook endpoint
config = WebhookConfig(
    url="https://your-server.com/webhook",
    events=["council.complete", "council.error"],
    secret="your-hmac-secret"  # Optional, for signature verification
)

# Dispatch events (used internally by council)
dispatcher = WebhookDispatcher()
result = await dispatcher.dispatch(config, payload)

Event Types:

HMAC Signature Verification:

Webhooks are signed using HMAC-SHA256 for security:

from llm_council.webhooks import verify_webhook_request

# Verify incoming webhook (in your server)
is_valid = verify_webhook_request(
    payload=request.body,
    headers=request.headers,
    secret="your-hmac-secret"
)

Headers included:

• X-Council-Signature: sha256=<hex-digest>
• X-Council-Timestamp: Unix timestamp
• X-Council-Version: 1.0

Configuration:

# Webhook timeout (default: 5s)
export LLM_COUNCIL_WEBHOOK_TIMEOUT=5.0

# Max retry attempts (default: 3)
export LLM_COUNCIL_WEBHOOK_RETRIES=3

# Require HTTPS (except localhost) - default: true
export LLM_COUNCIL_WEBHOOK_HTTPS_ONLY=true

SSE Streaming (ADR-025)

Stream council events in real-time using Server-Sent Events.

Built-in HTTP Server:

The library includes a built-in HTTP server with SSE streaming:

# Install with HTTP server support
pip install "llm-council-core[http]"

# Start the server
llm-council serve

The SSE endpoint is available at GET /v1/council/stream:

# Stream council deliberation
curl -N "http://localhost:8000/v1/council/stream?prompt=What+is+AI"

Custom Integration:

For custom FastAPI/Starlette applications:

from llm_council.webhooks import (
    council_event_generator,
    SSE_CONTENT_TYPE,
    get_sse_headers
)

# In your FastAPI/Starlette endpoint
@app.get("/v1/council/stream")
async def stream_council(prompt: str):
    return StreamingResponse(
        council_event_generator(prompt, models=None, api_key=None),
        media_type=SSE_CONTENT_TYPE,
        headers=get_sse_headers()
    )

Client-side (JavaScript):

const source = new EventSource('/v1/council/stream?prompt=...');

source.addEventListener('council.deliberation_start', (e) => {
  console.log('Started:', JSON.parse(e.data));
});

source.addEventListener('council.complete', (e) => {
  const result = JSON.parse(e.data);
  console.log('Final answer:', result.stage3_response);
  source.close();
});

Offline Mode (ADR-026)

Run LLM Council without any external metadata calls using the bundled model registry:

# Enable offline mode
export LLM_COUNCIL_OFFLINE=true

When offline mode is enabled:

• Uses StaticRegistryProvider exclusively with 31 bundled models
• No external API calls for metadata (context windows, pricing, capabilities)
• All core council operations continue to work
• Unknown models use safe defaults (4096 context window)

This implements the "Sovereign Orchestrator" philosophy: the system must function as a complete, independent utility without external dependencies.

Bundled Models (31 total):

Model Metadata API:

from llm_council.metadata import get_provider

# Get the metadata provider
provider = get_provider()

# Query model information
info = provider.get_model_info("openai/gpt-4o")
print(f"Context window: {info.context_window}")  # 128000
print(f"Quality tier: {info.quality_tier}")      # frontier

# Check capabilities
window = provider.get_context_window("anthropic/claude-opus-4.5")  # 200000
supports = provider.supports_reasoning("openai/o1")  # True

# List all available models
models = provider.list_available_models()  # 31 models

Agent Skills (ADR-034)

LLM Council includes agent skills for AI-assisted code verification, review, and CI/CD quality gates. Skills use progressive disclosure to minimize token usage while providing detailed scoring rubrics when needed.

Available Skills:

Exit Codes for CI/CD:

# In GitHub Actions or any CI pipeline
llm-council gate --snapshot $GITHUB_SHA --rubric-focus Security

# Exit codes:
# 0 = PASS (confidence >= threshold, no blocking issues)
# 1 = FAIL (blocking issues present)
# 2 = UNCLEAR (needs human review)

Skills are located in .github/skills/ and work with Claude Code, VS Code Copilot, Cursor, and other MCP-compatible clients.

For detailed documentation, see the Skills Guide.

All Environment Variables

Gateway Configuration (ADR-023)

Ollama Configuration (ADR-025)

Webhook Configuration (ADR-025)

Council Configuration

Credits & Attribution Continued

This project is a derivative work based on the original llm-council by Andrej Karpathy.

Original Work:

• Concept and 3-stage council orchestration: Andrej Karpathy
• Core council logic (Stage 1-3 process)
• OpenRouter integration

Derivative Work by Amiable:

• MCP (Model Context Protocol) server implementation
• Removal of web frontend (focus on MCP functionality)
• Python package structure for PyPI distribution
• User-configurable model selection
• Enhanced features (style normalization, self-vote exclusion, synthesis modes)
• Test suite and modern packaging standards

License

MIT License - see LICENSE file for details.