Vault MCP Bridge

Overview

FastAPI MCP-compatible server that manages agent-scoped secrets and crypto via HashiCorp Vault.
Auth: API Key, JWT (HS256 and RS256/JWKS), and mTLS via reverse-proxy headers.
Per-agent namespacing in KV v2; Transit support (encrypt/decrypt/sign/verify/rewrap/random).
Optional per-request child token issuance bound to per-agent policies; simple per-agent rate limiting.
Prometheus metrics at /metrics and optional OpenTelemetry tracing.

Quickstart

Python env:
- python3 -m venv .venv && source .venv/bin/activate
- python -m pip install -r requirements.txt
Start local Vault (dev) for testing:
- cd local-vault && docker compose up -d && cd ..
- This provisions KV v2 at secret/, a Transit key, and example policies. See local-vault/README.md.
Run server:
- Easiest: python main.py (env: HOST=0.0.0.0 PORT=8089 RELOAD=true LOG_LEVEL=debug)
- Or: python -m uvicorn main:app --reload
- Or factory: python -m uvicorn vault_mcp.app:create_app --factory --reload
Helper: scripts/run_dev_jwt.sh starts the app with Vault + JWT defaults (no manual env export).
Sanity checks:
- curl http://127.0.0.1:8089/healthz
- bash scripts/smoke.sh (expects local Vault and default dev auth)
Vault access (required before running the server):
- Set credentials so the app can authenticate to Vault, e.g. for the bundled dev compose stack:
```
export VAULT_ADDR=http://127.0.0.1:8200
export VAULT_TOKEN=root   # replace with your own token or AppRole
```
- Without these env vars (or the AppRole equivalents) /readyz will return 503 and observability/secret routes will be unauthorized.
- To avoid re-exporting each run you can source config/dev-jwt.env (new sample file) before launching, or use scripts/run_dev_jwt.sh which applies the same defaults automatically.
Optional admin UI:
- python3 -m venv .ui-venv && source .ui-venv/bin/activate
- pip install -r ui/requirements.txt
- streamlit run ui/streamlit_app.py
- Use the Manage Subjects & Keys page to manage config/users.json, rotate credentials, and sync profiles to the sidebar.
- JWT helpers in the UI require JWT_HS256_SECRET to be set before launch (e.g., source config/dev-jwt.env or run scripts/run_dev_jwt.sh).
- Use AWS KMS Operations to supply temporary AWS credentials (if needed) and call encrypt/decrypt, data-key, sign, and verify APIs.

JWT quickstart (HS256):

Install helper deps: pip install 'python-jose[cryptography]'

Generate a token with the bundled script (adjust subject/scopes as needed):

python scripts/gen_jwt.py \
  --secret dev-secret \
  --issuer mcp-auth \
  --audience mcp-agents \
  --sub agent_api \
  --scopes read,write,delete,list \
  --ttl 600

Use the printed value as the Authorization: Bearer <token> header when calling the API or configuring the Streamlit console.

Features

KV v2 secret CRUD with per-agent prefixes and safe pathing.
Transit: encrypt/decrypt, sign/verify, rewrap, and random bytes (base64/hex).
AWS KMS (optional): encrypt/decrypt, generate data keys, and sign/verify with native AWS keys.
Database: dynamic credentials issuance and lease management.
SSH: OTP credential and SSH certificate signing.
Auth: API Key, JWT (HS256 or RS256 via JWKS), mTLS via headers.
Child tokens per request (optional); per-agent in-memory rate limiting.
MCP: JSON-RPC over HTTP at POST /mcp/rpc (with GET /mcp/sse keepalive channel) and stdio transport via scripts/mcp_stdio.py.
Streamlit operations hub: Vault Operations covers Secrets, Transit, Database, SSH, and MCP tools, while a dedicated AWS KMS page lets you enter temporary AWS credentials and invoke encrypt/decrypt, data-key, and signing APIs.
Streamlit agent admin: create multiple AI agent profiles, toggle LLM usage, assign credentials (linked user/API key/JWT), define tasks, capture a secrets_backend plan (Vault/KMS/Hybrid), and monitor progress/status.
MCP lifecycle: initialize, tools/list, resources/list, prompts/list, tools/call, shutdown. Protocol version: 2025-06-18.
Tools exposed (with required scopes):
- KV: kv.read (read, supports version), kv.write (write), kv.list (list), kv.delete (delete), kv.undelete (write), kv.destroy (write)
- Transit: transit.encrypt (write), transit.decrypt (read), transit.sign (write), transit.verify (read), transit.rewrap (write), transit.random (read)
- DB: db.issue_creds (write), db.renew (write), db.revoke (write)
- SSH: ssh.otp (write), ssh.sign (write)
- KMS: kms.encrypt (write), kms.decrypt (read), kms.generate_data_key (write), kms.sign (write), kms.verify (read)
Observability endpoints: /observability/summary (Vault/API status + in-flight requests) and /observability/logs/{requests|responses|server} (tail JSON logs, read scope required).
Metrics at /metrics; optional OpenTelemetry via OTLP HTTP exporter.

Resources

Scheme kv://{subject}/{path} with optional ?version=N.
resources/list: advertises kv://{subject}/ (KV root) for the authenticated subject.
resources/get:
- kv://{subject}/foo/bar returns { data, version } (JSON) for that KV path.
- kv://{subject}/foo/ (trailing slash) returns { keys: [...] } listing under that prefix.
- Cross-subject access is forbidden.

Prompts

prompts/list: returns prompt specs for kv_read and kv_write with input schemas.
prompts/get:
- kv_read: returns example messages and a suggested_tool call for kv.read.
- kv_write: returns example messages and a suggested_tool call for kv.write.

Configuration (env)

Vault:
- VAULT_ADDR (default: http://localhost:8200)
- VAULT_NAMESPACE (Enterprise only)
- VAULT_TOKEN or VAULT_ROLE_ID + VAULT_SECRET_ID
- KV_MOUNT (default: secret)
- DEFAULT_PREFIX (default: mcp)
Config file (optional, no .env needed):
- Set APP_CONFIG_FILE to a JSON/TOML/YAML file path. Example defaults auto-detected from CWD: config.toml, config.json, config.yaml.
- Environment variables always override file values.
- Note: .env files are not auto-loaded anymore.
- Precedence: runtime args (where applicable) → environment variables → APP_CONFIG_FILE/auto-detected config → built-in defaults.
Auth enable flags:
- AUTH_API_KEY_ENABLED (default: true)
- AUTH_JWT_ENABLED (default: true)
- AUTH_MTLS_ENABLED (default: false)
- CLI helper: scripts/manage_user.py create <subject> writes metadata to config/users.json and prints policy/API key export commands.
- UI helper: in Streamlit, check “Generate JWT token” to issue a token during user creation. Generation failures show inline errors and the user entry is not written.
API Keys:
- API_KEYS_JSON JSON map: { "<api-key>": "<agent-id>" }
JWT:
- Common: JWT_ISSUER (default: mcp-auth), JWT_AUDIENCE (default: mcp-agents)
- HS256: JWT_HS256_SECRET
- RS256/JWKS: JWT_JWKS_URL or JWT_JWKS_FILE, JWT_JWKS_CACHE_SECONDS (default: 300), JWT_REQUIRE_KID (default: false)
- Validation toggles: JWT_VALIDATE_ISSUER (default: true), JWT_VALIDATE_AUDIENCE (default: true)
- Helper: python scripts/gen_jwt.py --secret <JWT_HS256_SECRET> --sub <agent> for quick dev tokens (see Quickstart example above).
- Metadata persisted per user: jwt_created_at, jwt_expires_at, jwt_ttl_seconds. The Streamlit Current users grid surfaces status, timestamps, TTL seconds, and offers CSV export.
mTLS via proxy headers:
- MTLS_IDENTITY_HEADER (default: x-ssl-client-s-dn)
- MTLS_VERIFY_HEADER (default: x-ssl-client-verify)
- MTLS_SUBJECT_CN_PREFIX (default: CN=)
Child token issuance:
- CHILD_TOKEN_ENABLED (default: false)
- CHILD_TOKEN_TTL (default: 90s)
- CHILD_TOKEN_POLICY_PREFIX (default: mcp-agent-)
Rate limiting:
- RATE_LIMIT_ENABLED (default: true)
- RATE_LIMIT_REQUESTS (default: 60)
- RATE_LIMIT_WINDOW_SECONDS (default: 60)
Server behavior:
- HOST, PORT, RELOAD, LOG_LEVEL for python main.py
- Logs directory: LOG_DIR (default: ./logs)
- EXPOSE_REST_ROUTES (default: true) — when false, disables REST feature routers (/secrets, /transit, /db, /ssh, /whoami) for MCP‑only deployments.
- AWS KMS (optional):
  - AWS_KMS_ENABLED (default: false)
  - AWS_REGION (required when enabling KMS unless provided by IAM role/default profile)
  - AWS_KMS_DEFAULT_KEY_ID (optional convenience default for encrypt/data-key/sign requests)
  - AWS_KMS_ENDPOINT (optional; point to LocalStack or custom endpoint)
  - AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN (optional; falls back to standard AWS credential resolution)
Observability:
- Prometheus at /metrics (always enabled)
- OpenTelemetry: OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_SERVICE_NAME (optional)

Auth Modes

API Key: send X-API-Key: <key>. Map keys to agents via API_KEYS_JSON.
JWT: send Authorization: Bearer <token> with sub and optional scopes.
mTLS: terminate TLS at proxy and pass DN via X-SSL-Client-S-DN; CN is used as subject.

Agent Path Namespace

Secrets live under {KV_MOUNT}/data/{DEFAULT_PREFIX}/{subject}/... (KV v2). The server enforces safe relative paths within the agent prefix.

Child Token Issuance

If CHILD_TOKEN_ENABLED=true, a child token is minted per request with policy {CHILD_TOKEN_POLICY_PREFIX}{subject} and TTL CHILD_TOKEN_TTL.
Ensure the policy exists and the parent token can create child tokens.

Policy

Generate HCL for an agent:
- python scripts/gen_policy.py --agent alice --mount secret --prefix mcp > alice.hcl
- Suggested policy name: mcp-agent-alice
The policy grants CRUD/list on data/{prefix}/{agent}/*, list/read on metadata/{prefix}/{agent}/*, and versioned ops on delete/undelete/destroy.
Apply with the Vault CLI: vault policy write mcp-agent-alice alice.hcl

Endpoints

KV v2
- PUT /secrets/{path} — write (scope: write)
- GET /secrets/{path} — read (scope: read) [query version optional]
- DELETE /secrets/{path} — delete latest version (scope: delete)
- GET /secrets?prefix=... — list keys (scope: list)
- POST /secrets/{path}:undelete — body { "versions": [1,2] } (scope: write)
- POST /secrets/{path}:destroy — body { "versions": [1,2] } (scope: write)
Transit
- POST /transit/encrypt — { "key": "k", "plaintext": "<b64>" } (scope: write)
- POST /transit/decrypt — { "key": "k", "ciphertext": "..." } (scope: read)
- POST /transit/sign — { key, input, hash_algorithm?, signature_algorithm? } (scope: write)
- POST /transit/verify — { key, input, signature, hash_algorithm? } (scope: read)
- POST /transit/rewrap — { key, ciphertext } (scope: write)
- GET /transit/random?bytes=32&format=base64|hex (scope: read)
KMS endpoints are always exposed for development but return 503 unless AWS_KMS_ENABLED=true
- POST /kms/encrypt — { plaintext: <b64>, key_id?, encryption_context?, grant_tokens? } (scope: write)
- POST /kms/decrypt — { ciphertext: <b64>, encryption_context?, grant_tokens? } (scope: read)
- POST /kms/data-key — { key_id?, key_spec? | number_of_bytes?, encryption_context?, grant_tokens? } (scope: write)
- POST /kms/sign — { key_id?, message? | message_digest?, signing_algorithm, message_type?, grant_tokens? } (scope: write)
- POST /kms/verify — { key_id?, signature: <b64>, message? | message_digest?, signing_algorithm, message_type?, grant_tokens? } (scope: read)
Database
- POST /db/creds/{role} — issue dynamic DB creds (scope: write)
- POST /db/renew — { lease_id, increment? } (scope: write)
- POST /db/revoke — { lease_id } (scope: write)
SSH
- POST /ssh/otp — { role, ip, username, port? } (scope: write)
- POST /ssh/sign — { role, public_key, cert_type?, valid_principals?, ttl? } (scope: write)
Health/Debug/Metrics
GET /healthz, /livez, /readyz, /whoami, /echo-headers, /metrics
- /readyz returns a JSON body detailing Vault (ok/detail) and, when enabled, AWS KMS readiness. Expect HTTP 503 with explanatory messages when Vault credentials are missing or KMS is misconfigured.
MCP
- Mounted at /mcp when fastapi-mcp is available

API Docs

Swagger UI: http://127.0.0.1:8089/docs
ReDoc: http://127.0.0.1:8089/redoc
OpenAPI: http://127.0.0.1:8089/openapi.json

MCP Usage

HTTP JSON-RPC: POST /mcp/rpc with JSON-RPC 2.0 messages; authenticate same as REST (API key / JWT / mTLS).
SSE channel: GET /mcp/sse provides periodic keepalives for server→client messaging (placeholder; extend as needed).
stdio: SUBJECT=agentA python scripts/mcp_stdio.py and write newline-delimited JSON-RPC messages to stdin.
Initialize result includes protocolVersion: 2025-06-18, basic capabilities, and lists tools/resources/prompts.
SSE events: server emits tool.completed events with {type, tool, subject, ts}; keepalives every 15s.

MCP Inspector

An official, interactive UI for MCP servers (Swagger-like, but for MCP) that discovers tools/resources/prompts and lets you call them live.
Connect via HTTP:
- RPC URL: http://127.0.0.1:8089/mcp/rpc
- SSE URL (optional): http://127.0.0.1:8089/mcp/sse
- Auth headers: add X-API-Key: dev-api-key (or Authorization: Bearer <JWT>) in the Inspector’s connection settings.
- If connecting from the hosted Inspector (HTTPS) to your local HTTP server, enable CORS:
  - export CORS_ALLOW_ORIGINS=https://inspector.modelcontextprotocol.io
  - Consider exposing your server via HTTPS (e.g., ngrok) to avoid mixed-content blocking.
Or connect via stdio:
- Command: SUBJECT=agent_api python scripts/mcp_stdio.py
- Inspector will spawn the process and speak JSON-RPC over stdio.
Once connected, Inspector should list the available tools: kv.read, kv.write, kv.list, kv.delete, kv.undelete, kv.destroy.
Current state: Resources/Prompts are empty; SSE sends keepalives only.

Troubleshooting Inspector

ModuleNotFoundError: vault_mcp when running stdio: ensure you run from repo root, or use SUBJECT=agent_api PYTHONPATH=$(pwd) python scripts/mcp_stdio.py. The script now auto-adds repo root to sys.path.
CORS errors in the browser: set CORS_ALLOW_ORIGINS=https://inspector.modelcontextprotocol.io (comma separate multiple origins) and restart the server.
Mixed content blocked: use an HTTPS tunnel to your local server (e.g., ngrok http 8089) and switch Inspector URLs to https.

Prometheus & OpenTelemetry

Prometheus endpoint: GET /metrics (text). Quick check: curl -s http://127.0.0.1:8089/metrics | head.
Metrics include http_requests_total and http_request_duration_seconds with labels method, route, status.
Additional telemetry: http_requests_with_correlation_total counts requests that include or receive a correlation ID. Every HTTP response returns X-Correlation-Id; if OTEL tracing is enabled, X-Trace-Id is also emitted.
OpenTelemetry tracing (optional): set OTEL_EXPORTER_OTLP_ENDPOINT (e.g., http://localhost:4318/v1/traces) and OTEL_SERVICE_NAME (default: vault-mcp).
Structured logs: JSON files under ./logs/ (requests.log, responses.log, server.log). Tail with tail -f logs/requests.log.

Logging Details

Format: newline-delimited JSON. Core fields: ts, lvl, msg, logger plus context in extra.
Request logs (vault_mcp.request): include request_id, client, method, path, status, duration_ms.
Response/event logs (vault_mcp.response): per-endpoint keys, e.g.,
- kv_put|kv_get|kv_delete: subject, path, keys, version, request_id
- kv_list: subject, prefix, count, request_id
- transit_*: subject, key, size/validity hints, request_id
- db_* and ssh_*: high-level descriptors (e.g., role, lease_id_suffix, ip, user), never secret values
Request ID: responses include X-Request-Id; it is echoed in logs for correlation.
Example request log line:
- { "ts": "2024-01-01T10:00:00", "lvl": "info", "msg": "request", "logger": "vault_mcp.request", "request_id": "...", "client": "127.0.0.1", "method": "GET", "path": "/secrets", "status": 200, "duration_ms": 12 }

Examples (curl)

Write then read a secret (API key dev-key for agent agent_api):
- curl -X PUT -H 'X-API-Key: dev-key' -H 'Content-Type: application/json' \ -d '{"data": {"foo":"bar"}}' http://127.0.0.1:8089/secrets/configs/demo
- curl -H 'X-API-Key: dev-key' http://127.0.0.1:8089/secrets/configs/demo
Random bytes from Transit (hex):
- curl -H 'X-API-Key: dev-key' 'http://127.0.0.1:8089/transit/random?bytes=16&format=hex'
RS256/JWKS quick test:
- See local-vault/jwks/README.md for generating keys, running JWKS, and testing.

Local Dev Helpers

Start server with sensible dev env: bash scripts/run_dev.sh
Enable all auth regardless of current env: bash scripts/run_all_auth.sh
Smoke test (server + local Vault): bash scripts/smoke.sh
Auth tests by agent:
- API key (agent_api): bash scripts/test_agent_api.sh
- JWT HS256 (agent_jwt): bash scripts/test_agent_jwt.sh
- JWT RS256/JWKS (agent_jwt): bash scripts/test_agent_jwt_rs256.sh
- mTLS headers (agent_mtls): bash scripts/test_agent_mtls.sh

End-to-End and Example Agents

One-shot E2E (server + MCP HTTP + stdio + optional REST):
- LOG_LEVEL=DEBUG bash scripts/e2e_local.sh
Example MCP agents (HTTP JSON-RPC):
- API key: bash scripts/run_example_agent.sh
- JWT: bash scripts/run_example_agent.sh --jwt 'YOUR_JWT'
- mTLS headers: bash scripts/run_example_agent.sh --mtls
- No‑LLM direct client (no model provider): bash scripts/run_example_agent.sh --no-llm

Examples

LangChain agent that wraps these endpoints as tools:
- See examples/langchain_agent/README.md and examples/langchain_agent/agent.py

Testing

Run tests: pytest
Pytest overview:
- tests/test_health.py: Verifies basic liveness endpoints — GET /healthz and GET /livez return ok: true.
- tests/test_auth_and_kv.py: Exercises API‑key auth and KV v2 CRUD.
  - Uses a mocked hvac KV client to avoid real Vault.
  - Flow: PUT /secrets/configs/demo writes data, GET reads it back, DELETE removes it, subsequent GET returns 404.
  - Also checks GET /whoami returns the expected subject for X-API-Key.
- tests/test_transit_random.py: Tests Transit random byte generation endpoint with deterministic mock.
  - Monkeypatches client_for_principal to return a stub where generate_random_bytes is predictable.
  - Validates both format=hex and default base64 responses for GET /transit/random.
- tests/test_health_ready.py: Covers /readyz for authenticated, unauthenticated, Vault error, and generic error cases.
- tests/test_kv_extras.py: Covers GET /secrets?prefix=... list and version ops (:undelete, :destroy).
- tests/test_transit_endpoints.py: Covers /transit/encrypt|decrypt|sign|verify|rewrap with a transit stub.
- tests/test_db_and_ssh_routes.py: Covers /db/creds|renew|revoke and /ssh/otp|sign with stubs.
- tests/test_auth_modes.py: HS256 JWT /whoami (valid and bad aud), mTLS header success/fail.
- tests/test_auth_jwt_rs256_local.py: Local RS256: generates RSA + JWKS and validates /whoami via monkeypatched JWKS.
- tests/test_rate_limit_and_metrics.py: Verifies /metrics and rate limiting on /transit/random (429 on third call).
- tests/test_security_path_and_scopes.py: Path sanitization and 403 when scopes are insufficient.
- tests/test_app_exception_handlers.py: Maps Vault Forbidden -> 403 and VaultError -> 502 JSON responses.
- (If you add MCP client tests) exercise POST /mcp/rpc for initialize, tools/list, and tools/call with a JWT or API key.

Run subsets

Keyword filter: pytest -k transit
Coverage detail: pytest -q --cov=vault_mcp --cov-report=term-missing

Security Notes

Use TLS end-to-end; for mTLS, terminate at a trusted proxy and pass identity headers.
Avoid logging secret values; the app uses structured logging with response metadata only.
Prefer JWT or mTLS in production; reserve API keys for development.
Enable Vault audit devices and keep token TTLs minimal.

Troubleshooting

Import errors (e.g., fastapi not found): ensure you use the same Python interpreter that installed deps.
- python -m uvicorn main:app --reload
Uvicorn targets: use <module>:<attribute> — e.g., main:app.
Change port/host: python -m uvicorn main:app --reload --port 8090 --host 0.0.0.0
Increase logs: add --log-level debug --access-log