Daraja API Documentation Scraper & MCP Server
A comprehensive toolkit for accessing Safaricom's Daraja API documentation through web scraping and Model Context Protocol (MCP) integration. This project provides both a powerful scraper for keeping documentation up-to-date and a professional MCP server for AI assistant integration.
⚠️ Important Legal Notice & Disclaimers
Data Source and Usage Rights
- Documentation Source: This project scrapes documentation from Safaricom's Daraja API portal
- Authentication Required: The scraper requires login credentials to access Safaricom's developer portal
- Terms of Service: Users must comply with Safaricom's Terms of Service when using the scraper
- Data Ownership: All scraped documentation remains the intellectual property of Safaricom PLC
Legal Compliance
- Personal Use: This tool is intended for personal development and learning purposes
- Commercial Use: For commercial applications, ensure you have appropriate licenses from Safaricom
- Rate Limiting: The scraper includes delays to respect Safaricom's servers - do not modify these
- Account Responsibility: Users are responsible for their own Safaricom developer account credentials
Disclaimers
- No Warranty: This software is provided "as is" without any warranties
- Data Accuracy: Scraped documentation may become outdated - always verify with official sources
- Service Availability: Safaricom may change their portal structure, potentially breaking the scraper
- Legal Responsibility: Users assume all legal responsibility for their use of this tool
Ethical Usage Guidelines
- Respect Rate Limits: Do not overwhelm Safaricom's servers with excessive requests
- Valid Credentials: Only use your own legitimate Safaricom developer account
- Data Sharing: Be mindful of sharing scraped data - respect Safaricom's intellectual property
- Updates: Keep scraped documentation current and don't redistribute outdated information
By using this software, you acknowledge that you have read, understood, and agree to comply with these terms and Safaricom's Terms of Service.
Features
- Complete Documentation Scraper: Automated scraping of all 22 Daraja APIs with images
- Professional MCP Server: Standards-compliant server for AI assistant integration
- Docker Support: Containerized deployment for consistent environments
- Multi-Platform Support: Works on Windows, macOS, and Linux
- Editor Integration: Compatible with Kiro, VSCode, Cursor, Windsurf, Claude Desktop, and more
- Offline Ready: Complete documentation dataset included for immediate use
- Auto-Discovery: Smart path detection for seamless setup
Table of Contents
- Important Legal Notice & Disclaimers
- Quick Start
- Installation
- Docker Deployment
- Scraper Usage
- MCP Server Setup
- Editor Integration
- API Documentation
- Troubleshooting
- Contributing
- License
- Contributing
Quick Start
Option 1: Docker Deployment (Recommended)
# Clone the repository
git clone https://github.com/JacksCodeVault/mpesa-daraja-mcp.git
cd mpesa-daraja-mcp
# Build and start Docker container
cd mpesa-daraja-mcp
pnpm run docker:build
pnpm run docker:start
# Configure your editor (see Editor Integration section)
Option 2: Native Installation
# Clone the repository
git clone https://github.com/JacksCodeVault/mpesa-daraja-mcp.git
cd mpesa-daraja-mcp
# Setup MCP server
cd mpesa-daraja-mcp
pnpm install
pnpm run build
# Configure your editor (see Editor Integration section)
Option 3: Fresh Scraping + MCP Setup
# Clone and setup scraper
git clone https://github.com/JacksCodeVault/mpesa-daraja-mcp.git
cd mpesa-daraja-mcp
# Setup Python environment
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -r requirements.txt
# Run scraper (requires Daraja portal login)
python scraper.py
# Setup MCP server
cd mpesa-daraja-mcp
pnpm install
pnpm run build
Installation
Prerequisites
- Docker (recommended) OR
- Python 3.8+ (for scraper) + Node.js 18+ (for MCP server)
- Git (for cloning)
Platform-Specific Setup
Windows
# Option 1: Docker (Recommended)
# Install Docker Desktop from https://docker.com
# Clone repository
git clone https://github.com/JacksCodeVault/mpesa-daraja-mcp.git
cd mpesa-daraja-mcp\mpesa-daraja-mcp
pnpm run docker:build
# Option 2: Native Installation
# Install Python and Node.js from official websites
# Clone repository
git clone https://github.com/JacksCodeVault/mpesa-daraja-mcp.git
cd mpesa-daraja-mcp
# Python setup (for scraper)
python -m venv venv
venv\Scripts\activate
pip install -r requirements.txt
# Node.js setup (for MCP server)
cd mpesa-daraja-mcp
pnpm install
pnpm run build
macOS
# Option 1: Docker (Recommended)
# Install Docker Desktop from https://docker.com
brew install git
git clone https://github.com/JacksCodeVault/mpesa-daraja-mcp.git
cd mpesa-daraja-mcp/mpesa-daraja-mcp
pnpm run docker:build
# Option 2: Native Installation
# Install dependencies via Homebrew
brew install python node git pnpm
# Clone repository
git clone https://github.com/JacksCodeVault/mpesa-daraja-mcp.git
cd mpesa-daraja-mcp
# Python setup (for scraper)
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
# Node.js setup (for MCP server)
cd mpesa-daraja-mcp
pnpm install
pnpm run build
Linux (Ubuntu/Debian)
# Option 1: Docker (Recommended)
# Install Docker
sudo apt update
sudo apt install docker.io docker-compose git
sudo usermod -aG docker $USER
# Log out and back in, then:
git clone https://github.com/JacksCodeVault/mpesa-daraja-mcp.git
cd mpesa-daraja-mcp/mpesa-daraja-mcp
pnpm run docker:build
# Option 2: Native Installation
# Install dependencies
sudo apt update
sudo apt install python3 python3-venv nodejs npm git
npm install -g pnpm
# Clone repository
git clone https://github.com/JacksCodeVault/mpesa-daraja-mcp.git
cd mpesa-daraja-mcp
# Python setup (for scraper)
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
# Node.js setup (for MCP server)
cd mpesa-daraja-mcp
pnpm install
pnpm run build
Docker Deployment
Docker provides the most reliable and consistent deployment method across all platforms.
Quick Docker Setup
cd mpesa-daraja-mcp
# Build the Docker image
pnpm run docker:build
# Start the container
pnpm run docker:start
# Check status
pnpm run docker:status
# View logs
pnpm run docker:logs
Docker Management Commands
# Build and deployment
pnpm run docker:build # Build the Docker image
pnpm run docker:start # Start the container
pnpm run docker:stop # Stop the container
pnpm run docker:restart # Restart the container
# Monitoring and debugging
pnpm run docker:status # Check container status
pnpm run docker:logs # View container logs
pnpm run docker:shell # Open shell in container
# Cleanup
pnpm run docker:clean # Remove container and image
Docker Configuration
The Docker setup includes:
- Multi-stage build for optimized image size
- Non-root user for security
- Volume mounting for documentation access
- Health checks for reliability
- Resource limits for production use
Docker Compose Configuration
# docker-compose.yml
services:
daraja-mcp:
build: .
container_name: daraja-mcp-server
restart: unless-stopped
volumes:
- ../daraja_docs_v3:/app/daraja_docs_v3:ro
environment:
- NODE_ENV=production
stdin_open: true
tty: true
MCP Configuration for Docker
When using Docker, update your editor's MCP configuration:
{
"mcpServers": {
"daraja-docs": {
"command": "docker",
"args": ["exec", "-i", "daraja-mcp-server", "node", "dist/index.js"],
"disabled": false,
"autoApprove": [
"search_daraja_apis",
"get_daraja_api_doc",
"list_apis_by_category",
"get_api_summary",
"get_server_stats",
"compare_apis"
]
}
}
}
Scraper Usage
⚠️ IMPORTANT: Before using the scraper, ensure you have:
- A valid Safaricom developer account
- Accepted Safaricom's Terms of Service
- Permission to access the documentation programmatically
- Understanding that you're responsible for compliance with Safaricom's policies
The scraper automatically downloads documentation for all 22 Daraja APIs with images.
First Time Setup
-
Activate Python environment:
source venv/bin/activate # Linux/macOS venv\Scripts\activate # Windows -
Run the scraper:
python scraper.py -
Login Process:
- Browser window opens automatically
- Login to Daraja portal manually when prompted
- Press ENTER in terminal after successful login
- Scraper saves session for future use
Updating Documentation
To get the latest API documentation:
# Activate environment
source venv/bin/activate
# Run scraper (uses saved session)
python scraper.py
# If session expired, login again when browser opens
Output Structure
daraja_docs_v3/
├── data_index.json # API metadata and index
├── docs/ # Markdown documentation
│ ├── Authorization.md
│ ├── MpesaExpressSimulate.md
│ └── ... (22 API files)
└── images/ # Downloaded images
├── Authorization_img_0.svg
└── ... (200+ images)
MCP Server Setup
The MCP server provides AI assistants with access to Daraja documentation through 6 powerful tools.
Native Build and Test
cd mpesa-daraja-mcp
# Install dependencies
pnpm install
# Build TypeScript
pnpm run build
# Test server can find documentation
pnpm test
# Run server (for testing)
pnpm run dev
Available MCP Tools
The MCP server provides 6 comprehensive tools:
search_daraja_apis- Advanced search with category filteringget_daraja_api_doc- Complete API documentation retrievallist_apis_by_category- Organized API listing by categoriesget_api_summary- Enhanced summaries with endpointsget_server_stats- Usage statistics and monitoringcompare_apis- Side-by-side API comparison
Server Configuration
The server automatically detects documentation in these locations:
../daraja_docs_v3(relative to server)./daraja_docs_v3(current directory)/app/daraja_docs_v3(Docker container)- Custom path via environment variable
Editor Integration
Kiro IDE
-
Create MCP configuration file:
# Create .kiro/settings/mcp.json in your workspace mkdir -p .kiro/settings -
Docker Configuration (Recommended):
{ "mcpServers": { "daraja-docs": { "command": "docker", "args": ["exec", "-i", "daraja-mcp-server", "node", "dist/index.js"], "disabled": false, "autoApprove": [ "search_daraja_apis", "get_daraja_api_doc", "list_apis_by_category", "get_api_summary", "get_server_stats", "compare_apis" ] } } } -
Native Configuration:
{ "mcpServers": { "daraja-docs": { "command": "node", "args": ["mpesa-daraja-mcp/dist/index.js"], "disabled": false, "autoApprove": [ "search_daraja_apis", "get_daraja_api_doc", "list_apis_by_category", "get_api_summary", "get_server_stats", "compare_apis" ] } } }
VSCode with MCP Extensions
-
Install MCP extension (if available)
-
Configure in settings.json:
{ "mcp.servers": { "daraja-docs": { "command": "docker", "args": ["exec", "-i", "daraja-mcp-server", "node", "dist/index.js"] } } }
Cursor IDE
-
Open Cursor settings
-
Add MCP server configuration:
{ "mcp": { "servers": { "daraja-docs": { "command": "docker", "args": ["exec", "-i", "daraja-mcp-server", "node", "dist/index.js"] } } } }
Windsurf
-
Access Windsurf MCP settings
-
Add server configuration:
{ "mcpServers": { "daraja-docs": { "command": "docker", "args": ["exec", "-i", "daraja-mcp-server", "node", "dist/index.js"] } } }
Claude Desktop
-
Edit Claude config file:
- macOS:
~/Library/Application Support/Claude/claude_desktop_config.json - Windows:
%APPDATA%\Claude\claude_desktop_config.json
- macOS:
-
Add server configuration:
{ "mcpServers": { "daraja-docs": { "command": "docker", "args": ["exec", "-i", "daraja-mcp-server", "node", "dist/index.js"] } } }
Generic MCP Client
For any MCP-compatible client using Docker:
{
"servers": {
"daraja-docs": {
"command": "docker",
"args": ["exec", "-i", "daraja-mcp-server", "node", "dist/index.js"],
"env": {}
}
}
}
For native installation:
{
"servers": {
"daraja-docs": {
"command": "node",
"args": ["dist/index.js"],
"cwd": "/absolute/path/to/mpesa-daraja-mcp",
"env": {}
}
}
}
API Documentation
Available APIs
The project includes comprehensive documentation for 22 Daraja APIs organized into 6 categories:
Core Payment APIs
- Authorization - OAuth token generation and management
- MpesaExpressSimulate - STK Push payment initiation
- MpesaExpressQuery - STK Push transaction status queries
Payment Processing
- CustomerToBusiness - C2B payment processing
- BusinessToCustomer - B2C payment disbursements
- CustomerToBusinessRegisterURL - C2B URL registration
Transaction Management
- TransactionStatus - Transaction status verification
- AccountBalance - Account balance inquiries
- Reversal - Transaction reversal operations
- PullTransaction - Transaction detail retrieval
Business Operations
- BusinessPayBill - Bill payment processing
- BusinessBuyGoods - Goods purchase payments
- B2BExpressCheckout - Business-to-business transactions
- BusinessToPochi - Business to Pochi wallet transfers
Advanced Features
- DynamicQRCode - Dynamic QR code generation
- BillManager - Bill management and processing
- TaxRemittance - Tax payment processing
- MpesaRatiba - Scheduled payment management
Specialized Services
- B2CAccountTopUp - Account top-up services
- Swap - Currency swap operations
- IMSI - SIM card management services
- IotSimManagement - IoT SIM card management
Using the MCP Tools
Advanced API Search
Use search_daraja_apis tool with powerful filtering:
- Query: "payment" + Category: "core" → Core payment APIs
- Query: "balance" → AccountBalance and related APIs
- Category: "business" → All business operation APIs
- No parameters → All 22 APIs with categories
Complete Documentation Access
Use get_daraja_api_doc tool for full documentation:
- api_name: "Authorization" → Complete OAuth implementation guide
- api_name: "MpesaExpressSimulate" → Full STK Push documentation
- Includes code examples, parameters, and response formats
Enhanced API Summaries
Use get_api_summary tool for quick overviews:
- api_name: "BusinessToCustomer" → Summary with key endpoints
- include_endpoints: true → Includes endpoint URLs
- Perfect for quick reference and comparison
API Comparison
Use compare_apis tool for side-by-side analysis:
- api_names: ["Authorization", "MpesaExpressSimulate"] → Compare auth vs payment
- comparison_aspects: ["endpoints", "authentication"] → Focus on specific aspects
- Supports 2-4 APIs simultaneously
Usage Statistics
Use get_server_stats tool for monitoring:
- Total API count and categories
- Most accessed APIs
- Request statistics
- Server health information
Category-Based Browsing
Use list_apis_by_category tool for organized access:
- category: "payments" → All payment-related APIs
- No category → List all categories with counts
- Perfect for discovering related APIs
Troubleshooting
Docker Issues
Container Won't Start
# Check Docker is running
docker --version
docker-compose --version
# View detailed logs
pnpm run docker:logs
# Rebuild if needed
pnpm run docker:clean
pnpm run docker:build
Documentation Not Found in Container
# Check volume mounting
docker inspect daraja-mcp-server
# Verify documentation exists
ls -la ../daraja_docs_v3/data_index.json
# Test container access
pnpm run docker:shell
ls -la /app/daraja_docs_v3/
MCP Connection Issues with Docker
# Ensure container is running
pnpm run docker:status
# Test MCP tools directly
docker exec -i daraja-mcp-server node dist/index.js
# Check container logs for errors
pnpm run docker:logs
Native Installation Issues
MCP Server Issues
Server won't start:
# Check Node.js version
node --version # Should be 18+
# Rebuild server
cd mpesa-daraja-mcp
pnpm run clean
pnpm install
pnpm run build
Documentation not found:
# Test documentation detection
pnpm test
# Check paths manually
ls -la ../daraja_docs_v3/data_index.json
TypeScript errors:
# Update dependencies
pnpm update
pnpm run build
# Check for syntax errors
pnpm run dev
Scraper Issues
Browser doesn't open:
# Install browser dependencies
playwright install chromium
# Check Python version
python --version # Should be 3.8+
Login session expired:
# Delete auth file and re-login
rm auth.json
python scraper.py
Permission errors:
# Check file permissions
chmod +x scraper.py
# Or run with python explicitly
python scraper.py
Editor Integration Issues
MCP Server Not Connecting
-
Verify Docker container is running:
pnpm run docker:status -
Check MCP configuration syntax:
- Ensure JSON is valid
- Verify command and args are correct
- Use absolute paths for native installation
-
Test server manually:
# Docker docker exec -i daraja-mcp-server node dist/index.js # Native cd mpesa-daraja-mcp node dist/index.js -
Restart editor after configuration changes
Tools Not Appearing
- Check server logs in editor's MCP panel
- Verify autoApprove list includes desired tools
- Ensure server builds successfully:
pnpm run build - Test with minimal configuration first
Platform-Specific Issues
Windows Platform Issues
- Use forward slashes in Docker paths:
C:/path/to/project - Run PowerShell as Administrator if Docker permission issues
- Check Windows Defender isn't blocking Docker or Node.js
- Use WSL2 for better Docker performance
macOS Platform Issues
- Install Xcode Command Line Tools:
xcode-select --install - Use Homebrew for dependency management:
brew install docker - Check Docker Desktop is running and configured
- Verify file permissions:
chmod +x docker-scripts.sh
Linux Platform Issues
- Add user to docker group:
sudo usermod -aG docker $USER - Install build essentials:
sudo apt install build-essential - Check Docker service:
sudo systemctl status docker - Verify Node.js installation:
which node
Performance Optimization
Docker Performance
# Limit container resources
docker update --memory=512m --cpus="0.5" daraja-mcp-server
# Use multi-stage builds for smaller images
# (Already implemented in Dockerfile)
# Clean up unused Docker resources
docker system prune -a
MCP Server Performance
# Monitor server stats
# Use get_server_stats tool to track usage
# Optimize documentation loading
# Server caches documentation on startup
# Use appropriate log levels
NODE_ENV=production pnpm run docker:start
Getting Help
- Check logs first: Both Docker and native installations provide detailed error messages
- Test components individually: Use
pnpm testto verify MCP server setup - Verify paths and permissions: Ensure all file paths are correct and accessible
- Update dependencies: Keep Docker, Node.js, and Python packages current
- Use Docker for consistency: Docker eliminates most platform-specific issues
Contributing
⚠️ Data Handling Guidelines for Contributors:
- No Scraped Data in PRs: Do not include scraped documentation in pull requests
- Respect IP Rights: Ensure contributions don't violate Safaricom's intellectual property
- Code Only: Contribute improvements to the scraper and MCP server code, not the data
- Documentation: Update README and code comments, not scraped API documentation
Development Setup
-
Fork the repository
-
Create feature branch:
git checkout -b feature/amazing-feature -
Setup development environment:
# Docker development (recommended) cd mpesa-daraja-mcp pnpm run docker:build pnpm run docker:start # Native development # Python development pip install -r requirements-dev.txt # If exists # Node.js development cd mpesa-daraja-mcp pnpm install pnpm run dev
Adding New APIs
-
Update scraper URLs:
URLS = [ "https://developer.safaricom.co.ke/apis/NewAPI", # ... existing URLs ] -
Test scraping:
python scraper.py -
Update API categorization:
// In mpesa-daraja-mcp/src/config.ts API_CATEGORIES: { new_category: ["NewAPI"], // ... existing categories } -
Verify MCP server:
cd mpesa-daraja-mcp pnpm test pnpm run docker:build
Code Style
- Python: Follow PEP 8, use
blackformatter - TypeScript: Use Prettier formatting, ESLint rules
- Docker: Follow best practices for multi-stage builds
- Commits: Use conventional commit format
Testing
# Test scraper
python scraper.py --test-mode
# Test MCP server (native)
cd mpesa-daraja-mcp
pnpm test
pnpm run build
# Test MCP server (Docker)
pnpm run docker:build
pnpm run docker:start
pnpm run docker:logs
Documentation
- Update README for new features
- Add inline code comments
- Update API documentation
- Include Docker-specific instructions
License
This project is licensed under the MIT License - see the LICENSE file for details.
⚠️ Important License Clarification:
- Software License: The MIT License applies only to the scraper and MCP server code
- Documentation Rights: Scraped Daraja API documentation remains the property of Safaricom PLC
- Separate Terms: Use of Safaricom's documentation is subject to their Terms of Service
- No Transfer: This license does not grant rights to Safaricom's intellectual property
What this means
- ✅ Commercial use - Use this project in commercial applications
- ✅ Modification - Modify the source code to fit your needs
- ✅ Distribution - Distribute copies of the software
- ✅ Private use - Use the software for private purposes
- ❌ Liability - The authors are not liable for any damages
- ❌ Warranty - The software is provided "as is" without warranty
Attribution
If you use this project, please consider:
- ⭐ Starring the repository on GitHub
- 📝 Mentioning the project in your documentation
- 🔗 Linking back to the original repository
Acknowledgments
- Safaricom for providing the Daraja API platform
- Model Context Protocol team for the MCP standard
- Playwright team for the excellent automation framework
- Docker community for containerization best practices
- Open source contributors who make projects like this possible
Repository Information
- Repository: https://github.com/JacksCodeVault/mpesa-daraja-mcp
- Issues: Report bugs or request features
- Discussions: Community discussions
- Releases: Latest releases
Support
If you find this project helpful, please consider:
- ⭐ Starring the repository
- 🐛 Reporting issues you encounter
- 💡 Suggesting improvements
- 🤝 Contributing to the project
- 📢 Sharing with others who might benefit
Ready to integrate Daraja API documentation with your AI assistant?
Quick Start with Docker: cd mpesa-daraja-mcp && pnpm run docker:build && pnpm run docker:start
Browse the documentation: Use the MCP tools to explore 22 comprehensive Daraja APIs
Need help? Check the Troubleshooting section or open an issue
Start with the Quick Start guide and have your AI assistant accessing Daraja documentation in minutes!