📚 Personal Research Assistant MCP
A production-ready MCP (Model Context Protocol) server that enables semantic search across your personal research library. Built for AI Engineers who need fast, accurate document retrieval integrated with Claude Desktop and other AI tools.
🎯 Problem Statement
Researchers and professionals accumulate dozens of papers and documents but struggle to:
- Find relevant information across multiple documents
- Remember which paper contained specific insights
- Connect related concepts across different sources
- Spend 2+ hours daily searching for information
Traditional keyword search misses semantic connections, and reading everything is impractical.
💡 Solution
An MCP server that:
- Indexes documents into a vector database using semantic embeddings
- Enables Claude (or any MCP client) to query your research library conversationally
- Provides sub-500ms response times with 85%+ retrieval accuracy
- Includes a Streamlit dashboard for management and metrics
🏗️ Architecture
Documents (PDF/DOCX/HTML/MD)
↓
Document Processor → Text Chunker → Embeddings
↓
ChromaDB Vector Store
↓
├─→ MCP Server (FastMCP) → Claude Desktop
└─→ Streamlit UI → Monitoring/Testing
✨ Features
- Semantic Search: Natural language queries across your entire library
- Multi-Format Support: PDF, DOCX, HTML, Markdown, TXT
- Fast Retrieval: <500ms query latency on 1000+ chunks
- MCP Integration: Works with Claude Desktop, VS Code, and any MCP client
- Metadata Extraction: Automatically extracts titles, authors, keywords
- Query Logging: Track usage and performance metrics
- Streamlit Dashboard: Upload, search, and visualize metrics
📊 Performance Metrics
| Metric | Target | Actual |
|---|---|---|
| Retrieval Accuracy | 85% | See METRICS.md |
| Query Latency | <500ms | See METRICS.md |
| Scale | 10k+ chunks | 1782+ chunks |
🚀 Installation
Prerequisites
- Python 3.11+
- 2GB RAM minimum
- Git
Setup
# Clone repository
git clone https://github.com/yourusername/research-assistant-mcp.git
cd research-assistant-mcp
# Create virtual environment
python -m venv venv
source venv/bin/activate # Windows: venv\Scripts\activate
# Install dependencies
pip install -r requirements.txt
# Install local embeddings
pip install sentence-transformers
# Configure environment
cp .env.example .env
# Edit .env - add OPENAI_API_KEY if using OpenAI embeddings
Download Sample Data
# Download 25 AI/ML papers from arXiv
python scripts/download_sample_papers.py --count 25
Index Documents
# Index sample papers
python scripts/index_docs.py --folder ./sample_papers
# Or index your own documents
python scripts/index_docs.py --folder /path/to/your/papers --recursive
📖 Usage
Start MCP Server
python mcp_server/server.py
Configure Claude Desktop
Add to claude_desktop_config.json:
Mac: ~/Library/Application Support/Claude/claude_desktop_config.json
Windows: %APPDATA%\Claude\claude_desktop_config.json
{
"mcpServers": {
"research-assistant": {
"command": "python",
"args": ["/full/path/to/research-assistant-mcp/mcp_server/server.py"],
"env": {}
}
}
}
Restart Claude Desktop.
Launch Streamlit UI
streamlit run ui/app.py
Opens at http://localhost:8501
🛠️ MCP Tools
search_documents
Semantic search across your library.
Query: "What are the challenges in RAG systems?"
Returns: Top-k results with sources, scores, and metadata
get_document_summary
Get quick overview of a document.
Input: Document path or title
Returns: Title, author, keywords, preview
find_related_papers
Find documents similar to a topic.
Query: "prompt engineering techniques"
Returns: Related papers with relevance scores
📁 Project Structure
research-assistant-mcp/
├── mcp_server/ # MCP server implementation
│ └── server.py
├── rag_pipeline/ # RAG components
│ ├── config.py
│ ├── document_processor.py
│ ├── chunker.py
│ ├── vector_store.py
│ ├── retriever.py
│ └── metadata_extractor.py
├── ui/ # Streamlit dashboard
│ ├── app.py
│ └── pages/
├── scripts/ # CLI utilities
│ ├── index_docs.py
│ └── download_sample_papers.py
├── tests/ # Testing & benchmarks
│ ├── sample_queries.json
│ └── benchmark_performance.py
├── data/ # Data storage
│ ├── chroma_db/
│ └── query_logs/
└── docs/ # Documentation
└── METRICS.md
🧪 Testing
# Run performance benchmarks
python tests/benchmark_performance.py
# Output: Accuracy, latency, scale metrics
🐳 Docker Deployment
# Build and run
docker-compose up -d
# Access UI at http://localhost:8501
# MCP server runs on localhost:8000
📈 Example Queries
-
Cross-document synthesis
"Compare different fine-tuning approaches for LLMs" -
Concept exploration
"How does RLHF improve model alignment?" -
Technical details
"Explain transformer attention mechanisms" -
Literature review
"What are recent developments in RAG systems?"
🔧 Customization
Change Embedding Model
Edit .env:
# OpenAI (paid, best quality)
EMBEDDING_MODEL=text-embedding-3-small
# Or use local (free) by default - already configured
Adjust Chunk Size
Edit .env:
CHUNK_SIZE=1000 # Characters per chunk
CHUNK_OVERLAP=200 # Overlap between chunks
Add Document Types
Edit rag_pipeline/document_processor.py to add new file type handlers.
🐛 Troubleshooting
ChromaDB errors: Delete data/chroma_db and re-index
Import errors: Verify pip install -r requirements.txt completed
UI blank: Check browser console, try Chrome/Firefox
Slow queries: Reduce TOP_K_RESULTS in .env
🚧 Future Enhancements
- Auto-watch folder for new documents
- Cross-encoder reranking for better accuracy
- Multi-modal support (images, diagrams)
- Citation network graph
- Export to Notion/Obsidian
- Web interface (FastAPI + React)
🎥 Demo Video
[Link to 2-minute demo video - Coming soon]
🤝 Contributing
Contributions welcome! Please open issues or PRs.
📄 License
MIT License - see LICENSE