News Aggregator API (MCP Server)
A modular, scalable news aggregation backend built with MCP server conventions and TypeScript. This API provides a unified interface to TheNewsAPI for retrieving current and historical news articles with advanced filtering capabilities. Designed specifically for consumption by AI agents, this API prioritizes structured data and consistent patterns.
Features
- Unified API: Consistent interface for accessing news data
- TypeScript: Fully typed for reliable development
- MCP Architecture: Scalable, modular design with separation of concerns
- Multiple Endpoints: Access various types of news content
- Environment Variables: Secure configuration management
- Error Handling: Consistent error formats with appropriate status codes
- Filtering: Support for various filtering parameters
- Caching: Intelligent caching system with management endpoints
- Interactive Documentation: Swagger-based API documentation
- AI-Friendly Responses: Structured data optimized for machine consumption
Endpoints
News Endpoints
| Endpoint | Description | Example |
|---|---|---|
/api/news/top | Get top news headlines | /api/news/top?categories=business |
/api/news/all | Get all news with advanced search | /api/news/all?search=technology |
/api/news/similar/:uuid | Get similar articles to a specific one | /api/news/similar/cc11e3ab-ced0-4a42-9146-e426505e2e67 |
/api/news/uuid/:uuid | Get a specific article by UUID | /api/news/uuid/cc11e3ab-ced0-4a42-9146-e426505e2e67 |
/api/news/sources | Get available news sources | /api/news/sources?language=en |
Cache Management Endpoints
| Endpoint | Description | Example |
|---|---|---|
/api/cache/stats | Get cache statistics | /api/cache/stats |
/api/cache/clear | Clear all cache | DELETE /api/cache/clear |
/api/cache/clear/:type | Clear cache by type | DELETE /api/cache/clear/top |
Utility Endpoints
| Endpoint | Description |
|---|---|
/health | Health check endpoint |
/docs | Interactive API documentation |
/docs.json | OpenAPI specification |
/examples | Usage examples |
Testing
This project implements a comprehensive testing strategy with multiple layers of tests to ensure reliability and correctness.
Test Types
-
Unit Tests
- Isolated tests for individual components
- Mocked dependencies for true unit isolation
- Focused on business logic validation
-
Controller Tests
- Verify controller behavior with mocked services
- Test proper request handling and response formatting
- Confirm error handling patterns are consistent
-
Integration Tests
- Test the complete request-response cycle
- Verify API endpoints with supertest
- Mock external API calls for reliability
Testing Best Practices
This project follows these testing best practices:
- Isolation - Tests don't depend on each other or external services
- Repeatable - Tests produce the same results on any environment
- Fast - Mocking external dependencies keeps tests efficient
- Comprehensive - Core functionality has high test coverage
- Maintainable - Tests follow consistent patterns and structure
Running Tests
# Run all tests
npm test
# Run tests with coverage
npm run test:coverage
# Run a specific test file
npx jest path/to/test.ts
Installation
# Clone the repository
git clone <repository-url>
# Install dependencies
npm install
# Set up environment variables
cp .env.example .env
# Edit .env file with your own API key and database URL
# Generate Prisma client
npx prisma generate
# Build the project
npm run build
# Start the server
npm start
Configuration
Create a .env file with the following variables:
# News Aggregator API Environment Variables
# API Configuration
NEWS_API_TOKEN=your_api_token_here
# Server Configuration
PORT=3000
NODE_ENV=development
# Database Configuration
DATABASE_URL="file:./dev.db"
Database Configuration
The application uses Prisma ORM with SQLite by default. The DATABASE_URL environment variable should point to your database file. For production, you can configure it to use PostgreSQL or MySQL.
Development
# Run in development mode with hot reload
npm run dev
# Run linting
npm run lint
# Run tests
npm test
Testing Infrastructure
The project includes a comprehensive test suite built with Jest and Supertest, covering API endpoints, database integration, and server connectivity checks.
Running Tests
# Run all tests
npm test
# Run only API endpoint tests
npm run test:api
# Run only database integration tests
npm run test:db
# Run tests with coverage report
npm run test:coverage
# Run tests in watch mode (for development)
npm run test:watch
# Check server connectivity
npm run check-server
Test Structure
-
API Endpoint Tests: Located in
src/__tests__/api/, these tests verify that all API endpoints return the expected responses, handle errors correctly, and apply appropriate filtering. -
Database Integration Tests: Located in
src/__tests__/database.test.ts, these tests verify database connectivity, query execution, and transaction support. -
Server Connectivity Script: Located in
scripts/check-server.ts, this script tests connectivity to a running server instance, verifying that critical endpoints are functioning correctly.
Test Environment
Tests use a separate environment configuration specified by setting NODE_ENV=test. This ensures tests don't interfere with development or production environments. The setup includes:
- Automatic database connection setup and teardown
- Cache clearing between tests to ensure isolation
- Server startup and shutdown for endpoint testing
Future Enhancements Todo List
Priority 1: Foundation Improvements
-
Caching System
- Implement in-memory caching
- Cache frequently accessed data like top news and sources
- Add cache invalidation strategies
- Configure TTL (Time-To-Live) for different content types
- Add cache management endpoints
-
Data Persistence Layer
- Add database integration (SQLite with Prisma)
- Create schema for storing enhanced article metadata
- Implement repository pattern for data access
- Add migration system for schema changes
-
API Documentation
- Generate OpenAPI/Swagger specification
- Add interactive API documentation
- Create usage examples for each endpoint
API Documentation
The API now includes comprehensive documentation using OpenAPI/Swagger:
- Interactive API Documentation: Available at
/docswhen the server is running - OpenAPI Specification: JSON format available at
/docs.json - Usage Examples: Available at
/exampleswith code samples in multiple languages
How to Access the Documentation
- Start the server with
npm run devornpm start - Open your browser to
http://localhost:3000/docsfor the interactive Swagger UI - Browse through the available endpoints, request parameters, and response formats
- Reference implementation examples at
http://localhost:3000/examples
API Documentation Features
- Complete endpoint descriptions with parameters and response types
- Interactive "Try it out" capability to test endpoints directly
- Request/response schema definitions
- Code snippets for API consumption
- Examples of integrating with the MCP server architecture
Priority 2: Security & Performance
-
Authentication & User Management
- Implement JWT-based authentication
- Create user registration and login endpoints
- Add role-based access control
- Implement secure password handling
-
Rate Limiting
- Add request throttling middleware
- Implement tiered access levels
- Create fair usage policies
- Add rate limit headers in responses
-
Monitoring & Analytics
- Set up request logging and monitoring
- Add performance metrics collection
- Create usage dashboards
- Implement error tracking and alerting
Priority 3: Enhanced Functionality
-
Content Processing
- Add text summarization capabilities
- Implement entity extraction
- Add sentiment analysis
- Create topic categorization beyond source categories
- Implement duplicate detection and grouping
-
Personalization
- Add user preference tracking
- Create personalized endpoints based on reading history
- Implement topic following functionality
- Add recommendation engine
-
Webhooks & Real-time Updates
- Implement webhook registration
- Create event-based notification system
- Add SSE (Server-Sent Events) for real-time updates
- Implement topic subscription functionality
Contributing
Contributions to the News Aggregator API project are welcome! This project is designed to be a friendly, well-documented codebase for those interested in MCP server architecture, especially with AI consumption in mind.
How to Contribute
- Fork the repository
- Create a new feature branch (
git checkout -b feature/amazing-feature) - Make your changes
- Run tests to ensure everything works (
npm test) - Commit your changes (
git commit -m 'Add some amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
Code Style
Please follow the existing code style and patterns in the project. This project uses ESLint for code quality and formatting.
Security Considerations
When contributing, please ensure:
- No API keys or credentials are hardcoded
- Environment variables are used for configuration
- No sensitive data is logged
- Input validation is properly implemented
Testing
All new features should include appropriate tests. Please follow the existing testing patterns in the project.
- Search History & Bookmarks
- Add saved searches functionality
- Implement article bookmarking
- Create reading history tracking
- Add user collections feature
Contributing
- Fork the repository
- Create your feature branch:
git checkout -b feature/my-new-feature - Commit your changes:
git commit -am 'Add some feature' - Push to the branch:
git push origin feature/my-new-feature - Submit a pull request
License
ISC