UseScraper MCP Server
This is a TypeScript-based MCP server that provides web scraping capabilities using the UseScraper API. It exposes a single tool 'scrape' that can extract content from web pages in various formats.
Features
Tools
scrape- Extract content from a webpage- Parameters:
url(required): The URL of the webpage to scrapeformat(optional): The format to save the content (text, html, markdown). Default: markdownadvanced_proxy(optional): Use advanced proxy to circumvent bot detection. Default: falseextract_object(optional): Object specifying data to extract
- Parameters:
Installation
Installing via Smithery
To install UseScraper for Claude Desktop automatically via Smithery:
npx -y @smithery/cli install usescraper-server --client claude
Manual Installation
-
Clone the repository:
git clone https://github.com/your-repo/usescraper-server.git cd usescraper-server -
Install dependencies:
npm install -
Build the server:
npm run build
Configuration
To use with Claude Desktop, add the server config:
On MacOS: ~/Library/Application Support/Claude/claude_desktop_config.json
On Windows: %APPDATA%/Claude/claude_desktop_config.json
{
"mcpServers": {
"usescraper-server": {
"command": "node",
"args": ["/path/to/usescraper-server/build/index.js"],
"env": {
"USESCRAPER_API_KEY": "your-api-key-here"
}
}
}
}
Replace /path/to/usescraper-server with the actual path to the server and your-api-key-here with your UseScraper API key.
Usage
Once configured, you can use the 'scrape' tool through the MCP interface. Example usage:
{
"name": "scrape",
"arguments": {
"url": "https://example.com",
"format": "markdown"
}
}
Development
For development with auto-rebuild:
npm run watch
Debugging
Since MCP servers communicate over stdio, debugging can be challenging. We recommend using the MCP Inspector, which is available as a package script:
npm run inspector
The Inspector will provide a URL to access debugging tools in your browser.