TL;DR
Scrape-LE is a Model Context Protocol (MCP) server. Analyse robots.txt content and report whether a path may be crawled. Install with: npx -y scrape-le-mcp. Source: https://github.com/nolindnaidoo/scrape-le. Compatible with MCP clients such as Claude Desktop, Cursor, Cline and Windsurf.
Scrape-LE
Analyse robots.txt content and report whether a path may be crawled.
Install Command
npx -y scrape-le-mcpFrequently Asked Questions
Scrape-LE is a Model Context Protocol server that Analyse robots.txt content and report whether a path may be crawled.
You can install it by running the install command: `npx -y scrape-le-mcp`. It then becomes available to any MCP-compatible client such as Claude, Cursor, Cline, Windsurf.
The Scrape-LE MCP Server is compatible with any client that speaks the Model Context Protocol — including Claude, Cursor, Cline, Windsurf.
Check the project license on its repository before use.
Reviews
0 reviews· 0.0 average
No reviews yet. Be the first to share your experience.
Leave a review
Rating
Details
- Agent Rank
- 0.0
- License
- Last Updated
- Aug 12, 2026
- Languages
- TypeScript
Client Compatibility
Speaks the Model Context Protocol — works with any compatible client.