MCP Server Scraper

🚀 MCP Server Configuration Scraper

This script is designed to scrape MCP server configuration information from 'https://github.com/modelcontextprotocol/servers/tree/main/src', primarily for use in the LiteLLM project.

🚀 Quick Start

💻 Usage Examples

Basic Usage

To use this script, follow these steps:

• Compile TypeScript: npm run build
• Run the script: npm start

🔧 Technical Details

Implementation Approach and Logical Deduction

The main executable function is responsible for navigating to the GitHub page and constructing objects. Most of the scraping logic is concentrated in the scrapeServerConfiguration function. The configuration information of MCP servers is presented in JSON format. Users can directly copy this content to the browser clipboard on certain pages (server directories). On other pages, the configuration information is hidden in dropdown menus, such as "Use Docker". This scraper primarily prioritizes obtaining Docker configurations, followed by npx configurations, and finally uvx configurations (if the preferred option does not exist). Since the Playwright selector tends to choose the npx configuration (which may be due to issues in the code logic), an additional for loop has been added to specifically obtain the Docker configuration (contact the author for more information).

Console logs have been added for debugging purposes. Remove or comment out the console logs as needed.