Fetch MCP

🚀 Web Content Fetching MCP Server

This MCP server offers web content scraping capabilities in various formats, including HTML, JSON, plain text, and Markdown.

✨ Features

• Utilize the modern Fetch API to scrape web content.
• Support requests with custom headers.
• Provide content in multiple formats: HTML, JSON, plain text, and Markdown.
• Use JSDOM for HTML parsing and text extraction.
• Employ TurndownService to convert HTML to Markdown format.

📦 Installation

1. Clone the repository.
2. Install dependencies: npm install
3. Build the server: npm run build

🚀 Quick Start

Usage

To use the server, simply run:

npm start

This will start the Web Content Fetching MCP Server and run it on the standard output.

Integration with Desktop Applications

To integrate this server into a desktop application, add the following to the application's server configuration:

{
  "mcpServers": {
    "fetch": {
      "command": "node",
      "args": [
        "{ABSOLUTE PATH TO FILE HERE}/dist/index.js"
      ]
    }
  }
}

💻 Components

Tools

• fetch_html

  • Scrape a website and return the HTML content.
  • Input:
    • url (string, required): The URL of the website to scrape.
    • headers (object, optional): Custom headers to include in the request.
  • Returns the raw HTML content of the webpage.

• fetch_json

  • Retrieve a JSON file from a URL.
  • Input:
    • url (string, required): The URL of the JSON to scrape.
    • headers (object, optional): Custom headers to include in the request.
  • Returns the parsed JSON content.

• fetch_txt

  • Scrape a website and return the plain text content (without HTML).
  • Input:
    • url (string, required): The URL of the website to scrape.
    • headers (object, optional): Custom headers to include in the request.
  • Returns the webpage text content after removing HTML tags, scripts, and styles.

• fetch_markdown

  • Scrape a website and return the content in Markdown format.
  • Input:
    • url (string, required): The URL of the website to scrape.
    • headers (object, optional): Custom headers to include in the request.
  • Returns the result of converting the webpage content to Markdown format.

Resources

This server does not provide any persistent resources. It is designed to scrape and convert web content on-demand.

🔧 Development

• Run npm run dev to start the TypeScript compiler in watch mode.
• Use npm test to run the test suite.

📄 License

This project is open-source under the MIT License.