Web Fetch

🚀 Fetch MCP Server

This MCP server offers web content scraping capabilities in multiple formats, including HTML, JSON, plain text, and Markdown.

🚀 Quick Start

Installation

1. Clone the repository.
2. Install dependencies: npm install
3. Build the server: npm run build

Usage

To use the server, you can directly run:

npm start

This will start the Fetch MCP Server running on standard input and output.

Integration with Desktop Applications

To integrate this server into a desktop application, add the following to the application's server configuration:

{
  "mcpServers": {
    "fetch": {
      "command": "node",
      "args": [
        "{ABSOLUTE PATH TO FILE HERE}/dist/index.js"
      ]
    }
  }
}

✨ Features

• Scrape web content using the modern fetch API.
• Support custom request headers.
• Provide content in multiple formats: HTML, JSON, plain text, and Markdown.
• Use JSDOM for HTML parsing and text extraction.
• Use TurndownService for HTML to Markdown conversion.

📦 Components

Tools

• fetch_html

  • Scrape a website and return the HTML content.
  • Input:
    • url (string, required): The URL of the website to scrape.
    • headers (object, optional): Custom headers to include in the request.
  • Returns the raw HTML content of the web page.

• fetch_json

  • Retrieve a JSON file from a URL.
  • Input:
    • url (string, required): The URL of the JSON to retrieve.
    • headers (object, optional): Custom headers to include in the request.
  • Returns the parsed JSON content.

• fetch_txt

  • Scrape a website and return the plain text content (without HTML).
  • Input:
    • url (string, required): The URL of the website to scrape.
    • headers (object, optional): Custom headers to include in the request.
  • Returns the web page text content with HTML tags, scripts, and styles removed.

• fetch_markdown

  • Scrape a website and return the content in Markdown format.
  • Input:
    • url (string, required): The URL of the website to scrape.
    • headers (object, optional): Custom headers to include in the request.
  • Returns the result of converting the web page content to Markdown format.

Resources

This server does not provide any persistent resources. It is designed to scrape and convert web content on demand.

🔧 Development

• Run npm run dev to start the TypeScript compiler in watch mode.
• Use npm test to run the test suite.

📄 License

This project is released under the MIT License.