MCP Web Extractor

🚀 MCP Web Extractor

A server based on the Model Context Protocol (MCP) that uses Readability.js to extract web page content. This tool can crawl web pages and extract the main information, making it ideal for saving clean, readable articles to Obsidian notes.

🚀 Quick Start

The MCP Web Extractor is a powerful tool that simplifies the process of extracting clean, readable content from web pages. It's especially useful for those who want to save web articles directly to their Obsidian notes.

✨ Features

• Extract readable content from any URL.
• Remove ads, sidebars, and other distractions.
• Return clean text along with metadata (title, excerpt, etc.).
• Easily integrate with Obsidian via MCP.

📦 Installation

# Clone the repository
git clone https://github.com/iemong/mcp-web-extractor.git
cd mcp-web-extractor

# Install dependencies
npm install

# Build the project
npm run build

# Start the server
npm start

The server will run at http://localhost:3000, and the MCP terminal is located at http://localhost:3000/mcp.

💻 Usage Examples

Basic Usage

You can use the included client example to extract content from a URL:

ts-node-esm client-example.ts

Advanced Usage

The obsidian-integration.ts file provides an example of how to integrate this MCP server with Obsidian. You can use it as a starting point to create an Obsidian plugin for extracting web content.

📚 Documentation

API Documentation

The MCP server offers the following functionality:

• extract-content: Extract readable content from a specified URL.
  • Parameters: { url: string }
  • Returns: { title, content, textContent, excerpt, siteName }

📄 License

This project is licensed under the MIT License.