MCP Getweb

MCP-GetWeb is a protocol server that provides web search and content extraction functions, supporting multiple search engines such as DuckDuckGo, Google, and Felo AI, and can crawl URL content, extract metadata, and convert it into Markdown format.

Search tools Research and data #Web Search #Content Extraction #URL Crawling #Metadata Extraction .Rust

rating : 2.5 points

downloads : 0

update time : 2025-09-09

Open Site

What is MCP-GetWeb?

MCP-GetWeb is a powerful network information acquisition tool designed specifically for AI assistants. It can obtain the latest information through multiple search engines and extract clean content from web pages, allowing AI to access and understand real-time information on the network.

How to use MCP-GetWeb?

Simply add the MCP-GetWeb server settings to your AI tool configuration file and provide the necessary API keys to start using various search and content extraction functions.

Applicable scenarios

Suitable for AI application scenarios that require access to the latest network information, such as real-time news queries, technical document retrieval, product information comparison, and academic research data collection.

Main Features

DuckDuckGo Search

Perform web searches using DuckDuckGo, supporting pagination and result quantity control, with an emphasis on privacy protection.

Google Search

Obtain precise search results through Google's programmable search engine, supporting various filtering and sorting options.

Felo AI Search

AI-driven intelligent search specifically tailored for developer-related content such as technical information, version releases, and security recommendations.

URL Content Extraction

Extract clean text content from any web page, with control over content length and included element types.

Web Page Metadata Extraction

Quickly obtain metadata information such as the title, description, and images of a web page.

Web Page to Markdown

Convert web page content into Markdown format for easy subsequent processing and analysis.

Jina Intelligent Reader

Use Jina AI's advanced content parsing service to provide LLM-friendly content formats and intelligent summaries.

Advantages

Supports multiple search engines, providing flexible search options

High-quality content extraction and cleaning functions

Specially optimized AI-friendly content formats

Rich filtering and configuration options

Privacy-protected search options

Limitations

Some functions require API keys and configuration

Content extraction may be affected by the website structure

Free APIs may have usage limitations

Requires a stable network connection

How to Use

Obtain API Keys

Obtain the corresponding API keys according to the functions you need to use: Google API key, search engine ID, and Jina AI API key.

Configure the MCP Server

Add the MCP-GetWeb server settings to your AI tool configuration file.

Start Using

Restart your AI tool, and now you can use various search and content extraction commands.

Usage Examples

Technical Research

Search for the latest technical documents and release information

Content Summarization

Extract long articles and perform intelligent summarization

Product Comparison

Collect information and user reviews of multiple products

Frequently Asked Questions

Do I need to pay to use all functions?

Does it support Chinese search?

How accurate is the content extraction?

How can I obtain API keys?

Related Resources

GitHub Project Address

Get the latest code, submit issues, and view the update log

Google API Documentation

Learn how to create and manage Google API keys

Jina AI Official Website

Model Context Protocol Specification

Understand the detailed specifications and working principles of the MCP protocol

🚀 MCP-GetWeb

MCP-GetWeb is a Model Context Protocol (MCP) server that offers web search and content extraction capabilities, facilitating access to a wide range of web information.

🚀 Quick Start

Before starting, you need to obtain the following keys:

Here is a configuration example:

{
  "mcpServers": {
    "getweb": {
      "command": "npx",
      "args": [
        "mcp-getweb"
      ],
      "type": "stdio",
      "env": {
        "GOOGLE_API_KEY": "XXXXXXXXX",
        "GOOGLE_SEARCH_ENGINE_ID": "XXXXXXXXX",
        "JINA_API_KEY": "jina_XXXXXXXXX"
      }
    }
  }
}

✨ Features

1) DuckDuckGo Search (`duckduckgo-search`)

Search the web using DuckDuckGo with HTML scraping.

Parameters:

Property	Details
`query`	(string, required) The search query
`page`	(integer, optional) Page number (default: 1, min: 1)
`numResults`	(integer, optional) Number of results to return (default: 10, min: 1, max: 20)

2) Google Search (`google-search`)

Search Google and return relevant results using the Programmable Search Engine.

Parameters:

Property	Details
`query`	(string, required) Search query; quotes enable exact matches
`num_results`	(integer, optional) Total results to return (default: 5, max: 10)
`site`	(string, optional) Restrict to a specific site/domain (e.g., `wikipedia.org`)
`language`	(string, optional) ISO 639-1 language code (e.g., `en`, `es`)
`dateRestrict`	(string, optional) Date filter, e.g., `d7`, `w4`, `m6`, `y1`
`exactTerms`	(string, optional) Exact phrase that must appear
`resultType`	(string, optional) Result type: `image`\|`images`\|`news`\|`video`\|`videos`
`page`	(integer, optional) Page number for pagination (default: 1, min: 1)
`resultsPerPage`	(integer, optional) Results per page (default: 5, max: 10)
`sort`	(string, optional) Sort order, `relevance` (default) or `date`

⚠️ Important Note

Requires GOOGLE_API_KEY and GOOGLE_SEARCH_ENGINE_ID to be set.

3) Felo AI Search (`felo-search`)

AI-powered search with contextual responses for up-to-date technical information (releases, advisories, migrations, benchmarks, community insights).

Parameters:

Property	Details
`query`	(string, required) The search query or prompt
`stream`	(boolean, optional) Whether to stream the response (default: false)

4) URL Content Fetcher (`fetch-url`)

Fetch the clean content of a URL and return it as text with extraction controls.

Parameters:

Property	Details
`url`	(string, required) The URL to fetch
`maxLength`	(integer, optional) Maximum content length (default: 10000, min: 1000, max: 50000)
`extractMainContent`	(boolean, optional) Attempt to extract main content (default: true)
`includeLinks`	(boolean, optional) Include link text (default: true)
`includeImages`	(boolean, optional) Include image alt text (default: true)
`excludeTags`	(array of string, optional) Tags to exclude (default includes `script`, `style`, `noscript`, `iframe`, `svg`, `nav`, `footer`, `header`, `aside`)

5) URL Metadata Extractor (`url-metadata`)

Extract metadata (title, description, image, favicon) from a URL.

Parameters:

Property	Details
`url`	(string, required) The URL to extract metadata from

6) URL Fetch to Markdown (`url-fetch`)

Fetch web pages and convert them to Markdown. Handles HTML, plaintext, and JSON (pretty-printed in a fenced block).

Parameters:

Property	Details
`url`	(string, required) The URL to fetch and convert to Markdown

7) Jina Reader (`jina-reader`)

Retrieve LLM-friendly content from a URL using Jina r.reader with optional summaries and formats.

Parameters:

Property	Details
`url`	(string, required) The URL to fetch and parse
`maxLength`	(integer, optional) Maximum output length (default: 10000, min: 1000, max: 50000)
`withLinksummary`	(boolean, optional) Include links summary (default: false)
`withImagesSummary`	(boolean, optional) Include images summary (default: false)
`withGeneratedAlt`	(boolean, optional) Generate alt text for images (default: false)
`returnFormat`	(string, optional) `markdown` (default) \| `html` \| `text` \| `screenshot` \| `pageshot`
`noCache`	(boolean, optional) Bypass cache (default: false)
`timeout`	(integer, optional) Max seconds to wait (default: 10, min: 5, max: 30)

⚠️ Important Note

Requires JINA_API_KEY to be set.

🙏 Acknowledgments

Model Context Protocol specification by Anthropic
DuckDuckGo for providing a privacy-focused web search experience
Google Programmable Search Engine and Custom Search JSON API
Jina AI r.reader API for high-quality content extraction
Felo AI for up-to-date, developer-focused search insights
Rust ecosystem and crates that power this server:
- tokio, reqwest, serde, serde_json, tracing, tracing-subscriber, clap
- scraper, html5ever, markup5ever_rcdom, regex, once_cell, futures, async-stream
- url, uuid, thiserror, tokio-util, rand, urlencoding
The broader MCP community for guidance, examples, and discussions