Local Wikipedia

🚀 Local-Wikipedia

Local-Wikipedia is an MCP server that enables you to access Wikipedia offline. After a one-time quick download, you can search for and read articles at your convenience. Here's why it stands out:

1. Full-text Search across Wikipedia
   • Most MCP servers only support exact title matches and basic redirect handling.
   • Since Local-Wikipedia stores the full text locally, you can conduct true full-text searches.
2. Offline Access after One-time Download
   • With data saved locally, you can search and read Wikipedia articles even without an internet connection.
3. High Search Frequency Handling
   • There are no web API rate limits, allowing you to iterate and refine your searches as much as you want.
   • It's ideal for use cases that flexibly interpret queries and perform repeated searches.
4. Easy to Extend
   • Having the entire corpus locally makes it easier to implement features that are difficult on other MCP servers.
   • Other MCP servers may be restricted by the web APIs they rely on, but Local-Wikipedia is not.
   • For example, adding a "fetch a random article" feature is straightforward.

This MCP server is well-suited for small local LLMs. Even when large LLMs are unavailable, you can still search Wikipedia flexibly and retrieve information. We tested it with a compact, mobile-oriented model called Gemma 3n E4B, and it runs quickly even on CPU-only environments.

To be friendly to small LLMs, we made the following thoughtful choices:

1. Only two tools with minimal arguments, ensuring reliable calls from LLMs.
2. Heuristic query correction to clean up over-specified or noisy inputs from an LLM.
3. Concise, situation-aware outputs for fast performance even with short context windows and limited compute.
4. Advanced DB indexing for fast search and low memory usage. The initial setup takes a bit longer, but once completed, it's very fast.

🚀 Quick Start

The following steps will guide you through setting up Local-Wikipedia.

✨ Features

This MCP server offers the following two tools:

search_article

Flexibly searches for an article by the specified title and returns appropriate results. Multiple search strategies are integrated into a single tool to simplify tool use for LLMs.

Search methods include:

1. Exact title match
2. Exact redirect match
3. Partial title match
4. Partial redirect match
5. Full-text search of the body

For 1 and 2, the article’s lead section (summary) is returned. For 3, 4, and 5, up to 20 results are returned.

There's also a heuristic query-fix feature to handle over-specified or noisy inputs from an LLM.

read_random_article

Fetches a random article from the specified language edition of Wikipedia.

📦 Installation

Everything can be set up using Docker Compose. Follow these steps:

1. Ensure Docker and Docker Compose are installed.
2. Clone the repository:

git clone https://github.com/soukouki/local-wikipedia.git

3. Navigate to the project directory:

cd local-wikipedia

4. Start the server:

docker-compose up

On the first run, the Wikipedia data for the language specified in config.yaml will be downloaded and indexed. This process may take some time - approximately tens of minutes for Japanese and several hours for English. We recommend using a stable fiber connection for the download.

Once the setup is complete, the MCP server will start. By default, it listens on port 29423. You can connect to it as follows:

{
  "servers": {
    "local-wikipedia": {
      "url": "http://localhost:29423"
    }
  }
}

🔧 Technical Details

Local-Wikipedia utilizes the official Wikipedia dump data for pages and redirects, along with the Markdown-formatted full-text dataset published at HuggingFaceFW/finewiki · Datasets at Hugging Face.

For full-text search, it employs PGroonga to provide fast, memory-efficient search in both Japanese and English.

📄 License

Wikipedia content is licensed under both the Creative Commons Attribution-ShareAlike 4.0 International License (CC BY-SA 4.0) and the GNU Free Documentation License (GFDL).

⚠️ Important Note

The current Local-Wikipedia implementation is not designed for public API access. When exposing the API externally, we recommend making appropriate code modifications for security purposes.