Duckduckgo Search

🚀 MCP Duck 🦆 Plugin

The MCP Duck 🦆 Plugin is a tool designed to enhance content extraction and web crawling capabilities. It supports content scraping through DuckDuckGo searches and offers rich data processing capabilities.

🚀 Quick Start

To get started with the MCP Duck 🦆 Plugin, you can follow the installation and usage steps below.

📦 Installation

Quick Installation

pip install mcp-duckduckgo

Clone the Repository (for Development)

If you need to modify the project or contribute code, you can clone the repository first:

git clone https://github.com/gianlucamazza/mcp-duckduckgo.git
cd mcp-duckduckgo
pip install -e .

💻 Usage Examples

Initialize the MCP Server

Run the following command to start the MCP server:

mcp-duckduckgo serve --port=8000

Install Dependencies

In the project root directory, install all dependencies:

pip install -r requirements.txt

✨ Features

Enhanced Content Extraction

The plugin provides powerful content analysis and scraping capabilities:

• Web Page Analysis: Extract and parse HTML content from DuckDuckGo search results.
• Intelligent Content Recognition: Automatically locate the main information area based on the website type.
• Metadata Extraction: Extract information such as title, description, author, keywords, and publication date.
• Image Detection: Identify the main images and media resources on the web page.
• Social Link Extraction: Retrieve the embedded social media links on the web page.
• Content Structure Analysis: Extract headings and sections from the web page to help understand the page organization.
• Official Source Identification: Determine whether the information source is an official channel through domain names and content signals.

Web Crawling Function

The plugin also provides basic web crawler functionality:

• Configurable Depth: Traverse up to 3 levels of links starting from the original URL.
• Link Limit: Control the maximum number of links to be crawled per page (1 - 5).
• Domain Limit: Optionally enable the "Crawl only within the same domain" option.
• Related Content Discovery: Analyze and extract related web page content based on the crawling results.

📚 Documentation

Project Structure

The project contains the following main files:

mcp_duckduckgo/
├── mcp_duckduckgo/    # Main plugin code
│   ├── models.py       # Data model definition
│   ├── search.py       # Search function implementation
│   └── server.py       # MCP server implementation
└── scripts/            # Development scripts
    ├── install_dev.sh  # Set up the development environment
    ├── run.sh          # Run the MCP server in development mode
    ├── test.sh         # Execute test cases and generate a coverage report
    └── lint.sh         # Perform code format checking and fixing

Development Scripts

The project provides some convenient scripts located in the scripts/ directory:

• install_dev.sh: Install the dependencies required for the development environment.
• run.sh: Run the MCP server in development mode.
• test.sh: Execute test cases and generate a coverage report.
• lint.sh: Perform code format checking and fixing.

Testing Guide

To execute tests in the project, you can run the following command:

python -m pytest tests/ --cov=mcp_duckduckgo/

Dependency Management

The project's dependency information is recorded in the requirements.txt file. Install all dependencies:

pip install -r requirements.txt

Contribution Guide

Any contributions are welcome! Please follow these steps:

1. Clone the repository: git clone https://github.com/gianlucamazza/mcp-duckduckgo.git
2. Create a feature branch: git checkout -b feature/your-feature-name
3. Commit your changes: git add . && git commit -m "Add new feature"
4. Push to the remote repository: git push origin feature/your-feature-name
5. Submit a Pull Request to the main branch.

📄 License

The project uses the MIT License, and the specific content is as follows:

MIT License

Thank you for your interest in the MCP Duck 🦆 Plugin! If you have any questions or suggestions, please feel free to contact us.