MCP Server Webscan

🚀 MCP Web Scan Server

A Model Context Protocol (MCP) server for web content scraping, analysis, and information extraction. This server provides tools for retrieving, analyzing, and extracting information from web pages.

✨ Features

• Page Scraping: Convert web pages to Markdown format for easy analysis.
• Link Extraction: Extract and analyze links from web pages.
• Website Crawling: Recursively crawl website content to discover information.
• Error Handling: Comprehensive error handling (invalid parameters, network errors, content parsing errors, URL validation).

📦 Installation

Install with Smithery

# Command to install using Smithery

Build

npm run build

💻 Usage Examples

Development Mode

npm run dev

Project Structure

├── src/
│   ├── initialize.ts
│   └── index.ts    # Main server entry file
├── build/          # Compiled JavaScript (corrected)
├── node_modules/
├── .clinerules
├── .gitignore
├── Dockerfile
├── LICENSE
├── mcp-consistant-servers-guide.md
├── package.json
├── package-lock.json
├── README.md
├── RFC-2025-001-Refactor.md
├── smithery.yaml
└── tsconfig.json

Installation Instructions

Install with Smithery

# Command to install using Smithery

Manual Installation

1. Clone the repository: git clone <repository-url>
2. Navigate to the directory: cd mcp-webscan-server
3. Install dependencies: npm install

Start the Server

node index.ts

Tool List

1. fetch-page

   • Purpose: Convert a web page to Markdown format.
   • Parameters: URL (required).
   • Return Value: Extracted Markdown content.

2. extract-links

   • Purpose: Extract all links from a web page.
   • Parameters: URL (required).
   • Return Value: List of all extracted links.

3. crawl-site

   • Purpose: Recursively crawl website content.
   • Parameters: Starting URL (required), depth limit (optional).
   • Return Value: List of all crawled web page links.

🔧 Error Handling

• Invalid parameters
• Network errors
• Content parsing errors
• URL validation failures

All errors are formatted according to the MCP specification.

🤝 Contribution Guidelines

1. Steps:

   • Fork the repository.
   • Create a feature branch (e.g., git checkout -b feature/amazing-feature).
   • Commit your changes (e.g., git commit -m 'Add some amazing feature').
   • Push to the branch (e.g., git push origin feature/amazing-feature).
   • Submit a Pull Request.

2. Code Standards:

   • Follow the project's existing coding style.
   • Ensure new features pass existing test cases.
   • Provide detailed commit descriptions.

📄 License

MIT License - See the LICENSE file for details.