MCP Webpage Timestamps

🚀 MCP Webpage Timestamps

A powerful Model Context Protocol (MCP) server for extracting webpage creation, modification, and publication timestamps. Ideal for web scraping and temporal analysis of web content.

🚀 Quick Start

This tool can be used for extracting webpage timestamps. You can quickly install it via npm or use it with npx.

✨ Features

• Comprehensive Timestamp Extraction: Extract creation, modification, and publication timestamps from webpages.
• Multiple Data Sources: Support HTML meta tags, HTTP headers, JSON-LD, microdata, OpenGraph, Twitter cards, and heuristic analysis.
• Confidence Scoring: Provide confidence levels (high/medium/low) for extracted timestamps.
• Batch Processing: Extract timestamps from multiple URLs simultaneously.
• Configurable: Customize timeout, user agent, redirect handling, and heuristic options.
• Production Ready: Feature robust error handling, comprehensive logging, and TypeScript support.

📦 Installation

Quick Install

npm install -g mcp-webpage-timestamps

Usage with npx

npx mcp-webpage-timestamps

Installing via Smithery

To automatically install mcp-webpage-timestamps for Claude Desktop via Smithery:

npx -y @smithery/cli install @Fabien-desablens/mcp-webpage-timestamps --client claude

Prerequisites

• Node.js 18.0.0 or higher
• npm or yarn

Development Install

git clone https://github.com/Fabien-desablens/mcp-webpage-timestamps.git
cd mcp-webpage-timestamps
npm install
npm run build

💻 Usage Examples

As MCP Server

The server can be used with any MCP-compatible client. Here's how to configure it:

Claude Desktop Configuration

Add to your claude_desktop_config.json:

{
  "mcpServers": {
    "webpage-timestamps": {
      "command": "npx",
      "args": ["mcp-webpage-timestamps"],
      "env": {}
    }
  }
}

Cline Configuration

Add to your MCP settings:

{
  "mcpServers": {
    "webpage-timestamps": {
      "command": "npx",
      "args": ["mcp-webpage-timestamps"]
    }
  }
}

Direct Usage

# Start the server
npm start

# Or run in development mode
npm run dev

📚 Documentation

API Reference

Tools

extract_timestamps

Extract timestamps from a single webpage.

Parameters:

• url (string, required): The URL of the webpage to extract timestamps from.
• config (object, optional): Configuration options.

Configuration Options:

• timeout (number): Request timeout in milliseconds (default: 10000).
• userAgent (string): User agent string for requests.
• followRedirects (boolean): Whether to follow HTTP redirects (default: true).
• maxRedirects (number): Maximum number of redirects to follow (default: 5).
• enableHeuristics (boolean): Enable heuristic timestamp detection (default: true).

Example:

{
  "name": "extract_timestamps",
  "arguments": {
    "url": "https://example.com/article",
    "config": {
      "timeout": 15000,
      "enableHeuristics": true
    }
  }
}

batch_extract_timestamps

Extract timestamps from multiple webpages in batch.

Parameters:

• urls (array of strings, required): Array of URLs to extract timestamps from.
• config (object, optional): Same configuration options as extract_timestamps.

Example:

{
  "name": "batch_extract_timestamps",
  "arguments": {
    "urls": [
      "https://example.com/article1",
      "https://example.com/article2",
      "https://example.com/article3"
    ],
    "config": {
      "timeout": 10000
    }
  }
}

Response Format

Both tools return a JSON object with the following structure:

{
  url: string;
  createdAt?: Date;
  modifiedAt?: Date;
  publishedAt?: Date;
  sources: TimestampSource[];
  confidence: 'high' | 'medium' | 'low';
  errors?: string[];
}

TimestampSource:

{
  type: 'html-meta' | 'http-header' | 'json-ld' | 'microdata' | 'opengraph' | 'twitter' | 'heuristic';
  field: string;
  value: string;
  confidence: 'high' | 'medium' | 'low';
}

Supported Timestamp Sources

HTML Meta Tags

• article:published_time
• article:modified_time
• date
• pubdate
• publishdate
• last-modified
• dc.date.created
• dc.date.modified
• dcterms.created
• dcterms.modified

HTTP Headers

• Last-Modified
• Date

JSON-LD Structured Data

• datePublished
• dateModified
• dateCreated

Microdata

• datePublished
• dateModified

OpenGraph

• og:article:published_time
• og:article:modified_time
• og:updated_time

Twitter Cards

• twitter:data1 (when containing date information)

Heuristic Analysis

• Time elements with datetime attributes
• Common date patterns in text
• Date-related CSS classes

🔧 Technical Details

Development

Scripts

# Development with hot reload
npm run dev

# Build the project
npm run build

# Run tests
npm test

# Run tests in watch mode
npm run test:watch

# Lint code
npm run lint

# Fix linting issues
npm run lint:fix

# Format code
npm run format

Testing

The project includes comprehensive tests:

# Run all tests
npm test

# Run tests with coverage
npm test -- --coverage

# Run specific test file
npm test -- extractor.test.ts

Code Quality

• TypeScript: Full TypeScript support with strict type checking.
• ESLint: Code linting with recommended rules.
• Prettier: Code formatting.
• Jest: Unit and integration testing.
• 95%+ Test Coverage: Comprehensive test suite.

Examples

Basic Usage

import { TimestampExtractor } from './src/extractor.js';

const extractor = new TimestampExtractor();
const result = await extractor.extractTimestamps('https://example.com/article');

console.log('Published:', result.publishedAt);
console.log('Modified:', result.modifiedAt);
console.log('Confidence:', result.confidence);
console.log('Sources:', result.sources.length);

Custom Configuration

const extractor = new TimestampExtractor({
  timeout: 15000,
  userAgent: 'MyBot/1.0',
  enableHeuristics: false,
  maxRedirects: 3
});

const result = await extractor.extractTimestamps('https://example.com');

Batch Processing

const urls = [
  'https://example.com/article1',
  'https://example.com/article2',
  'https://example.com/article3'
];

const results = await Promise.all(
  urls.map(url => extractor.extractTimestamps(url))
);

Use Cases

• Content Analysis: Analyze temporal aspects of web content.
• Web Scraping: Extract temporal metadata from scraped pages.
• SEO Analysis: Analyze publication and modification patterns.
• Research: Study temporal aspects of web content.
• Content Management: Track content lifecycle and updates.

Error Handling

The extractor handles various error conditions gracefully:

• Network Errors: Timeout, connection refused, DNS resolution failures.
• HTTP Errors: 404, 500, and other HTTP status codes.
• Parsing Errors: Invalid HTML, malformed JSON-LD, unparseable dates.
• Configuration Errors: Invalid URLs, timeout values, etc.

All errors are captured in the errors array of the response, allowing for robust error handling and debugging.

📄 License

This project is under the MIT License - see the LICENSE file for details.

Support

• Issues: GitHub Issues
• Discussions: GitHub Discussions
• Documentation: Wiki

Changelog

See CHANGELOG.md for a detailed history of changes.

Acknowledgments

• Model Context Protocol for the excellent MCP framework.
• Cheerio for HTML parsing.
• Axios for HTTP requests.
• date-fns for date parsing and manipulation.