Linkedin Scraper MCP

🚀 LinkedIn Scraper MCP Server

A Model Context Protocol (MCP) server enabling LinkedIn profile scraping with manual credential input. It extracts comprehensive profile data such as experience, education, skills, and contact information.

🚀 Quick Start

The LinkedIn Scraper MCP Server allows you to scrape LinkedIn profiles by providing manual credentials. It offers multiple transport methods and can extract a wide range of data from profiles.

✨ Features

• Manual Credential Input: No need to set environment variables. Provide LinkedIn credentials directly when calling the tool.
• Comprehensive Data Extraction: Scrapes name, headline, location, about section, experience, education, skills, and contact information.
• Multiple Transport Methods: Supports both HTTP (for production) and STDIO (for development) transports.
• Browser Automation: Utilizes Selenium WebDriver with Chrome for reliable scraping.
• Anti - Detection: Incorporates human - like delays and browser settings to avoid detection.

📦 Installation

1. Install dependencies:

npm install

2. Install Chrome browser (if not already installed):

   • The server uses Chrome WebDriver, which will be automatically managed.
   • Ensure that the Chrome browser is installed on your system.

3. Build the project:

npm run build

💻 Usage Examples

Basic Usage

HTTP Transport (Recommended)

Start the server with HTTP transport:

npm start
# or
node dist/index.js

The server will start on http://localhost:8080 by default.

STDIO Transport (Development)

For local development with STDIO transport:

npm run start:stdio
# or
node dist/index.js --stdio

Advanced Usage

Command Line Options

• --port <PORT>: Specify the HTTP server port (default: 8080).
• --stdio: Use STDIO transport instead of HTTP.
• --help: Show the help message.

📚 Documentation

MCP Client Configuration

Add this to your MCP client configuration:

{
  "mcpServers": {
    "linkedin-scraper": {
      "url": "http://localhost:8080/mcp"
    }
  }
}

Available Tools

scrape_linkedin_profile

Scrapes a LinkedIn profile and returns comprehensive profile data.

Parameters:

• url (required): LinkedIn profile URL (e.g., "https://www.linkedin.com/in/username/").
• email (required): LinkedIn account email for authentication.
• password (required): LinkedIn account password for authentication.
• headless (optional): Run the browser in headless mode (default: false).

Example Usage:

{
  "tool": "scrape_linkedin_profile",
  "arguments": {
    "url": "https://www.linkedin.com/in/johndoe/",
    "email": "your-email@example.com",
    "password": "your-password",
    "headless": false
  }
}

Response Format:

{
  "success": true,
  "timestamp": "2024-01-01T12:00:00.000Z",
  "profile": {
    "url": "https://www.linkedin.com/in/johndoe/",
    "name": "John Doe",
    "headline": "Software Engineer at Tech Company",
    "location": "San Francisco, CA",
    "about": "Passionate software engineer...",
    "experience_count": 3,
    "experiences": [...],
    "education_count": 2,
    "education": [...],
    "skills_count": 15,
    "skills": [...],
    "websites": [...],
    "email": "john@example.com"
  }
}

Development

Project Structure

src/
├── index.ts            # Main entry point
├── cli.ts              # Command-line argument parsing
├── config.ts           # Configuration management
├── server.ts           # Server instance creation
├── client.ts           # LinkedIn scraper client
├── types.ts            # TypeScript type definitions
├── tools/
│   ├── index.ts        # Tool exports
│   └── linkedin.ts     # LinkedIn scraping tool
└── transport/
    ├── index.ts        # Transport exports
    ├── http.ts         # HTTP transport
    └── stdio.ts        # STDIO transport

Building

npm run build       # Build once
npm run watch       # Build and watch for changes

Running in Development

npm run dev         # Build and run HTTP transport
npm run dev:stdio   # Build and run STDIO transport

Security Considerations

• Credential Handling: LinkedIn credentials are passed directly in tool calls and not stored.
• Rate Limiting: The scraper includes human - like delays to avoid being blocked.
• Browser Settings: Uses realistic browser settings to minimize detection.
• Session Management: Each scraping session is isolated and cleaned up properly.

Troubleshooting

Common Issues

1. Chrome Driver Issues: The server automatically manages Chrome WebDriver, but ensure that the Chrome browser is installed.
2. LinkedIn Authentication: If authentication fails:
   • Verify that your LinkedIn credentials are correct.
   • Check if your account has two - factor authentication enabled.
   • Try logging in manually first to resolve any security challenges.
3. Scraping Failures: If scraping fails:
   • Try running with headless: false to see what's happening.
   • Check if LinkedIn has updated their page structure.
   • Ensure a stable internet connection.
4. Port Conflicts: If port 8080 is in use:

   node dist/index.js --port 3000
   

Environment Variables

Optional environment variables:

• PORT: HTTP server port (default: 8080).
• NODE_ENV: Set to 'production' for production mode.

📄 License

This project is for educational and research purposes. Please respect LinkedIn's Terms of Service and use it responsibly.

Contributing

1. Fork the repository.
2. Create a feature branch.
3. Make your changes.
4. Test thoroughly.
5. Submit a pull request.

Disclaimer

This tool is for educational purposes only. Users are responsible for complying with LinkedIn's Terms of Service and applicable laws. The authors are not responsible for any misuse of this software.