Linkedin Spider

🚀 LinkedIn Spider

Effortlessly scrape LinkedIn data with zero detection. Extract, export, and automate your LinkedIn data.

🚀 Quick Start

📦 Installation

Choose your preferred installation method:

Option 1: pip (Recommended for general use)

# For Python library only
pip install linkedin-spider

# For CLI usage
pip install linkedin-spider[cli]

# For MCP server usage
pip install linkedin-spider[mcp]

# For all features (CLI + MCP + library)
pip install linkedin-spider[all]

Option 2: Development setup with uv

# Clone the repo
git clone https://github.com/vertexcover-io/linkedin-spider
cd linkedin-spider
# Install with uv
uv sync

⚠️ Important Note

Linkedin has enhanced their anti-bot mechanisms, temporarily affecting cookie-based authentication. We recommend using the email/password authentication method for reliable access. We are actively working on restoring full cookie authentication support.

✨ Features

• 🔍 Search LinkedIn profiles with advanced filters (location, connection type, current company, position)
• 📝 Search and extract LinkedIn posts by keywords with comprehensive metadata
• 📄 Extract complete profile information (experience, education, skills, contact details)
• 🏢 Get company details and information
• 📨 Retrieve incoming and outgoing connection requests
• 📤 Send connection requests to profiles
• 💬 Get conversations list and detailed conversation history
• 🛡️ Built-in anti-detection and session management

💻 Usage Examples

1. Python Library

Perfect for integration into your existing Python applications:

🔍 Basic Usage

from linkedin_spider import LinkedinSpider, ScraperConfig

config = ScraperConfig(headless=True, page_load_timeout=30)

# Authenticate (use either email/password or cookie).
# Authentication is mostly done once and the session is saved in the chrome profile
scraper = LinkedinSpider(
    email="your_email@example.com",
    password="your_password",
    config=config
)

# Search for profiles
results = scraper.search_profiles("software engineer", max_results=10)

Output sample:

[
  {
    "name": "John Doe",
    "title": "Senior Software Engineer at Google",
    "location": "San Francisco, CA",
    "profile_url": "https://linkedin.com/in/johndoe",
    "connections": "500+"
  },
  {
    "name": "Jane Smith",
    "title": "Software Engineer at Microsoft",
    "location": "Seattle, WA",
    "profile_url": "https://linkedin.com/in/janesmith",
    "connections": "200+"
  }
]

# Search for posts by keywords
posts = scraper.search_posts("artificial intelligence", max_results=10, scroll_pause=2.0)

Output sample:

[
  {
    "author_name": "John Doe",
    "author_headline": "AI Research Scientist at OpenAI",
    "author_profile_url": "https://linkedin.com/in/johndoe",
    "connection_degree": "2nd",
    "post_time": "2024-01-15T14:30:00+00:00",
    "post_text": "Excited to share our latest research on [large language models](https://example.com/paper)...",
    "hashtags": ["#AI", "#MachineLearning", "#Research"],
    "links": ["https://example.com/paper"],
    "post_url": "https://linkedin.com/feed/update/urn:li:activity:123456789",
    "media_urls": ["https://media.licdn.com/dms/image/..."],
    "likes_count": 1247,
    "comments_count": 89,
    "reposts_count": 234,
    "comments": [
      {
        "author_name": "Jane Smith",
        "author_profile_url": "https://linkedin.com/in/janesmith",
        "comment_text": "Great insights! Looking forward to reading the full paper.",
        "comment_time": "2024-01-15T15:45:00+00:00",
        "reactions_count": 12
      }
    ]
  }
]

# Scrape individual profile
profile = scraper.scrape_profile("https://linkedin.com/in/someone")

Output sample:

{
  "name": "John Doe",
  "title": "Senior Software Engineer",
  "location": "San Francisco, CA",
  "about": "Passionate software engineer with 8+ years of experience...",
  "experience": [
    {
      "title": "Senior Software Engineer",
      "company": "Google",
      "duration": "2021 - Present",
      "description": "Leading backend development for search infrastructure..."
    }
  ],
  "education": [
    {
      "school": "Stanford University",
      "degree": "BS Computer Science",
      "years": "2013 - 2017"
    }
  ],
  "skills": ["Python", "Java", "Kubernetes", "AWS"]
}

# Scrape company information
company = scraper.scrape_company("https://linkedin.com/company/tech-corp")

Output sample:

{
  "name": "TechCorp Inc",
  "industry": "Software Development",
  "company_size": "1,001-5,000 employees",
  "headquarters": "San Francisco, CA",
  "founded": "2010",
  "specialties": ["Cloud Computing", "AI/ML", "Data Analytics"],
  "description": "Leading technology company focused on enterprise solutions...",
  "website": "https://techcorp.com",
  "follower_count": "45,230"
}

# Don't forget to clean up
scraper.close()

For more examples: examples

2. Command Line Interface

Great for quick data extraction and scripting:

# If installed via pip
# Search for profiles
linkedin-spider-cli search -q "product manager" -n 10 -o results.json --email your@email.com --password yourpassword

# Search for posts
linkedin-spider-cli search-posts -k "artificial intelligence" -n 10 -s 2.0 -o posts.json --email your@email.com --password yourpassword

# Scrape individual profile
linkedin-spider-cli profile -u "https://linkedin.com/in/johndoe" -o profile.json --email your@email.com --password yourpassword

# Scrape company
linkedin-spider-cli company -u "https://linkedin.com/company/openai" -o company.json --email your@email.com --password yourpassword

# Get connection requests
linkedin-spider-cli connections -n 20 -o connections.json --email your@email.com --password yourpassword

# If using development setup
# Search for profiles
uv run linkedin-spider-cli search -q "product manager" -n 10 -o results.json --email your@email.com --password yourpassword

# Search for posts
uv run linkedin-spider-cli search-posts -k "artificial intelligence" -n 10 -s 2.0 -o posts.json --email your@email.com --password yourpassword

# Scrape individual profile
uv run linkedin-spider-cli profile -u "https://linkedin.com/in/johndoe" -o profile.json --email your@email.com --password yourpassword

# Scrape company
uv run linkedin-spider-cli company -u "https://linkedin.com/company/openai" -o company.json --email your@email.com --password yourpassword

# Get connection requests
uv run linkedin-spider-cli connections -n 20 -o connections.json --email your@email.com --password yourpassword

💡 Usage Tip

You typically only need to provide --email and --password once. The CLI saves your authentication session and will reuse it for subsequent commands until the session expires (usually after several hours or days). You can also set LINKEDIN_EMAIL and LINKEDIN_PASSWORD environment variables to avoid typing them repeatedly.

3. MCP Server

Set up environment variables in .env file:

# Authentication (choose one method)
LINKEDIN_EMAIL=your_email@example.com
LINKEDIN_PASSWORD=your_password
# OR
LINKEDIN_COOKIE=your_li_at_cookie_value

# Configuration
HEADLESS=true

# Transport (optional, defaults to stdio)
TRANSPORT=sse
HOST=127.0.0.1
PORT=8000

Start the MCP server:

# If installed via pip
# Show available transport options
linkedin-spider-mcp

# Start with specific transport
linkedin-spider-mcp serve sse --email your@email.com --password yourpassword
linkedin-spider-mcp serve http --host 0.0.0.0 --port 9000 --email your@email.com --password yourpassword
linkedin-spider-mcp serve stdio --email your@email.com --password yourpassword

# Or use environment variables
TRANSPORT=sse linkedin-spider-mcp serve

# If using development setup
# Show available transport options
uv run linkedin-spider-mcp

# Start with specific transport
uv run linkedin-spider-mcp serve sse --email your@email.com --password yourpassword
uv run linkedin-spider-mcp serve http --host 0.0.0.0 --port 9000 --email your@email.com --password yourpassword
uv run linkedin-spider-mcp serve stdio --email your@email.com --password yourpassword

# Or use environment variables
TRANSPORT=sse uv run linkedin-spider-mcp serve

Claude Code Integration

# Add to Claude Code
claude mcp add linkedin-spider --transport sse <server-url>
# Example server URL format: http://localhost:8080/sse

Claude Desktop Integration

Add to your Claude Desktop configuration file:

Windows: %APPDATA%\Claude\claude_desktop_config.json macOS: ~/Library/Application Support/Claude/claude_desktop_config.json Linux: ~/.config/Claude/claude_desktop_config.json

Option 1: Docker (Recommended)

The Docker approach provides reliable, isolated execution with all dependencies included.

First, build the Docker image:

# Build the stdio server image
docker build -f Dockerfile.stdio -t linkedin-mcp-stdio .

Then add this to your Claude Desktop configuration:

{
  "mcpServers": {
    "linkedin-spider": {
      "command": "docker",
      "args": [
        "run",
        "--rm",
        "-i",
        "-e",
        "LINKEDIN_EMAIL=your_email@example.com",
        "-e",
        "LINKEDIN_PASSWORD=your_password",
        "-e",
        "HEADLESS=true",
        "-e",
        "TRANSPORT=stdio",
        "linkedin-mcp-stdio"
      ]
    }
  }
}

🐳 Docker Development & Testing

For development and testing with Docker, you can use a single image with different transport configurations:

🏗️ Build the Docker Image

# Build once for all transport types
docker build -t linkedin-mcp .

🚀 Run with Different Transports

SSE Server

docker run -p 8000:8000 -e TRANSPORT=sse --env-file .env linkedin-mcp

HTTP Server

docker run -p 8000:8000 -e TRANSPORT=http --env-file .env linkedin-mcp

STDIO Server

docker run --rm -i -e TRANSPORT=stdio --env-file .env linkedin-mcp

🔑 Authentication Methods

Method 1: LinkedIn Cookie

1. Login to LinkedIn in your browser
2. Open Developer Tools (F12)
3. Go to Application/Storage → Cookies → linkedin.com
4. Copy the li_at cookie value
5. Use it in your code:

scraper = LinkedinSpider(li_at_cookie="your_cookie_value")

Method 2: Email & Password (Recommended)

scraper = LinkedinSpider(
    email="your_email@example.com",
    password="your_password"
)

🤝 Contributing

We welcome contributions! Please see CONTRIBUTING.md for guidelines.

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

⚠️ Disclaimer

This tool is for personal use only. Please:

• Respect LinkedIn's Terms of Service
• Use reasonable rate limits
• Don't spam or harass users
• Be responsible with the data you collect

Ready to extract LinkedIn data like a pro? Star this repo and start scraping!