Lancedb Vector Search

🚀 LanceDB Node.js Vector Search

A vector search project implemented in Node.js, integrating LanceDB and Ollama's embedding models.

🚀 Quick Start

This project demonstrates how to:

• Connect to a LanceDB database.
• Create a custom embedding function using Ollama.
• Perform vector similarity searches on stored documents.
• Process and display search results.

✨ Features

• Connect to LanceDB and perform vector similarity searches.
• Use Ollama to create custom embedding functions.
• Integrate with Claude Desktop as an MCP service.

📦 Installation

Prerequisites

• Node.js (v14 or higher).
• Ollama running locally with the nomic-embed-text model.
• A LanceDB storage location with read and write permissions.

Steps

1. Clone the repository.
2. Install dependencies:

pnpm install

📚 Documentation

Dependencies

• @lancedb/lancedb: LanceDB client for Node.js.
• apache-arrow: Used for processing columnar data.
• node-fetch: Used to send requests to the Ollama API.

Usage

Run the vector search test script:

pnpm test-vector-search

Or execute directly:

node test-vector-search.js

Configuration

The script connects to:

• The specified path of LanceDB.
• The Ollama API running at http://localhost:11434/api/embeddings.

MCP Configuration

To integrate it into Claude Desktop as an MCP service, add the following to your MCP configuration JSON file:

{
  "mcpServers": {
    "lanceDB": {
      "command": "node",
      "args": [
        "/path/to/lancedb-node/dist/index.js",
        "--db-path",
        "/path/to/your/lancedb/storage"
      ]
    }
  }
}

Replace the paths with your actual installation paths:

• /path/to/lancedb-node/dist/index.js: The actual path of the index.js file.
• /path/to/your/lancedb/storage: The path of your LanceDB storage directory.

Custom Embedding Function

The project includes a custom OllamaEmbeddingFunction that:

• Sends text to the Ollama API.
• Receives 768-dimensional embeddings.
• Formats them into a data format suitable for LanceDB.

Vector Search Example

The example demonstrates how to search for "How to define success criteria" in the "ai-rag" table and display the results along with their similarity scores.

📄 License

MIT License

💡 Contribution

Contributions are welcome! Feel free to submit a Pull Request.