Meet LEANN: An Open-Source Vector Index that Shrinks a 60M-Chunk RAG Index from 201 GB to 6 GB
Back to Tutorials
aiTutorialbeginner

Meet LEANN: An Open-Source Vector Index that Shrinks a 60M-Chunk RAG Index from 201 GB to 6 GB

October 27, 202631 views5 min read

Learn how to work with LEANN, UC Berkeley's open-source vector index that reduces storage requirements for RAG systems from 201 GB to 6 GB while maintaining high accuracy.

Introduction

In this tutorial, you'll learn how to work with LEANN (Low-Entropy Approximate Nearest Neighbor), a new open-source vector index developed by UC Berkeley that dramatically reduces storage requirements for large-scale RAG (Retrieval-Augmented Generation) systems. LEANN uses a clever approach of storing a pruned graph and recomputing embeddings at query time, reducing a 201 GB index to just 6 GB while maintaining high accuracy. This tutorial will guide you through setting up LEANN and performing basic vector search operations.

Prerequisites

  • Basic understanding of Python programming
  • Python 3.7 or higher installed on your system
  • Basic knowledge of vector databases and similarity search concepts
  • Internet connection for downloading packages

Step-by-Step Instructions

Step 1: Install Required Dependencies

First, we need to install the necessary Python packages for working with LEANN. Open your terminal or command prompt and run:

pip install numpy scikit-learn

This installs NumPy for numerical operations and scikit-learn for machine learning utilities. LEANN will work with these libraries to handle vector operations.

Step 2: Create a Simple Vector Dataset

Before we can use LEANN, we need some sample vectors to index. Create a new Python file called vector_dataset.py and add the following code:

import numpy as np

# Create sample vectors for demonstration
np.random.seed(42)  # For reproducible results

def create_sample_vectors(num_vectors=1000, vector_dim=128):
    # Generate random vectors
    vectors = np.random.rand(num_vectors, vector_dim)
    # Normalize vectors to unit length for cosine similarity
    vectors = vectors / np.linalg.norm(vectors, axis=1, keepdims=True)
    return vectors

# Create 1000 vectors of dimension 128
sample_vectors = create_sample_vectors(1000, 128)
print(f"Created {len(sample_vectors)} vectors of dimension {sample_vectors.shape[1]}")

This code creates 1000 random vectors, each with 128 dimensions. We normalize them to unit length to make similarity calculations more meaningful.

Step 3: Set Up LEANN Index

Now we'll create a basic LEANN index. Create a new file called leann_index.py:

import numpy as np
from sklearn.neighbors import NearestNeighbors

# Simple implementation of LEANN-like approach
# In a real LEANN implementation, you'd use the actual library

class SimpleLEANN:
    def __init__(self, vectors, n_neighbors=10):
        self.vectors = vectors
        self.n_neighbors = n_neighbors
        self.nbrs = NearestNeighbors(n_neighbors=n_neighbors, algorithm='ball_tree')
        self.nbrs.fit(vectors)
        
    def search(self, query_vector, k=5):
        # Find k nearest neighbors
        distances, indices = self.nbrs.kneighbors([query_vector])
        return indices[0], distances[0]

# Create sample vectors
sample_vectors = create_sample_vectors(1000, 128)

# Initialize LEANN index
leann_index = SimpleLEANN(sample_vectors)
print("LEANN index initialized successfully")

This creates a simplified LEANN implementation using scikit-learn's NearestNeighbors. In a real LEANN implementation, you would use the actual LEANN library which stores a pruned graph structure.

Step 4: Perform Vector Search

Let's add code to search for similar vectors in our index:

# Create a query vector
query_vector = np.random.rand(128)
query_vector = query_vector / np.linalg.norm(query_vector)

# Perform search
indices, distances = leann_index.search(query_vector, k=5)

print(f"Query vector: {query_vector[:5]}...")
print(f"Top 5 similar vectors indices: {indices}")
print(f"Distances: {distances}")

# Show some information about the results
print(f"\nFirst similar vector: {sample_vectors[indices[0]][:5]}...")
print(f"Second similar vector: {sample_vectors[indices[1]][:5]}...")

This search operation demonstrates how LEANN finds similar vectors by computing distances between the query vector and stored vectors.

Step 5: Measure Storage Efficiency

Let's compare storage requirements to understand LEANN's efficiency:

# Calculate storage requirements
vector_size_bytes = sample_vectors.nbytes
print(f"Original vectors storage: {vector_size_bytes / (1024**3):.2f} GB")

# In LEANN, you'd store only the pruned graph structure
# For demonstration, let's assume LEANN uses 1% of original storage
leann_storage = vector_size_bytes * 0.01
print(f"LEANN storage (pruned graph): {leann_storage / (1024**3):.2f} GB")

# Calculate space reduction
reduction_ratio = vector_size_bytes / leann_storage
print(f"Space reduction: {reduction_ratio:.1f}x smaller")

This shows how LEANN dramatically reduces storage requirements by storing only a pruned graph structure instead of all vectors.

Step 6: Run the Complete Example

Create a main script main.py that combines everything:

from vector_dataset import create_sample_vectors
from leann_index import SimpleLEANN

# Create sample dataset
print("Creating sample vectors...")
sample_vectors = create_sample_vectors(1000, 128)

# Initialize LEANN index
print("Initializing LEANN index...")
leann_index = SimpleLEANN(sample_vectors)

# Perform search
print("Performing vector search...")
query_vector = np.random.rand(128)
query_vector = query_vector / np.linalg.norm(query_vector)

indices, distances = leann_index.search(query_vector, k=5)
print(f"Search results: indices {indices}, distances {distances}")

# Show storage comparison
vector_size_bytes = sample_vectors.nbytes
leann_storage = vector_size_bytes * 0.01
print(f"\nStorage comparison:")
print(f"Original storage: {vector_size_bytes / (1024**3):.2f} GB")
print(f"LEANN storage: {leann_storage / (1024**3):.2f} GB")
print(f"Space reduction: {vector_size_bytes / leann_storage:.1f}x")

This complete example demonstrates the core concepts of LEANN's approach to vector search with reduced storage.

Step 7: Understanding LEANN's Key Concepts

LEANN works by:

  1. Pruning the Graph: Instead of storing all vectors, it creates a pruned graph that maintains the essential similarity relationships
  2. Recomputing Embeddings: At query time, it recomputes embeddings rather than storing them, saving significant space
  3. Efficient Search: The pruned graph structure allows for fast search operations

While our example uses a simplified approach, real LEANN implementations achieve 90% recall accuracy with 4GB storage instead of 188GB, at the cost of slightly slower search times.

Summary

In this tutorial, you've learned how to work with LEANN (Low-Entropy Approximate Nearest Neighbor) vector indexing. You've created sample vectors, set up a basic LEANN-like index, performed vector searches, and compared storage requirements. LEANN's approach of storing pruned graphs and recomputing embeddings at query time dramatically reduces storage needs from 201GB to just 6GB while maintaining high accuracy. This makes it ideal for large-scale RAG systems where storage efficiency is crucial. While we used simplified implementations for demonstration, real LEANN systems provide the foundation for building efficient vector search systems at scale.

Source: MarkTechPost

Related Articles