Zyphra Releases ZUNA1.1: An Apache 2.0 EEG Foundation Model With Variable-Length Inputs From 0.5 To 30 Seconds

Introduction

In this tutorial, we'll explore how to work with Zyphra's ZUNA1.1, an Apache 2.0 licensed EEG foundation model that supports variable-length inputs from 0.5 to 30 seconds. This advancement allows researchers and developers to process EEG signals with greater flexibility compared to previous fixed-length models. We'll walk through setting up the environment, loading the model, and processing EEG data with variable lengths.

Prerequisites

• Python 3.8 or higher
• Basic understanding of EEG signal processing
• Knowledge of PyTorch and neural networks
• Installed packages: torch, numpy, scipy, huggingface_hub

Step-by-Step Instructions

1. Setting Up the Environment

First, we need to install the required dependencies for working with ZUNA1.1. The model is available through the Hugging Face Hub, so we'll need to install the necessary packages.

1.1 Install Required Packages

pip install torch numpy scipy huggingface_hub

This command installs the core libraries needed for working with the model and EEG data. PyTorch is essential for model inference, while numpy and scipy provide the mathematical tools for signal processing.

1.2 Import Libraries

import torch
import numpy as np
from huggingface_hub import snapshot_download
from transformers import AutoModel, AutoTokenizer

We import the necessary libraries for model loading, tensor operations, and Hugging Face integration. The AutoModel and AutoTokenizer classes will help us load the ZUNA1.1 model and its associated tokenizer.

2. Loading the ZUNA1.1 Model

Now we'll load the ZUNA1.1 model from the Hugging Face Hub. This model supports variable-length inputs, which is a key feature we'll leverage.

2.1 Download Model Files

model_name = "zyphra/ZUNA1.1"
model_path = snapshot_download(repo_id=model_name, revision="main")

The snapshot_download function downloads the complete model repository from Hugging Face. This ensures we have all necessary files including the model weights and configuration.

2.2 Load the Model

model = AutoModel.from_pretrained(model_path)
model.eval()

We load the model using the AutoModel.from_pretrained method and set it to evaluation mode. The model will be ready for inference once loaded.

3. Preparing EEG Data

EEG data needs to be preprocessed before feeding it into the model. We'll create synthetic EEG data to demonstrate the process.

3.1 Generate Synthetic EEG Data

# Generate synthetic EEG data with variable lengths
sample_rate = 128  # Hz
lengths = [0.5, 1.0, 2.0, 5.0, 10.0, 30.0]  # seconds

# Create EEG data for each length
eeg_data = {}
for length in lengths:
    n_samples = int(length * sample_rate)
    # Generate random EEG-like data (32 channels)
    data = np.random.randn(n_samples, 32) * 100
    eeg_data[length] = data

This code generates synthetic EEG data with varying lengths. The data represents 32 EEG channels collected at 128 Hz sample rate. Each dataset is stored in a dictionary with its length as the key.

3.2 Preprocess Data for Model Input

def preprocess_eeg(data, sample_rate):
    # Convert to tensor
    tensor_data = torch.tensor(data, dtype=torch.float32)
    # Normalize the data (optional but recommended)
    mean = tensor_data.mean()
    std = tensor_data.std()
    normalized_data = (tensor_data - mean) / std
    return normalized_data

# Preprocess all EEG data
processed_data = {}
for length, data in eeg_data.items():
    processed_data[length] = preprocess_eeg(data, sample_rate)

Here we convert the EEG data to PyTorch tensors and normalize it. Normalization helps ensure consistent input to the model, which can improve performance and stability.

4. Processing Variable-Length EEG Inputs

The key advantage of ZUNA1.1 is its ability to handle variable-length inputs. We'll demonstrate how to process different lengths of EEG data.

4.1 Process Each EEG Length

results = {}
for length, data in processed_data.items():
    print(f"Processing {length} seconds of EEG data")
    
    # Add batch dimension
    batch_data = data.unsqueeze(0)  # Shape: (1, n_samples, n_channels)
    
    # Forward pass through the model
    with torch.no_grad():
        output = model(batch_data)
        
    # Store results
    results[length] = output
    print(f"Output shape: {output.shape}")

We process each EEG length individually. The model expects a batch dimension, so we add one using unsqueeze(0). The torch.no_grad() context ensures we don't track gradients, which is more efficient for inference.

4.2 Handle Variable-Length Inputs

# Demonstrate that model handles variable lengths
print("\nVariable-length input demonstration:")
print(f"0.5 seconds input shape: {processed_data[0.5].shape}")
print(f"30 seconds input shape: {processed_data[30.0].shape}")

# Process short and long inputs
short_output = model(processed_data[0.5].unsqueeze(0))
long_output = model(processed_data[30.0].unsqueeze(0))

print(f"Short input output shape: {short_output.shape}")
print(f"Long input output shape: {long_output.shape}")

This code demonstrates that ZUNA1.1 can handle both short (0.5 seconds) and long (30 seconds) inputs. The model's architecture allows it to process inputs of varying lengths without retraining.

5. Analyzing Results

After processing the EEG data, we'll analyze the outputs to understand what the model has learned.

5.1 Examine Output Statistics

# Analyze the outputs
for length, output in results.items():
    print(f"\nResults for {length} seconds:")
    print(f"Output mean: {output.mean().item():.4f}")
    print(f"Output std: {output.std().item():.4f}")
    print(f"Output shape: {output.shape}")

The output statistics help us understand the model's behavior with different input lengths. These statistics should be consistent across different lengths, demonstrating the model's robustness.

5.2 Visualize Results

import matplotlib.pyplot as plt

# Plot a sample output
fig, axes = plt.subplots(2, 3, figsize=(15, 10))
axes = axes.ravel()

for i, (length, output) in enumerate(list(results.items())[:6]):
    # Plot first channel of output
    axes[i].plot(output[0, :, 0].numpy())
    axes[i].set_title(f'{length}s EEG Output')
    axes[i].set_xlabel('Time')
    axes[i].set_ylabel('Amplitude')

plt.tight_layout()
plt.show()

This visualization shows how the model processes EEG data of different lengths. The plots help us understand the temporal dynamics of the model's outputs.

6. Practical Applications

ZUNA1.1's variable-length capability opens up several practical applications:

• Real-time EEG analysis where processing time varies
• Long-term monitoring with flexible input durations
• Integration with other EEG processing pipelines

Summary

In this tutorial, we've explored how to work with Zyphra's ZUNA1.1 EEG foundation model. We've demonstrated how to set up the environment, load the model, preprocess EEG data, and process variable-length inputs. The model's ability to handle inputs from 0.5 to 30 seconds represents a significant advancement in EEG processing capabilities, allowing for more flexible and practical applications in research and clinical settings.

By following these steps, you can now implement variable-length EEG processing in your own projects. The key advantages of ZUNA1.1 include its Apache 2.0 licensing, support for arbitrary channel layouts, and the ability to reconstruct, denoise, and upsample scalp-EEG signals efficiently.