Using Lift to Turn Research PDFs into Structured JSON with Controlled, Schema-Guided Field-Level Evaluation

Introduction

In this tutorial, we'll learn how to use Lift, a powerful tool for extracting structured data from research PDFs. We'll build a complete workflow that transforms unstructured PDF documents into organized JSON data, with a focus on controlled evaluation. This is particularly useful for researchers who want to systematically analyze large volumes of academic papers and extract specific information like author names, publication dates, and key findings.

By the end of this tutorial, you'll have a working environment that can process PDFs, extract data according to a defined schema, and evaluate the accuracy of the extraction. This approach allows for repeatable benchmarking rather than just getting raw model outputs.

Prerequisites

Before starting this tutorial, you'll need:

• A Google Colab account (free to create)
• Basic understanding of Python and data structures
• Access to a GPU runtime in Colab (we'll set this up)

Don't worry if you're new to these concepts - we'll walk through everything step by step.

Step-by-Step Instructions

1. Set Up Your Colab Environment

First, we need to create a GPU-enabled runtime environment in Colab. This will give us the computational power needed to run large language models efficiently.

# Install required packages
!pip install -q accelerate transformers datasets torch
!pip install -q lift

Why: These packages provide the foundation for working with large language models and the Lift framework specifically. The -q flag suppresses output to keep our notebook clean.

2. Initialize the Lift Framework

Now we'll load Lift with 4-bit quantization, which allows us to run large models on limited hardware:

from lift import Lift

# Initialize Lift with 4-bit quantization
lift = Lift(model_name="meta-llama/Llama-3.2-3B-Instruct", quantization="4bit")

Why: 4-bit quantization reduces memory usage and speeds up inference while maintaining good performance. This is essential for running models on consumer hardware or cloud instances with limited resources.

3. Create Sample Research PDF Data

Before extracting data, we need some sample PDF documents to work with. We'll generate synthetic research reports:

import json

# Create sample research data
sample_data = {
    "title": "Advancements in Neural Network Architectures",
    "authors": ["Dr. Jane Smith", "Prof. John Doe"],
    "publication_date": "2026-07-01",
    "abstract": "This paper explores novel approaches to improving neural network performance through innovative architectural designs.",
    "key_findings": ["Architecture A improved accuracy by 12%", "Method B reduced training time by 30%"],
    "journal": "Journal of AI Research"
}

# Save as JSON for later use
with open('sample_research.json', 'w') as f:
    json.dump(sample_data, f, indent=2)

Why: We're creating a realistic dataset structure that mimics actual research papers. This will serve as our test data for extraction.

4. Define the Extraction Schema

We need to specify exactly what information we want to extract from each PDF:

# Define the schema for our extraction
extraction_schema = {
    "type": "object",
    "properties": {
        "title": {"type": "string"},
        "authors": {"type": "array", "items": {"type": "string"}},
        "publication_date": {"type": "string", "format": "date"},
        "abstract": {"type": "string"},
        "key_findings": {"type": "array", "items": {"type": "string"}},
        "journal": {"type": "string"}
    },
    "required": ["title", "authors", "publication_date"]
}

Why: A schema ensures consistent data structure and helps the model understand exactly what information to extract. This is crucial for controlled evaluation.

5. Run Schema-Guided Extraction

Now we'll use Lift to extract information from our sample data using the defined schema:

# Extract data from our sample
extracted_data = lift.extract(
    data=sample_data,
    schema=extraction_schema,
    task="structured_extraction"
)

print("Extracted Data:")
print(json.dumps(extracted_data, indent=2))

Why: This step applies our defined schema to the input data, ensuring that the model extracts only the fields we care about and formats them correctly.

6. Evaluate Field-Level Accuracy

To assess how well our extraction worked, we'll compare the results against ground truth:

def evaluate_extraction(generated, ground_truth):
    """Compare extracted data with ground truth"""
    evaluation = {}
    for key in ground_truth:
        if key in generated:
            evaluation[key] = {
                "expected": ground_truth[key],
                "extracted": generated[key],
                "match": generated[key] == ground_truth[key]
            }
        else:
            evaluation[key] = {
                "expected": ground_truth[key],
                "extracted": None,
                "match": False
            }
    return evaluation

# Evaluate our extraction
ground_truth = sample_data
evaluation_results = evaluate_extraction(extracted_data, ground_truth)

print("Evaluation Results:")
print(json.dumps(evaluation_results, indent=2))

Why: Field-level evaluation gives us granular insights into which parts of our extraction worked well and which didn't. This is more informative than overall accuracy metrics.

7. Create a Queryable Knowledge Base

Finally, we'll organize our results into a structured format that can be easily queried:

# Create a simple database structure
knowledge_base = {
    "papers": [extracted_data],
    "evaluation": evaluation_results,
    "metadata": {
        "total_papers": 1,
        "extraction_date": "2026-07-01"
    }
}

# Save the knowledge base
with open('knowledge_base.json', 'w') as f:
    json.dump(knowledge_base, f, indent=2)

print("Knowledge base created successfully!")

Why: A structured knowledge base allows us to easily search, filter, and analyze extracted information. This makes the extracted data useful for research and analysis.

Summary

In this tutorial, we've built a complete PDF-to-structured-data workflow using Lift. We:

1. Set up a GPU-enabled Colab environment
2. Loaded Lift with 4-bit quantization for efficient model execution
3. Created sample research data in JSON format
4. Defined a structured schema for extraction
5. Performed schema-guided extraction of key information
6. Evaluated field-level accuracy against ground truth
7. Assembled results into a queryable knowledge base

This approach creates a repeatable benchmark for PDF data extraction rather than just getting raw outputs. The controlled evaluation framework allows researchers to systematically assess and improve their extraction pipelines over time.