Developing an End-to-End Document Intelligence Pipeline with docTR for OCR, Layout Analysis, KIE, Benchmarking, and Searchable PDFs

Introduction

In this tutorial, you'll learn how to build an end-to-end document intelligence pipeline using docTR, a powerful Python library for document analysis. You'll create a system that can automatically extract text from documents, understand their layout, identify key information, and generate searchable PDFs. This pipeline is perfect for processing invoices, contracts, and other business documents.

What You'll Build

You'll develop a complete document processing workflow that:

• Performs OCR (Optical Character Recognition) on images
• Analyzes document layout to identify text regions
• Extracts key information (KIE) from documents
• Creates searchable PDFs from processed documents

Prerequisites

To follow this tutorial, you'll need:

1. A computer with Python 3.7 or higher installed
2. Basic understanding of Python programming
3. Internet connection for downloading models

Step-by-Step Instructions

1. Setting Up Your Environment

1.1 Install Required Packages

First, we need to install docTR and other required libraries. Open your terminal or command prompt and run:

pip install doctr[torch] pdfplumber pillow

Why: The doctr[torch] package installs docTR with PyTorch support, which is needed for the document analysis models. We also install pdfplumber for PDF handling and pillow for image processing.

1.2 Create a Project Directory

Create a new folder for your project and navigate to it:

mkdir document_intelligence_pipeline
 cd document_intelligence_pipeline

2. Loading and Preparing Documents

2.1 Create a Sample Document

For testing, create a sample document image. You can use any image with text, or download one from the internet. Save it as sample_document.jpg in your project directory.

2.2 Load the Document

Create a Python file called document_processor.py and start with importing required libraries:

import cv2
from doctr.models import ocr_predictor
from doctr.io import DocumentFile
import numpy as np

2.3 Initialize the OCR Model

Next, load the OCR model that will analyze your document:

model = ocr_predictor('db_resnet50', 'crnn_vgg16_bn')

Why: This command loads a pre-trained model for document analysis. The first parameter db_resnet50 is for layout analysis, and the second crnn_vgg16_bn is for text recognition.

3. Performing OCR and Layout Analysis

3.1 Load and Process Your Document

Add the following code to load and process your document:

doc = DocumentFile.from_file('sample_document.jpg')
result = model(doc)

Why: This loads your document image and processes it through the OCR pipeline, returning structured data about text locations and content.

3.2 Extract Text and Layout Information

Now, let's extract the text and layout information:

for page in result.pages:
    print(f'Page {page.page_id}:')
    for block in page.blocks:
        print(f'  Block: {block.geometry}')
        for line in block.lines:
            print(f'    Line: {line.geometry}')
            for word in line.words:
                print(f'      Word: {word.value} at {word.geometry}')

4. Extracting Key Information (KIE)

4.1 Using KIE Models

docTR also supports Key Information Extraction (KIE) models. For this tutorial, we'll use a simple approach to extract specific information:

def extract_key_info(result):
    key_info = {}
    for page in result.pages:
        for block in page.blocks:
            for line in block.lines:
                for word in line.words:
                    # Simple example: extract words that are in uppercase
                    if word.value.isupper() and len(word.value) > 3:
                        key_info[word.value] = word.geometry
    return key_info

key_info = extract_key_info(result)
print('Key Information Extracted:', key_info)

Why: This function demonstrates how to extract specific information from the document by looking for patterns in the text (like uppercase words).

5. Creating Searchable PDFs

5.1 Generate a Searchable PDF

Finally, let's create a searchable PDF from our processed document:

from reportlab.pdfgen import canvas
from reportlab.lib.pagesizes import letter

# Create a new PDF
pdf_path = 'processed_document.pdf'
c = canvas.Canvas(pdf_path, pagesize=letter)
width, height = letter

# Add text to PDF
for page in result.pages:
    for block in page.blocks:
        for line in block.lines:
            for word in line.words:
                x, y = word.geometry[0][0] * width, height - word.geometry[0][1] * height
                c.drawString(x, y, word.value)

c.save()
print(f'Searchable PDF created: {pdf_path}')

Why: This code creates a new PDF and places the extracted text at the correct positions, making the document searchable.

6. Running Your Pipeline

6.1 Execute the Complete Pipeline

Save your document_processor.py file and run it:

python document_processor.py

You should see output showing the extracted text and layout information, followed by the creation of a searchable PDF.

6.2 Test with Different Documents

Try running your pipeline with different document images to see how it handles various layouts and text types.

Summary

In this tutorial, you've built a complete document intelligence pipeline using docTR. You've learned how to:

• Set up the environment with required packages
• Load and process document images
• Perform OCR and layout analysis
• Extract key information from documents
• Create searchable PDFs from processed documents

This pipeline can be extended further by integrating more advanced KIE models, adding custom processing logic, or connecting it to cloud storage for batch processing. You now have a solid foundation for building document intelligence applications.