AI chatbots reading X-rays can be dangerously confident even when they're wrong

Learn how to build a basic AI model that analyzes chest X-rays and determines confidence levels in its diagnoses, demonstrating why AI systems must know when to defer to human expertise.

Introduction

In this tutorial, you'll learn how to create a basic AI model that can analyze chest X-rays and determine whether it's confident enough in its diagnosis to make a recommendation. This is an important step toward building reliable AI systems for medical diagnosis, as we've seen from recent research that AI models can be dangerously confident when they're actually wrong.

By the end of this tutorial, you'll have built a simple AI model that can analyze X-ray images and output both a diagnosis and a confidence score. This will help you understand why it's crucial for AI systems to know when they're uncertain.

Prerequisites

Before starting this tutorial, you'll need:

A computer with internet access
Basic understanding of Python programming
Some familiarity with machine learning concepts (no advanced math required)
Python 3.7 or higher installed
Access to a dataset of chest X-ray images (we'll use a sample dataset)

Why these prerequisites? Python is the most common language for AI development, and understanding basic ML concepts will help you grasp how the model works. We'll be using a simplified approach so no prior deep learning experience is required.

Step-by-Step Instructions

Step 1: Set Up Your Python Environment

First, create a new folder for your project and open a terminal or command prompt in that folder. Then, install the required packages:

pip install tensorflow numpy matplotlib scikit-learn

Why this step? These packages provide the tools we need to build and test our AI model. TensorFlow is the main deep learning library, while NumPy and Matplotlib help with data manipulation and visualization.

Step 2: Prepare Your Dataset

For this tutorial, we'll use a simplified version of a chest X-ray dataset. Create a folder called data in your project directory, and inside it, create two subfolders: positive and negative.

The positive folder should contain X-ray images showing signs of pneumonia, and the negative folder should contain images showing healthy lungs.

Why this step? Having a labeled dataset is essential for training an AI model to recognize patterns in X-ray images. The labels (positive/negative) tell our model what to look for.

Step 3: Create the AI Model

Create a new file called model.py and add the following code:

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
import numpy as np
import matplotlib.pyplot as plt

# Create a simple CNN model
model = keras.Sequential([
    layers.Conv2D(32, (3, 3), activation='relu', input_shape=(224, 224, 3)),
    layers.MaxPooling2D((2, 2)),
    layers.Conv2D(64, (3, 3), activation='relu'),
    layers.MaxPooling2D((2, 2)),
    layers.Conv2D(64, (3, 3), activation='relu'),
    layers.Flatten(),
    layers.Dense(64, activation='relu'),
    layers.Dense(2, activation='softmax')  # 2 classes: positive/negative
])

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

model.summary()

Why this step? This creates a Convolutional Neural Network (CNN), which is perfect for image analysis. The model will learn to recognize patterns in X-ray images and classify them as either positive or negative.

Step 4: Load and Preprocess Your Data

Now create a file called data_loader.py:

import os
import numpy as np
from tensorflow.keras.preprocessing.image import load_img, img_to_array

# Function to load images from directory
def load_images_from_directory(directory, label):
    images = []
    labels = []
    for filename in os.listdir(directory):
        if filename.endswith('.jpg') or filename.endswith('.png'):
            img_path = os.path.join(directory, filename)
            img = load_img(img_path, target_size=(224, 224))
            img_array = img_to_array(img)
            images.append(img_array)
            labels.append(label)
    return np.array(images), np.array(labels)

# Load data
positive_dir = 'data/positive'
negative_dir = 'data/negative'

positive_images, positive_labels = load_images_from_directory(positive_dir, 1)
negative_images, negative_labels = load_images_from_directory(negative_dir, 0)

# Combine data
all_images = np.vstack([positive_images, negative_images])
all_labels = np.concatenate([positive_labels, negative_labels])

# Normalize pixel values
all_images = all_images / 255.0

Why this step? Loading and preprocessing the data is crucial for training an AI model. We need to convert images to arrays and normalize the pixel values so the model can learn effectively.

Step 5: Train Your Model

Add this code to your train.py file:

import model
import data_loader
from sklearn.model_selection import train_test_split

# Split data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(
    data_loader.all_images, data_loader.all_labels, test_size=0.2, random_state=42)

# Train the model
history = model.model.fit(
    X_train, y_train,
    epochs=5,
    validation_data=(X_test, y_test),
    batch_size=32
)

# Save the model
model.model.save('xray_model.h5')

Why this step? Training the model is where it learns to recognize patterns in the X-ray images. We split our data to test how well the model performs on unseen images.

Step 6: Test Your Model with Confidence Scores

Create a file called predict.py:

import tensorflow as tf
import numpy as np
from tensorflow.keras.preprocessing.image import load_img, img_to_array

# Load the trained model
model = tf.keras.models.load_model('xray_model.h5')

# Function to predict with confidence
def predict_with_confidence(image_path):
    # Load and preprocess image
    img = load_img(image_path, target_size=(224, 224))
    img_array = img_to_array(img)
    img_array = np.expand_dims(img_array, axis=0)
    img_array = img_array / 255.0
    
    # Make prediction
    predictions = model.predict(img_array)
    confidence = np.max(predictions[0])
    predicted_class = np.argmax(predictions[0])
    
    # Return result with confidence
    if predicted_class == 1:
        result = 'Pneumonia detected'
    else:
        result = 'Healthy lung'
    
    return result, confidence

# Test with an image
result, confidence = predict_with_confidence('test_image.jpg')
print(f'Diagnosis: {result}')
print(f'Confidence: {confidence:.2f}')

Why this step? This is where we see how our model works in practice. The confidence score is crucial because it tells us how certain the model is about its prediction. As we learned from the article, a model that's confident but wrong can be dangerous in medical settings.

Step 7: Understand the Importance of Confidence

Modify your predict.py to include confidence thresholds:

# Add this to your predict.py file

# Function to make recommendations based on confidence
def make_recommendation(image_path, threshold=0.8):
    result, confidence = predict_with_confidence(image_path)
    
    if confidence > threshold:
        print(f'AI Recommendation: {result} (Confidence: {confidence:.2f})')
        print('AI is confident in this diagnosis')
    else:
        print(f'AI Recommendation: {result} (Confidence: {confidence:.2f})')
        print('AI is uncertain - recommend human review')

# Test with a threshold
make_recommendation('test_image.jpg', threshold=0.8)

Why this step? This demonstrates why it's so important for AI systems to know when they're uncertain. As shown in the article, models can be dangerously confident when they're wrong. By setting confidence thresholds, we can ensure that uncertain predictions are flagged for human review.

Summary

In this tutorial, you've learned how to build a basic AI model for analyzing chest X-rays. You've created a model that can:

Classify X-ray images as positive or negative
Provide a confidence score for each prediction
Make recommendations based on confidence levels

This simple model demonstrates the core concept from the article: AI systems must learn to recognize their own uncertainty. In real-world medical applications, a model should only make a diagnosis when it's highly confident, and flag uncertain cases for human review.

While this tutorial uses a simplified approach, it illustrates the fundamental principles of building trustworthy AI systems. As you continue learning, you'll explore more advanced techniques for improving model reliability and uncertainty quantification.