Introduction
In this beginner-friendly tutorial, you'll learn how to work with the NVIDIA Open-SWE-Traces dataset to prepare data for training AI models in software engineering tasks. This dataset contains real-world trajectories of how agents (AI systems) solve coding problems. We'll focus on parsing these trajectories, analyzing the data, and preparing a subset suitable for supervised fine-tuning.
By the end of this tutorial, you'll have a clear understanding of how to load and process this data using Python and popular libraries like Hugging Face and Pandas. You'll also learn how to filter and curate this data based on specific criteria like success rate, code patch size, and programming language.
Prerequisites
Before starting this tutorial, ensure you have:
- A basic understanding of Python programming
- Access to Google Colab (or a local Python environment with internet access)
- Basic knowledge of data analysis using Pandas
No prior experience with AI or machine learning is required. We'll walk you through everything step by step.
Step-by-Step Instructions
1. Setting Up Your Environment
First, we'll install the necessary libraries. In Google Colab, run the following code:
!pip install datasets pandas tqdm
Why? The datasets library from Hugging Face allows us to easily load and stream datasets. pandas is essential for data manipulation, and tqdm gives us progress bars for long-running operations.
2. Loading the Dataset
Next, we'll load the NVIDIA Open-SWE-Traces dataset directly from Hugging Face:
from datasets import load_dataset
dataset = load_dataset("nvidia/Open-SWE-traces")
Why? Loading directly from Hugging Face avoids downloading the entire dataset to your local machine, saving time and storage space. This is especially useful in environments like Colab.
3. Exploring the Dataset Structure
Let's take a look at what the dataset contains:
print(dataset)
print(dataset["train"][0])
Why? Understanding the dataset structure helps us know what data we're working with and how to access specific fields. Each entry in the dataset represents a trajectory of an AI agent solving a coding problem.
4. Normalizing Multi-Turn Conversations
Agent conversations often involve multiple turns. We'll normalize these to make them easier to work with:
import pandas as pd
# Create a simple function to normalize conversations
def normalize_conversation(entry):
# Extract the conversation history
conversation = entry["conversation"]
# Normalize by flattening the conversation turns
normalized = []
for turn in conversation:
normalized.append(turn["content"])
return "\n".join(normalized)
# Apply to a sample
sample_entry = dataset["train"][0]
normalized_conv = normalize_conversation(sample_entry)
print(normalized_conv)
Why? Normalizing conversations ensures we have consistent text for analysis, making it easier to build features or train models later.
5. Parsing Final Code Patches
Each trajectory includes a final code patch. Let's extract this:
def extract_patch(entry):
# The patch is usually in the last turn
conversation = entry["conversation"]
if conversation:
last_turn = conversation[-1]
# Extract code from the patch
if "patch" in last_turn:
return last_turn["patch"]
return "No patch found"
# Apply to sample
sample_patch = extract_patch(sample_entry)
print(sample_patch)
Why? Code patches represent the actual solution that the agent produced. Analyzing these helps us understand the quality and type of solutions generated.
6. Building an Analysis DataFrame
We'll now create a DataFrame to store key metrics about each trajectory:
import pandas as pd
# Initialize an empty list to store data
data_list = []
# Process a small subset for demonstration
for i in range(10): # Process first 10 entries
entry = dataset["train"][i]
# Extract features
trajectory_length = len(entry["conversation"])
patch_size = len(extract_patch(entry))
# Add to list
data_list.append({
"trajectory_length": trajectory_length,
"patch_size": patch_size,
"success": entry.get("success", False),
"language": entry.get("language", "unknown")
})
# Create DataFrame
df = pd.DataFrame(data_list)
print(df.head())
Why? A DataFrame allows us to analyze and visualize the data easily. It also makes it simple to filter and select subsets for fine-tuning.
7. Filtering for Supervised Fine-Tuning
Now, we'll filter our dataset to include only successful trajectories with reasonable patch sizes:
# Filter successful trajectories with patch size > 0
filtered_df = df[(df["success"] == True) & (df["patch_size"] > 0)]
# Further filter by language
filtered_df = filtered_df[filtered_df["language"] == "python"]
print("Filtered dataset shape:", filtered_df.shape)
print(filtered_df.head())
Why? Filtering ensures that we only include high-quality data for training. Successful trajectories are more likely to contain correct solutions, and filtering by language helps us focus on specific domains.
8. Saving the Processed Dataset
Finally, save your processed dataset for future use:
# Save to CSV
filtered_df.to_csv("processed_swe_traces.csv", index=False)
print("Dataset saved successfully!")
Why? Saving the dataset allows you to reuse the processed data without re-running the entire pipeline. This is especially useful for large datasets or complex preprocessing steps.
Summary
In this tutorial, we walked through how to load, process, and filter the NVIDIA Open-SWE-Traces dataset for supervised fine-tuning. We learned how to:
- Stream data from Hugging Face
- Normalize multi-turn conversations
- Extract and analyze code patches
- Build and filter a DataFrame for training
- Save processed data for future use
This foundational knowledge will help you prepare datasets for training AI systems in software engineering tasks. You can extend this approach to include more sophisticated features like token budgets or tool-use metrics as discussed in the original article.
