Nvidia open-sources Flora to make robot video labeling 125x cheaper than GPT-6 Astra
Back to Tutorials
aiTutorialbeginner

Nvidia open-sources Flora to make robot video labeling 125x cheaper than GPT-6 Astra

October 25, 202628 views4 min read

Learn how to set up and use Nvidia's open-source Flora system for robot video labeling, reducing costs from $88 to 72 cents per hour.

Introduction

In this tutorial, you'll learn how to use the open-source Flora system developed by Nvidia for robot video labeling. Flora is a powerful data engine that can label robot videos at a fraction of the cost of competing systems like GPT-6 Astra. You'll set up a basic Flora environment, understand its core components, and run a simple labeling task to see how it works.

This tutorial is designed for beginners with no prior experience in robotics or AI data labeling. By the end, you'll have a working understanding of how Flora processes robot video data and how it can be integrated into your own projects.

Prerequisites

To follow along with this tutorial, you'll need:

  • A computer with internet access
  • Python 3.8 or higher installed
  • Basic knowledge of command-line operations
  • Some familiarity with AI concepts (no deep technical knowledge required)

Note: Flora is an open-source system, so you'll need to download and install it from Nvidia's repository.

Step-by-Step Instructions

1. Install Required Dependencies

First, we need to set up our Python environment. Open your terminal or command prompt and run the following commands:

pip install torch torchvision
pip install transformers
pip install opencv-python
pip install numpy

Why: These packages are essential for handling video data, running AI models, and processing images. PyTorch is the deep learning framework used by Flora, while OpenCV helps with video processing.

2. Clone the Flora Repository

Next, we'll download the Flora source code from Nvidia's GitHub repository:

git clone https://github.com/NVIDIA/flora.git
 cd flora

Why: This gives us access to all the Flora components, including the pre-trained models and example scripts needed to run labeling tasks.

3. Install Flora in Development Mode

With the repository cloned, we need to install Flora so we can use it in our projects:

pip install -e .

Why: The -e flag installs the package in 'editable' mode, meaning any changes you make to the source code will be reflected immediately without reinstalling.

4. Prepare a Sample Robot Video

For testing purposes, create a simple folder structure and add a sample video file:

mkdir test_videos
# Place a sample robot video file (e.g., robot_demo.mp4) in this folder

Why: Flora needs video input to process and label. This step simulates the real-world scenario where you'd have robot footage to analyze.

5. Configure Flora Parameters

Now, create a basic configuration file for Flora. Create a file named config.yaml with the following content:

model:
  name: "qwen3.5"
  checkpoint: "qwen3.5-robot-labeler"

input:
  video_path: "test_videos/robot_demo.mp4"
  frame_rate: 30

output:
  label_path: "output_labels.json"
  format: "json"

processing:
  batch_size: 8
  device: "cuda"

Why: This configuration tells Flora how to process your video, what models to use, and where to save the results. It's the key to customizing Flora for your specific needs.

6. Run the Flora Labeling Process

With everything set up, we can now run Flora on your video:

python -m flora.label --config config.yaml

Why: This command executes the labeling process using the configuration we just created. Flora will process your video, detect robot actions, and generate labeled data.

7. View the Output Labels

Once the process completes, check the output file:

cat output_labels.json

Why: The JSON file contains the labels generated by Flora. You'll see how the system identified different robot actions in your video, such as 'grasping', 'moving', or 'releasing' objects.

8. Analyze Performance

Flora's performance is measured by both cost and accuracy. To compare, you can create a simple Python script to analyze the output:

import json

with open('output_labels.json', 'r') as f:
    labels = json.load(f)

print(f"Total frames labeled: {len(labels)}")
print(f"Unique actions detected: {set([label['action'] for label in labels])}")

Why: This analysis helps you understand how well Flora performed on your video. It shows both the quantity of labeled frames and the variety of actions detected.

Summary

In this tutorial, you've learned how to set up and use Nvidia's open-source Flora system for robot video labeling. You installed the required dependencies, downloaded the Flora code, configured it for a sample video, and ran a labeling task. You now understand how Flora can reduce the cost of robot video labeling from $88 per hour to just 72 cents while maintaining high accuracy.

Flora represents a significant advancement in robotics data processing, offering a cost-effective solution for labeling robot actions. As you continue working with Flora, you can explore more advanced features like custom model training and integration with robotic control systems.

Source: The Decoder

Related Articles