Meta tracks US employees' clicks and keystrokes to train AI agents

Introduction

In this tutorial, you'll learn how to build a simple data collection tool that monitors user interactions on a computer - similar to what Meta is doing with its employees. This is a beginner-friendly guide that demonstrates the core concepts of user interaction tracking using Python. We'll create a basic application that records mouse movements, clicks, and keystrokes, and stores this data in a structured format. Understanding these techniques can be valuable for learning about human-computer interaction, usability research, or AI training data collection.

Prerequisites

• Basic computer literacy
• Python 3.6 or higher installed on your system
• Familiarity with command line interface (terminal or command prompt)
• Text editor (like VS Code, Sublime Text, or Notepad)

Step-by-Step Instructions

Step 1: Set Up Your Development Environment

Install Required Python Packages

First, we need to install the necessary Python libraries for capturing user interactions. Open your terminal or command prompt and run:

pip install pynput

This package provides cross-platform support for monitoring keyboard and mouse events. It's essential for our tracking tool.

Step 2: Create the Main Tracking Script

Write the Core Monitoring Code

Create a new file called user_tracker.py and paste the following code:

import time
from pynput import mouse, keyboard
from datetime import datetime
import json

class UserTracker:
    def __init__(self):
        self.data = []
        self.start_time = None

    def on_move(self, x, y):
        # Record mouse movement
        self.data.append({
            'event': 'move',
            'position': (x, y),
            'timestamp': datetime.now().isoformat()
        })

    def on_click(self, x, y, button, pressed):
        # Record mouse clicks
        if pressed:
            self.data.append({
                'event': 'click',
                'position': (x, y),
                'button': str(button),
                'timestamp': datetime.now().isoformat()
            })

    def on_press(self, key):
        # Record key presses
        try:
            self.data.append({
                'event': 'press',
                'key': key.char,
                'timestamp': datetime.now().isoformat()
            })
        except AttributeError:
            # Special keys (ctrl, alt, etc.)
            self.data.append({
                'event': 'press',
                'key': str(key),
                'timestamp': datetime.now().isoformat()
            })

    def start_tracking(self):
        # Start monitoring
        self.start_time = datetime.now()
        print("Tracking started. Press Ctrl+C to stop.")
        
        # Create listeners
        mouse_listener = mouse.Listener(on_move=self.on_move, on_click=self.on_click)
        keyboard_listener = keyboard.Listener(on_press=self.on_press)
        
        # Start listeners
        mouse_listener.start()
        keyboard_listener.start()
        
        try:
            while True:
                time.sleep(1)
        except KeyboardInterrupt:
            # Stop tracking
            self.stop_tracking()

    def stop_tracking(self):
        # Stop monitoring and save data
        print("\nTracking stopped. Saving data...")
        
        # Save to JSON file
        filename = f"user_activity_{self.start_time.strftime('%Y%m%d_%H%M%S')}.json"
        with open(filename, 'w') as f:
            json.dump(self.data, f, indent=2)
        
        print(f"Data saved to {filename}")
        print(f"Total events recorded: {len(self.data)}")

if __name__ == "__main__":
    tracker = UserTracker()
    tracker.start_tracking()

Step 3: Run the Tracking Tool

Execute Your Script

Save the file and run it from your terminal:

python user_tracker.py

When you run the script, you'll see a message indicating that tracking has started. The program will continue running until you press Ctrl+C to stop it.

Step 4: Interact with Your Computer

Generate Tracking Data

While the script is running, interact with your computer normally:

• Move your mouse around the screen
• Click on different parts of the screen
• Type on your keyboard
• Press various keys including special keys like Ctrl, Alt, Shift

As you perform these actions, the program records each event and stores it in memory.

Step 5: Stop Tracking and View Results

Save Your Data

After you've generated some interaction data, press Ctrl+C in the terminal to stop the tracking. The program will automatically save your data to a JSON file with a timestamp in the filename.

Open the generated JSON file to see your recorded interactions. The data will look something like this:

[{
  "event": "move",
  "position": [100, 200],
  "timestamp": "2026-03-15T14:30:45.123456"
}, {
  "event": "click",
  "position": [150, 250],
  "button": "Button.left",
  "timestamp": "2026-03-15T14:30:47.654321"
}, {
  "event": "press",
  "key": "a",
  "timestamp": "2026-03-15T14:30:48.789012"
}]

Step 6: Analyze Your Data

Understand What You've Collected

The JSON file contains structured data about your computer interactions. Each entry includes:

• event: Type of interaction (move, click, press)
• position: Mouse coordinates (for move and click events)
• button: Which mouse button was pressed (for click events)
• key: The specific key pressed (for keyboard events)
• timestamp: When the event occurred

This type of data is valuable for understanding user behavior patterns and can be used to train AI systems to better predict and respond to human interactions.

Step 7: Explore Further Possibilities

Enhance Your Tracking Tool

Once you understand the basics, consider extending your tool by:

1. Adding more event types (scrolling, dragging)
2. Implementing data filtering or categorization
3. Adding real-time visualization of user activity
4. Integrating with databases for long-term storage

Remember that this is a learning exercise. When implementing similar tools in real-world applications, always consider privacy implications and user consent.

Summary

In this tutorial, you've built a basic user interaction tracking tool that records mouse movements, clicks, and keyboard presses. You learned how to use the pynput library to monitor system events, how to structure this data in JSON format, and how to save it for later analysis. This demonstrates the fundamental techniques used in systems like the one described in the Meta news article. Understanding these concepts is crucial for anyone interested in human-computer interaction, AI development, or user experience research.

The key takeaway is that modern software can capture detailed user behavior patterns, which can then be used to improve AI systems. However, it's important to always approach such technologies with awareness of privacy considerations and ethical implications.