Introduction
ChatGPT's Computer History feature represents a significant leap in how AI assistants understand and interact with user workflows. This tutorial will teach you how to harness this technology by creating a system that tracks your computer interactions and uses them to train AI models to understand your patterns. You'll learn to build a basic interaction tracking system that mirrors the functionality described in the news article.
Prerequisites
- Python 3.8 or higher installed
- Basic understanding of Python programming and AI concepts
- Access to a macOS system (for the actual ChatGPT app)
- Knowledge of JSON data structures and file I/O operations
- Basic understanding of machine learning concepts and libraries like scikit-learn
Step-by-Step Instructions
Step 1: Set Up Your Development Environment
Install Required Python Packages
First, we need to install the necessary Python packages for our tracking system. The system will require libraries to monitor interactions, process data, and train models.
pip install pyautogui keyboard scikit-learn pandas numpyWhy: PyAutoGUI and keyboard libraries will help us monitor user interactions, while scikit-learn and pandas provide the machine learning and data processing capabilities needed to analyze the tracked data.
Step 2: Create the Core Interaction Tracker
Initialize the Tracking System
Let's create the main tracking module that will capture user activities:
import json
import time
import datetime
import pyautogui
import keyboard
from collections import defaultdict
class InteractionTracker:
def __init__(self, log_file="user_activities.json"):
self.log_file = log_file
self.activities = defaultdict(list)
self.current_session = None
def start_session(self):
self.current_session = {
"session_id": str(time.time()),
"start_time": datetime.datetime.now().isoformat(),
"activities": []
}
print("Session started")
def log_activity(self, activity_type, details):
if not self.current_session:
self.start_session()
activity = {
"timestamp": datetime.datetime.now().isoformat(),
"type": activity_type,
"details": details
}
self.current_session["activities"].append(activity)
self.save_log()
def save_log(self):
with open(self.log_file, 'w') as f:
json.dump(self.current_session, f, indent=2)
def get_timeline(self):
return self.current_session["activities"]Why: This class creates a structured way to log user activities with timestamps and types, which is essential for building a timeline similar to ChatGPT's Computer History feature.
Step 3: Implement Keyboard and Mouse Monitoring
Create Activity Detection Functions
Next, we'll implement functions that detect and log keyboard and mouse activities:
import threading
tracker = InteractionTracker()
def monitor_keyboard():
def on_key_event(event):
if event.event_type == keyboard.KEY_DOWN:
tracker.log_activity("keyboard_input", {
"key": event.name,
"scan_code": event.scan_code
})
keyboard.hook(on_key_event)
keyboard.wait()
def monitor_mouse():
def on_mouse_event(event):
if event.event_type == "double_click":
tracker.log_activity("mouse_double_click", {
"x": event.x,
"y": event.y
})
elif event.event_type == "click":
tracker.log_activity("mouse_click", {
"x": event.x,
"y": event.y,
"button": event.button
})
pyautogui.hook(on_mouse_event)
pyautogui.wait()Why: These functions create a monitoring system that captures the exact user interactions, mimicking how ChatGPT tracks your computer history to understand workflow patterns.
Step 4: Build the Timeline Analysis System
Develop Pattern Recognition
Now we'll create a system that analyzes the collected timeline data to identify patterns:
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.cluster import KMeans
import pandas as pd
class TimelineAnalyzer:
def __init__(self, tracker):
self.tracker = tracker
self.vectorizer = TfidfVectorizer()
self.kmeans = KMeans(n_clusters=3)
def analyze_timeline(self):
timeline = self.tracker.get_timeline()
# Extract activity types and details
activity_strings = []
for activity in timeline:
activity_str = f"{activity['type']} {str(activity['details'])}"
activity_strings.append(activity_str)
# Vectorize activities
tfidf_matrix = self.vectorizer.fit_transform(activity_strings)
# Cluster similar activities
labels = self.kmeans.fit_predict(tfidf_matrix)
# Group activities by cluster
clusters = defaultdict(list)
for i, label in enumerate(labels):
clusters[label].append(timeline[i])
return dict(clusters)
def generate_suggestions(self):
clusters = self.analyze_timeline()
suggestions = []
for cluster_id, activities in clusters.items():
if len(activities) > 2: # Only suggest if cluster has multiple activities
activity_types = [a['type'] for a in activities]
suggestion = {
"cluster_id": cluster_id,
"common_activities": list(set(activity_types)),
"suggestion": f"Consider automating these {len(activities)} similar activities"
}
suggestions.append(suggestion)
return suggestionsWhy: This system uses machine learning to identify patterns in your interaction history, similar to how ChatGPT builds timelines to suggest automations and understand your workflow.
Step 5: Create the Main Execution Script
Run the Complete Tracking System
Finally, we'll create the main script that runs all components together:
import threading
import time
# Initialize components
tracker = InteractionTracker()
analyzer = TimelineAnalyzer(tracker)
# Start monitoring threads
keyboard_thread = threading.Thread(target=monitor_keyboard)
mouse_thread = threading.Thread(target=monitor_mouse)
# Start monitoring
keyboard_thread.start()
mouse_thread.start()
print("Tracking started. Press Ctrl+C to stop.")
try:
while True:
# Periodically analyze timeline
time.sleep(60) # Analyze every minute
suggestions = analyzer.generate_suggestions()
print("\nAI Suggestions:")
for suggestion in suggestions:
print(f"- {suggestion['suggestion']}")
except KeyboardInterrupt:
print("\nStopping tracker...")
tracker.save_log()
print("Data saved successfully")Why: This script ties everything together, running the monitoring in background threads while periodically analyzing the timeline to generate suggestions, just like ChatGPT's Computer History feature.
Step 6: Test and Extend Your System
Validate Your Implementation
Run your system and observe how it tracks your interactions:
python main_tracker.pyTest by typing, clicking, and performing various activities. The system should log these actions and periodically suggest automations based on patterns it detects.
Why: Testing validates that your system properly captures and analyzes user interactions, demonstrating the core functionality of the Computer History feature.
Summary
This tutorial demonstrated how to create a system that tracks computer interactions similar to ChatGPT's Computer History feature. You've learned to implement keyboard and mouse monitoring, create structured timelines of user activities, and apply machine learning techniques to identify patterns and suggest automations. The system captures your workflow patterns and can generate AI-driven suggestions for task automation, mirroring the functionality described in the news article.
While this is a simplified implementation, it demonstrates the fundamental concepts behind how AI assistants like ChatGPT can understand user workflows and provide intelligent automation suggestions based on interaction history.
