Introduction
In this tutorial, you'll learn how to build a basic AI model routing system similar to what Coinbase uses to choose between different AI models based on cost and performance. This system will help you understand how companies like Coinbase optimize their AI spending by automatically selecting the most cost-effective model for each task.
By the end of this tutorial, you'll have created a simple Python application that can:
- Compare different AI models based on pricing and performance
- Automatically route requests to the most cost-effective model
- Track usage and costs
This tutorial is perfect for beginners who want to understand how AI model selection works in real-world applications.
Prerequisites
Before starting this tutorial, you'll need:
- A computer with Python 3.7 or higher installed
- Basic understanding of Python programming concepts
- Access to an AI model API (we'll use free or open-source alternatives for demonstration)
Note: For this tutorial, we'll simulate AI model responses rather than making actual API calls to avoid any real costs.
Step-by-Step Instructions
Step 1: Set Up Your Python Environment
First, create a new Python file called ai_router.py. We'll start by importing the necessary libraries and creating a basic structure.
import random
import time
from typing import Dict, List, Tuple
class AIModel:
def __init__(self, name: str, cost_per_token: float, performance_score: int):
self.name = name
self.cost_per_token = cost_per_token
self.performance_score = performance_score
def __str__(self):
return f"{self.name} (Cost: ${self.cost_per_token:.6f}/token, Performance: {self.performance_score}/10)"
Why we do this: This creates a basic class to represent each AI model with its key properties: cost, performance, and name. This structure will allow us to easily compare models later.
Step 2: Create a Model Database
Next, we'll create a list of AI models that our system will choose from. This represents what Coinbase might use in their system.
# Define our AI models
models = [
AIModel("GLM 5.2", 0.0001, 8), # Chinese model
AIModel("Kimi 2.7", 0.00015, 9), # Chinese model
AIModel("GPT-4", 0.0003, 10), # Western model
AIModel("Claude 3", 0.00025, 9), # Western model
AIModel("Gemini Pro", 0.0002, 8) # Western model
]
print("Available AI Models:")
for model in models:
print(f" {model}")
Why we do this: This creates a database of models with different cost and performance characteristics, similar to what Coinbase would have access to. The models are ordered to show different pricing and performance levels.
Step 3: Create a Task Simulation System
Now we'll create a way to simulate different tasks that require different amounts of tokens and different performance levels.
def simulate_task(task_type: str) -> Tuple[int, int]:
"""Simulate a task with random token count and performance requirement"
# Task types: simple, complex, very_complex
if task_type == "simple":
tokens = random.randint(100, 500)
performance_required = random.randint(1, 3)
elif task_type == "complex":
tokens = random.randint(500, 2000)
performance_required = random.randint(4, 7)
else: # very_complex
tokens = random.randint(2000, 5000)
performance_required = random.randint(8, 10)
return tokens, performance_required
# Test our task simulation
print("\nSimulated Tasks:")
for i in range(3):
tokens, performance = simulate_task("complex")
print(f"Task {i+1}: {tokens} tokens, performance requirement: {performance}/10")
Why we do this: This simulates the real-world scenario where different tasks require different amounts of processing and different levels of performance. This helps us understand how to route different types of requests to different models.
Step 4: Implement the Routing Logic
Now we'll create the core logic that selects the best model for each task based on cost and performance requirements.
def route_task(task_tokens: int, performance_required: int) -> AIModel:
"""Route a task to the most cost-effective model that meets performance requirements"""
# Filter models that meet the performance requirement
suitable_models = [model for model in models if model.performance_score >= performance_required]
if not suitable_models:
# If no model meets requirements, return the best available
return max(models, key=lambda m: m.performance_score)
# Calculate cost per token for each suitable model
# We'll use a simple formula: cost = (cost_per_token * tokens) + (performance_penalty)
def calculate_cost(model: AIModel) -> float:
# More performance = more cost, but we're keeping it simple
base_cost = model.cost_per_token * task_tokens
performance_penalty = (10 - model.performance_score) * 0.0001
return base_cost + performance_penalty
# Find the model with the lowest calculated cost
best_model = min(suitable_models, key=calculate_cost)
return best_model
# Test our routing logic
print("\nRouting Results:")
for i in range(3):
tokens, performance = simulate_task("complex")
selected_model = route_task(tokens, performance)
print(f"Task {i+1}: {tokens} tokens, performance required: {performance}/10")
print(f" Selected model: {selected_model}")
Why we do this: This is the core of Coinbase's system. It filters models based on performance requirements and then selects the one that offers the best value (lowest cost for the required performance).
Step 5: Add Cost Tracking and Reporting
Let's add functionality to track costs and provide reports similar to what Coinbase would need.
class CostTracker:
def __init__(self):
self.total_cost = 0
self.model_usage = {}
def add_cost(self, model_name: str, cost: float):
self.total_cost += cost
if model_name not in self.model_usage:
self.model_usage[model_name] = 0
self.model_usage[model_name] += cost
def get_report(self):
print(f"\n--- Cost Report ---")
print(f"Total Cost: ${self.total_cost:.6f}")
print("Model Usage:")
for model, cost in self.model_usage.items():
print(f" {model}: ${cost:.6f}")
# Initialize cost tracker
tracker = CostTracker()
# Simulate a series of tasks
print("\n--- Simulating AI Tasks ---")
for i in range(5):
tokens, performance = simulate_task(random.choice(["simple", "complex", "very_complex"]))
selected_model = route_task(tokens, performance)
cost = selected_model.cost_per_token * tokens
tracker.add_cost(selected_model.name, cost)
print(f"Task {i+1}: {tokens} tokens, performance required: {performance}/10")
print(f" Selected: {selected_model.name} - Cost: ${cost:.6f}")
# Show final report
tracker.get_report()
Why we do this: This simulates how Coinbase would track their AI spending and understand which models are being used most frequently. This is essential for cost optimization and budget planning.
Step 6: Run the Complete System
Let's run the complete system to see how it works end-to-end.
def main():
print("=== AI Model Routing System ===")
print("This simulates how Coinbase might route AI tasks to different models based on cost and performance.")
# Show available models
print("\nAvailable AI Models:")
for model in models:
print(f" {model}")
# Run simulation
tracker = CostTracker()
print("\n--- Running Simulation ---")
for i in range(10):
task_type = random.choice(["simple", "complex", "very_complex"])
tokens, performance = simulate_task(task_type)
selected_model = route_task(tokens, performance)
cost = selected_model.cost_per_token * tokens
tracker.add_cost(selected_model.name, cost)
print(f"Task {i+1:2d}: {tokens:4d} tokens, performance: {performance}/10 -> {selected_model.name}")
tracker.get_report()
if __name__ == "__main__":
main()
Why we do this: This puts everything together into a complete, runnable system that demonstrates how the routing logic works in practice.
Summary
In this tutorial, you've built a simplified version of Coinbase's AI model routing system. You learned how to:
- Create a database of AI models with different costs and performance levels
- Simulate different types of AI tasks with varying requirements
- Implement routing logic that selects the most cost-effective model for each task
- Track costs and usage across different models
This system demonstrates how companies like Coinbase can reduce their AI spending by automatically choosing the best model for each specific task. The key insight is that not all AI models are equal in cost or performance, and smart routing can save significant money while maintaining quality.
Remember, this is a simplified simulation. Real-world implementations would include actual API calls, more sophisticated performance metrics, and additional features like caching and load balancing.
