Introduction
In the rapidly evolving world of AI, OpenRouter represents a significant shift toward multi-model AI infrastructure. This tutorial will teach you how to build and deploy an AI model router that can dynamically select the best AI model for different tasks, similar to what OpenRouter enables. You'll learn to create a system that can route requests between different AI services based on performance metrics, cost, or task suitability.
Prerequisites
- Basic understanding of Python and REST APIs
- Python 3.8+ installed
- Access to OpenAI, Anthropic, and Hugging Face API keys
- Basic knowledge of Docker (optional but recommended)
- Installed packages: requests, flask, python-dotenv
Step 1: Setting Up Your Development Environment
Install Required Dependencies
First, create a virtual environment and install the necessary packages:
python -m venv ai-router-env
source ai-router-env/bin/activate # On Windows: ai-router-env\Scripts\activate
pip install flask requests python-dotenvWhy: Creating a virtual environment isolates your project dependencies and prevents conflicts with other Python projects on your system.
Step 2: Create API Key Management
Create Environment Configuration
Create a .env file in your project directory to store your API keys:
OPENAI_API_KEY=your_openai_key_here
ANTHROPIC_API_KEY=your_anthropic_key_here
HUGGINGFACE_API_KEY=your_huggingface_key_hereWhy: Storing API keys in environment variables keeps them secure and prevents accidental exposure in version control.
Step 3: Implement Model Selection Logic
Create the Router Class
Create a router.py file with the core routing logic:
import os
import requests
from dotenv import load_dotenv
import time
load_dotenv()
class AIRouter:
def __init__(self):
self.models = {
'gpt-4': {
'api_key': os.getenv('OPENAI_API_KEY'),
'url': 'https://api.openai.com/v1/chat/completions',
'cost_per_token': 0.03,
'performance_score': 9.5
},
'claude-3': {
'api_key': os.getenv('ANTHROPIC_API_KEY'),
'url': 'https://api.anthropic.com/v1/messages',
'cost_per_token': 0.015,
'performance_score': 8.7
},
'llama-2': {
'api_key': os.getenv('HUGGINGFACE_API_KEY'),
'url': 'https://api-inference.huggingface.co/models/meta-llama/Llama-2-7b-chat-hf',
'cost_per_token': 0.001,
'performance_score': 7.2
}
}
def select_best_model(self, task_type, prompt_length):
# Simple scoring algorithm
best_model = None
best_score = -1
for model_name, model_info in self.models.items():
# Adjust score based on task type and prompt length
score = model_info['performance_score']
# Give bonus for long prompts for models that handle them better
if prompt_length > 1000 and 'gpt' in model_name:
score += 0.5
elif prompt_length > 1000 and 'claude' in model_name:
score += 0.3
if score > best_score:
best_score = score
best_model = model_name
return best_model
def query_model(self, model_name, prompt):
model_info = self.models[model_name]
headers = {
'Authorization': f'Bearer {model_info["api_key"]}'
}
if 'openai' in model_name:
data = {
'model': model_name,
'messages': [{'role': 'user', 'content': prompt}]
}
elif 'anthropic' in model_name:
headers['anthropic-version'] = '2023-06-01'
data = {
'model': model_name,
'max_tokens': 1000,
'messages': [{'role': 'user', 'content': prompt}]
}
else:
# Hugging Face
data = {'inputs': prompt}
response = requests.post(model_info['url'], headers=headers, json=data)
return response.json()Why: This class implements the core logic for selecting the most appropriate AI model based on task characteristics, similar to how OpenRouter might route requests.
Step 4: Build the API Endpoint
Create Flask Application
Create a app.py file to expose your router as a web service:
from flask import Flask, request, jsonify
from router import AIRouter
app = Flask(__name__)
router = AIRouter()
@app.route('/route', methods=['POST'])
def route_request():
try:
data = request.get_json()
prompt = data.get('prompt', '')
task_type = data.get('task_type', 'general')
# Determine the best model
model_name = router.select_best_model(task_type, len(prompt))
# Query the selected model
result = router.query_model(model_name, prompt)
return jsonify({
'selected_model': model_name,
'response': result,
'prompt_length': len(prompt)
})
except Exception as e:
return jsonify({'error': str(e)}), 500
if __name__ == '__main__':
app.run(debug=True, host='0.0.0.0', port=5000)Why: This creates a RESTful API endpoint that accepts requests, routes them to the appropriate AI model, and returns the response, mimicking the functionality of OpenRouter's API.
Step 5: Test Your Router
Run and Test the Application
Start your Flask application:
python app.pyThen test it with a curl command or Postman:
curl -X POST http://localhost:5000/route \
-H "Content-Type: application/json" \
-d '{"prompt": "Explain quantum computing in simple terms", "task_type": "explanation"}'Why: Testing ensures your routing logic works correctly and that the system properly selects models based on the task requirements.
Step 6: Enhance with Performance Monitoring
Add Response Time Tracking
Update your router class to track performance:
class AIPerformanceRouter(AIRouter):
def __init__(self):
super().__init__()
self.performance_logs = {}
def query_model(self, model_name, prompt):
start_time = time.time()
try:
result = super().query_model(model_name, prompt)
end_time = time.time()
response_time = end_time - start_time
# Log performance
if model_name not in self.performance_logs:
self.performance_logs[model_name] = []
self.performance_logs[model_name].append(response_time)
return result
except Exception as e:
print(f"Error querying {model_name}: {e}")
raiseWhy: Performance monitoring allows your router to make better decisions over time by learning which models perform best under different conditions.
Step 7: Deploy Your Router
Containerize with Docker
Create a Dockerfile:
FROM python:3.9-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
EXPOSE 5000
CMD ["python", "app.py"]Create a requirements.txt:
flask==2.3.3
requests==2.31.0
python-dotenv==1.0.0Build and run with Docker:
docker build -t ai-router .
docker run -p 5000:5000 ai-routerWhy: Containerization makes your AI router portable and easily deployable across different environments, similar to how OpenRouter services are deployed at scale.
Summary
In this tutorial, you've built a functional AI model router that can dynamically select the best AI model for different tasks based on performance metrics and task requirements. This system mimics the core functionality of OpenRouter's multi-model routing capabilities, allowing you to optimize for factors like cost, performance, and task suitability. The router can be extended with more sophisticated decision-making algorithms, additional models, and real-time performance monitoring to create a production-ready AI infrastructure similar to what OpenRouter has built.
Key concepts learned include API key management, model selection algorithms, RESTful API development, and containerization - all essential skills for working with modern AI infrastructure.
