Introduction
In this tutorial, we'll build a smart price tracking system that monitors Amazon product prices and alerts users when deals drop below their target thresholds. This system combines web scraping, data storage, and notification mechanisms to help you identify the best early Amazon Spring Sale deals, just like the ones mentioned in the ZDNet article. You'll learn how to work with Amazon's product data, store it in a database, and set up automated notifications when prices meet your criteria.
Prerequisites
- Basic Python programming knowledge
- Python 3.7+ installed
- Basic understanding of web scraping concepts
- Access to a MongoDB database (or alternative database)
- Basic understanding of APIs and HTTP requests
Step-by-step instructions
1. Setting up the Project Structure
1.1 Create Project Directory
First, create a new directory for our price tracking project and navigate into it:
mkdir amazon_price_tracker
cd amazon_price_tracker
1.2 Initialize Virtual Environment
Create and activate a virtual environment to manage dependencies:
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
1.3 Install Required Packages
Install the necessary Python packages for our project:
pip install requests beautifulsoup4 pymongo schedule python-dotenv
2. Setting Up Database Connection
2.1 Create Database Configuration
Create a .env file to store your database connection string:
MONGO_URI=mongodb://localhost:27017
DATABASE_NAME=price_tracker
COLLECTION_NAME=products
2.2 Initialize Database Connection
Create database.py to handle database operations:
import pymongo
from dotenv import load_dotenv
import os
load_dotenv()
def get_database():
# Connect to MongoDB
client = pymongo.MongoClient(os.getenv('MONGO_URI'))
db = client[os.getenv('DATABASE_NAME')]
return db
# Test connection
if __name__ == "__main__":
try:
db = get_database()
print("Database connection successful")
except Exception as e:
print(f"Database connection failed: {e}")
3. Web Scraping Implementation
3.1 Create Product Scraper Class
Create scraper.py to handle Amazon product data extraction:
import requests
from bs4 import BeautifulSoup
import time
import random
class AmazonScraper:
def __init__(self):
self.session = requests.Session()
# Set headers to mimic a real browser
self.session.headers.update({
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36',
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
'Accept-Language': 'en-US,en;q=0.5',
'Accept-Encoding': 'gzip, deflate',
'Connection': 'keep-alive',
})
def get_product_info(self, product_url):
try:
response = self.session.get(product_url)
response.raise_for_status()
soup = BeautifulSoup(response.content, 'html.parser')
# Extract product title
title = soup.find('span', {'id': 'productTitle'})
title = title.get_text().strip() if title else 'Title not found'
# Extract price
price_element = soup.find('span', {'class': 'a-price-whole'})
if price_element:
price = price_element.get_text().strip()
else:
price = 'Price not found'
return {
'title': title,
'price': price,
'url': product_url,
'timestamp': time.time()
}
except Exception as e:
print(f"Error scraping {product_url}: {e}")
return None
3.2 Add Delay and Error Handling
Amazon's servers may block requests if they come too frequently. Add random delays between requests:
def get_product_info(self, product_url):
# Add random delay to avoid being blocked
time.sleep(random.uniform(1, 3))
try:
# ... existing code ...
except requests.RequestException as e:
print(f"Request failed for {product_url}: {e}")
return None
except Exception as e:
print(f"Unexpected error for {product_url}: {e}")
return None
4. Product Tracking System
4.1 Create Product Tracker Class
Create tracker.py to manage the tracking logic:
import time
from database import get_database
from scraper import AmazonScraper
class ProductTracker:
def __init__(self):
self.db = get_database()
self.scraper = AmazonScraper()
self.collection = self.db["products"]
def add_product(self, url, target_price):
"""Add a new product to track"""
product_info = self.scraper.get_product_info(url)
if product_info:
product_data = {
'url': url,
'target_price': target_price,
'current_price': product_info['price'],
'title': product_info['title'],
'added_at': time.time(),
'last_checked': time.time()
}
# Insert into database
result = self.collection.insert_one(product_data)
print(f"Added product with ID: {result.inserted_id}")
return result.inserted_id
return None
def check_all_products(self):
"""Check all tracked products for price drops"""
products = self.collection.find()
for product in products:
print(f"Checking {product['title']}")
current_info = self.scraper.get_product_info(product['url'])
if current_info:
# Update database with new price
self.collection.update_one(
{'_id': product['_id']},
{
'$set': {
'current_price': current_info['price'],
'last_checked': time.time()
}
}
)
# Check if price dropped below target
if self.is_price_below_target(product, current_info['price']):
self.send_notification(product, current_info['price'])
def is_price_below_target(self, product, current_price):
"""Check if current price is below target price"""
# Simple price comparison (you might want to extract numeric values)
return float(current_price.replace('$', '').replace(',', '')) <= product['target_price']
def send_notification(self, product, current_price):
"""Send notification when price drops"""
print(f"ALERT: {product['title']} is now ${current_price} (below target ${product['target_price']})")
# Here you could integrate with email, SMS, or other notification services
5. Setting Up Automated Checks
5.1 Create Main Application
Create main.py to orchestrate the entire system:
import schedule
import time
from tracker import ProductTracker
# Initialize tracker
tracker = ProductTracker()
# Add some sample products (replace with actual Amazon URLs)
tracker.add_product('https://www.amazon.com/product1', 50)
tracker.add_product('https://www.amazon.com/product2', 100)
# Schedule checks every hour
schedule.every().hour.do(tracker.check_all_products)
# Run the scheduler
print("Price tracker started. Checking every hour.")
while True:
schedule.run_pending()
time.sleep(60)
5.2 Run the Application
Start your price tracker:
python main.py
6. Enhancing with Real-World Features
6.1 Add Email Notifications
Enhance your notification system by adding email alerts:
import smtplib
from email.mime.text import MIMEText
# Add to tracker.py
def send_email_notification(self, product, current_price):
# Email configuration
smtp_server = "smtp.gmail.com"
port = 587
sender_email = "[email protected]"
password = "your_app_password"
recipient_email = "[email protected]"
# Create message
message = MIMEText(f"{product['title']} is now ${current_price} (below target ${product['target_price']})\n\n{product['url']}")
message['Subject'] = 'Price Drop Alert!'
message['From'] = sender_email
message['To'] = recipient_email
# Send email
try:
server = smtplib.SMTP(smtp_server, port)
server.starttls()
server.login(sender_email, password)
server.sendmail(sender_email, recipient_email, message.as_string())
server.quit()
print("Email notification sent successfully")
except Exception as e:
print(f"Failed to send email: {e}")
6.2 Implement Price History Tracking
Track price changes over time to identify trends:
# Add to ProductTracker class
def update_price_history(self, product_id, price):
"""Store price history for trend analysis"""
self.collection.update_one(
{'_id': product_id},
{
'$push': {
'price_history': {
'price': price,
'timestamp': time.time()
}
}
}
)
Summary
In this tutorial, you've built a comprehensive Amazon price tracking system that monitors product prices and alerts you when deals drop below your target thresholds. The system uses web scraping to extract product information, stores data in a MongoDB database, and implements automated checks using the schedule library. You've also learned how to add email notifications and track price history for better deal identification. This system helps you stay ahead of Amazon's Big Spring Sale deals by automatically monitoring prices and alerting you to the best opportunities, just like the early deals mentioned in the ZDNet article.



