5 Memorial Day deals that are worth your time (including a Costco membership discount)

Learn to build a Python web scraper that monitors Memorial Day deals from online retailers like Costco, helping you find the best discounts without manual checking.

Introduction

In this tutorial, you'll learn how to create a simple web scraper using Python to monitor Memorial Day deals from online retailers like Costco. This tool will help you track price changes and find the best deals without constantly checking websites manually. We'll build a basic scraper that can fetch deal information and save it to a file for later review.

Prerequisites

Before starting this tutorial, you'll need:

A computer with internet access
Python 3.6 or higher installed (you can download it from python.org)
A code editor like Visual Studio Code or any text editor
Basic understanding of how to open and run Python files

Step-by-Step Instructions

1. Setting Up Your Python Environment

1.1 Install Required Python Packages

First, we need to install the packages that will help us scrape web data. Open your command prompt or terminal and run:

pip install requests beautifulsoup4

This installs two essential packages: requests for making web requests and beautifulsoup4 for parsing HTML content.

1.2 Create Your Project Folder

Create a new folder on your computer called deal_scraper. Inside this folder, create a file named deal_scraper.py. This will be our main Python script.

2. Writing the Basic Scraper Code

2.1 Import Required Libraries

Open your deal_scraper.py file and start by importing the necessary libraries:

import requests
from bs4 import BeautifulSoup
import time
import csv

We import requests to fetch web pages, BeautifulSoup to parse HTML, time to add delays between requests, and csv to save our data.

2.2 Create the Main Function

Next, create a function that will handle the scraping process:

def scrape_deals():
    # This function will contain our scraping logic
    pass

The pass statement is a placeholder that tells Python to do nothing. We'll replace this with actual code soon.

3. Adding Scraping Logic

3.1 Define Target Website

For this tutorial, we'll create a simple scraper that looks for deals on a hypothetical deal website. In a real-world scenario, you'd target actual retailer sites:

def scrape_deals():
    url = 'https://example-deals-site.com/memorial-day-deals'
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36'
    }
    response = requests.get(url, headers=headers)
    
    if response.status_code == 200:
        soup = BeautifulSoup(response.content, 'html.parser')
        return soup
    else:
        print(f'Failed to retrieve page. Status code: {response.status_code}')
        return None

We set a user agent header to mimic a real browser, which helps avoid being blocked by websites. The status_code check ensures we only proceed if the website responded successfully.

3.2 Extract Deal Information

Now let's add code to find and extract deal information:

def scrape_deals():
    url = 'https://example-deals-site.com/memorial-day-deals'
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36'
    }
    response = requests.get(url, headers=headers)
    
    if response.status_code == 200:
        soup = BeautifulSoup(response.content, 'html.parser')
        deals = []
        
        # Find all deal elements (this would be specific to the target site)
        deal_elements = soup.find_all('div', class_='deal-item')
        
        for element in deal_elements:
            title = element.find('h3', class_='deal-title').text.strip()
            price = element.find('span', class_='deal-price').text.strip()
            discount = element.find('span', class_='deal-discount').text.strip()
            
            deals.append({
                'title': title,
                'price': price,
                'discount': discount
            })
        
        return deals
    else:
        print(f'Failed to retrieve page. Status code: {response.status_code}')
        return None

This code looks for specific HTML elements that contain deal information. In practice, you'd need to inspect the actual website's HTML structure to find the correct class names and tags.

4. Saving Deal Data

4.1 Create a Function to Save Data

Let's add functionality to save our scraped deals to a CSV file:

def save_deals_to_csv(deals, filename='memorial_day_deals.csv'):
    if deals:
        with open(filename, 'w', newline='', encoding='utf-8') as file:
            writer = csv.DictWriter(file, fieldnames=['title', 'price', 'discount'])
            writer.writeheader()
            writer.writerows(deals)
        print(f'Deals saved to {filename}')
    else:
        print('No deals to save')

This function creates a CSV file with columns for deal title, price, and discount percentage. The DictWriter ensures our data is properly formatted.

4.2 Complete the Main Script

Finally, let's put everything together in our main script:

import requests
from bs4 import BeautifulSoup
import time
import csv

def scrape_deals():
    url = 'https://example-deals-site.com/memorial-day-deals'
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36'
    }
    response = requests.get(url, headers=headers)
    
    if response.status_code == 200:
        soup = BeautifulSoup(response.content, 'html.parser')
        deals = []
        
        # Find all deal elements (this would be specific to the target site)
        deal_elements = soup.find_all('div', class_='deal-item')
        
        for element in deal_elements:
            title = element.find('h3', class_='deal-title').text.strip()
            price = element.find('span', class_='deal-price').text.strip()
            discount = element.find('span', class_='deal-discount').text.strip()
            
            deals.append({
                'title': title,
                'price': price,
                'discount': discount
            })
        
        return deals
    else:
        print(f'Failed to retrieve page. Status code: {response.status_code}')
        return None

def save_deals_to_csv(deals, filename='memorial_day_deals.csv'):
    if deals:
        with open(filename, 'w', newline='', encoding='utf-8') as file:
            writer = csv.DictWriter(file, fieldnames=['title', 'price', 'discount'])
            writer.writeheader()
            writer.writerows(deals)
        print(f'Deals saved to {filename}')
    else:
        print('No deals to save')

# Main execution
if __name__ == '__main__':
    print('Starting Memorial Day deal scraping...')
    deals = scrape_deals()
    save_deals_to_csv(deals)
    print('Scraping complete!')

5. Running Your Scraper

5.1 Execute the Script

Save your deal_scraper.py file and run it from the command line:

python deal_scraper.py

If everything works correctly, you should see messages about starting the scraping, saving the deals, and completing the process. A CSV file named memorial_day_deals.csv will be created in your folder.

5.2 View Your Results

Open the generated CSV file with any spreadsheet program or text editor to see the scraped deal information. You can sort and filter the data to find the best deals.

Summary

In this tutorial, you've learned how to create a basic web scraper using Python to monitor Memorial Day deals. You installed necessary packages, wrote code to fetch and parse web content, and saved the results to a file. This scraper can be extended to monitor multiple retailers or add features like email notifications when deals meet certain criteria. Remember that web scraping should always follow website terms of service and be done responsibly with appropriate delays between requests.