Meet Shepherd: An Open-Source Python Substrate That Lets Meta-Agents Fork, Replay, and Revert Any Agent Run

Introduction

In this tutorial, you'll learn how to use Shepherd, an open-source Python substrate that allows you to fork, replay, and revert agent runs. This technology is especially useful for AI agents that interact with code, files, and environments — think of it like a time machine for AI agent execution. Whether you're debugging an agent or testing different approaches, Shepherd lets you quickly undo or replay actions without starting from scratch.

Prerequisites

• Basic understanding of Python programming
• Python 3.8 or higher installed
• Git installed on your system
• Access to a terminal or command line

Step-by-Step Instructions

1. Install Shepherd

First, you need to install the Shepherd package. Open your terminal and run the following command:

pip install shepherd-agent

This installs the core package needed to run Shepherd. It will also install necessary dependencies like git and pydantic for managing agent states.

2. Create a Simple Agent Script

Next, create a basic Python script that simulates an agent performing a task. We'll create a script that modifies a file and then reverts the change using Shepherd.

Save the following code in a file named simple_agent.py:

import os
import shepherd

# Initialize Shepherd to track the agent's actions
shepherd.init()

# Create a test file
with open('test_file.txt', 'w') as f:
    f.write('Hello, world!\nThis is a test file.\n')

print('File created successfully.')

# Modify the file
with open('test_file.txt', 'a') as f:
    f.write('This line was added by the agent.\n')

print('File modified successfully.')

# Commit the state to Shepherd
shepherd.commit('Modified file')

# Revert to the previous state
shepherd.revert()

print('Reverted to previous state.')

# Read the file to confirm it was reverted
with open('test_file.txt', 'r') as f:
    content = f.read()
    print('Current file content:\n', content)

This script creates a file, modifies it, commits the changes, and then reverts back to the original state. The shepherd.commit() and shepherd.revert() functions are key to managing agent states.

3. Run the Agent Script with Shepherd

Now, run the script using the Shepherd runtime:

python -m shepherd simple_agent.py

This command tells Python to run your script using the Shepherd runtime, which records all changes to the file system and agent state. You'll see output like:

File created successfully.
File modified successfully.
Commit: Modified file
Reverted to previous state.
Current file content:
 Hello, world!
This is a test file.

The shepherd.commit() function creates a snapshot of the current state, and shepherd.revert() restores the previous state — all without manually managing files or environment changes.

4. Explore the Shepherd Trace

Shepherd stores all agent actions in a trace that you can explore. The trace is stored in a hidden folder called .shepherd in your project directory.

To see what Shepherd has recorded, run:

ls -la .shepherd

You'll see something like:

drwxr-xr-x  3 user  staff  96 Aug  8 10:00 .
drwxr-xr-x  5 user  staff  160 Aug  8 10:00 ..
drwxr-xr-x  3 user  staff  96 Aug  8 10:00 commits

Inside the commits directory, you'll find snapshots of your agent's state at different points in time. This is how Shepherd enables fast forks and replays.

5. Replay an Agent Run

Shepherd also allows you to replay an agent run from a specific commit. Let's simulate a replay by modifying the script to replay the state:

import os
import shepherd

# Initialize Shepherd
shepherd.init()

# Create a test file
with open('test_file.txt', 'w') as f:
    f.write('Hello, world!\nThis is a test file.\n')

print('File created successfully.')

# Modify the file
with open('test_file.txt', 'a') as f:
    f.write('This line was added by the agent.\n')

print('File modified successfully.')

# Commit the state
shepherd.commit('Modified file')

# Replay the commit
shepherd.replay('Modified file')

print('Replayed commit successfully.')

# Read the file to confirm
with open('test_file.txt', 'r') as f:
    content = f.read()
    print('Current file content:\n', content)

Run this script with:

python -m shepherd replay_agent.py

This will replay the commit and show how Shepherd can restore the state of the agent and file system exactly as it was at that time.

6. Fork a New Agent Run

One of the most powerful features of Shepherd is the ability to fork a new agent run from a previous commit. This allows you to test new ideas without affecting the original run.

Create a new file fork_agent.py:

import os
import shepherd

# Initialize Shepherd
shepherd.init()

# Fork from a specific commit
shepherd.fork('Modified file')

# Modify the file again
with open('test_file.txt', 'a') as f:
    f.write('This line was added in the forked run.\n')

print('Forked run modified successfully.')

# Commit the new state
shepherd.commit('Forked and modified')

# Read the file to confirm
with open('test_file.txt', 'r') as f:
    content = f.read()
    print('Current file content:\n', content)

Run this script with:

python -m shepherd fork_agent.py

This will fork the agent from a previous commit and allow you to modify the state without affecting the original run.

Summary

In this tutorial, you've learned how to use Shepherd, an open-source Python substrate that allows you to fork, replay, and revert agent runs. You've seen how to install Shepherd, create and run simple agent scripts, and manage agent states using commit, revert, replay, and fork functions. This technology is especially useful for AI agents that work with files and environments, as it provides a fast and reliable way to undo or replay actions without losing progress or starting over.