Introduction
In this tutorial, we'll explore how to set up and deploy IBM Bob, an agentic software development platform, in a self-hosted environment. This approach allows enterprises to maintain full control over their code while leveraging AI-powered development capabilities. We'll focus on deploying Bob in a private cloud environment using NVIDIA's Nemotron model for complete isolation, ensuring your code never leaves your network.
Prerequisites
- Basic understanding of containerization with Docker
- Access to a private cloud or on-premises environment with Docker installed
- NVIDIA GPU with CUDA support (for Nemotron model)
- Basic knowledge of Kubernetes or container orchestration
- Access to IBM Cloud account with appropriate permissions
- Network connectivity to download required Docker images
Step-by-Step Instructions
1. Prepare Your Environment
Before deploying Bob, ensure your environment meets the requirements. First, verify that Docker is installed and running:
docker --version
Also, confirm your NVIDIA GPU drivers are properly installed:
nvidia-smi
Why: These checks ensure your hardware can support the AI model requirements and that Docker is properly configured for containerized deployment.
2. Create Deployment Configuration
Create a configuration file that defines how Bob will be deployed. This file will specify the model to use, network settings, and resource allocation:
mkdir bob-deployment
cd bob-deployment
Then create a bob-config.yaml file:
apiVersion: v1
kind: ConfigMap
metadata:
name: bob-config
namespace: bob-system
data:
model: "nemotron-4-340b"
model-path: "/models/nemotron"
gpu-cores: "4"
memory-gb: "16"
network-isolation: "true"
Why: This configuration file centralizes all deployment settings, making it easier to manage and modify deployment parameters without changing code.
3. Set Up Required Directories
Create the necessary directory structure for Bob's deployment:
mkdir -p /opt/bob/models
mkdir -p /opt/bob/logs
mkdir -p /opt/bob/data
Why: Organizing files into dedicated directories helps maintain system cleanliness and makes it easier to manage Bob's resources and data.
4. Pull and Prepare Docker Images
Download the required Docker images for Bob's deployment:
docker pull ibm/bob-platform:latest
docker pull nvidia/cuda:12.2.0-devel-ubuntu20.04
Also, pull the Nemotron model:
docker pull ibm/nemotron-4-340b:latest
Why: Preparing Docker images ensures all necessary components are available locally, which is essential for air-gapped environments where internet access is limited.
5. Deploy Bob Using Kubernetes
Create a Kubernetes deployment manifest for Bob:
apiVersion: apps/v1
kind: Deployment
metadata:
name: bob-deployment
namespace: bob-system
spec:
replicas: 1
selector:
matchLabels:
app: bob
template:
metadata:
labels:
app: bob
spec:
containers:
- name: bob-platform
image: ibm/bob-platform:latest
ports:
- containerPort: 8080
volumeMounts:
- name: model-volume
mountPath: /models
- name: data-volume
mountPath: /data
resources:
requests:
memory: "16Gi"
cpu: "4"
limits:
memory: "16Gi"
cpu: "4"
volumes:
- name: model-volume
hostPath:
path: /opt/bob/models
- name: data-volume
hostPath:
path: /opt/bob/data
nodeSelector:
kubernetes.io/hostname: your-node-name
Why: This Kubernetes manifest defines how Bob will be deployed, including resource allocation, volume mounts, and node selection for optimal performance.
6. Configure Network Isolation
Set up network isolation by creating a dedicated namespace and network policies:
kubectl create namespace bob-system
kubectl apply -f network-policy.yaml
Where network-policy.yaml defines:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: bob-isolation
namespace: bob-system
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
ingress:
- from:
- namespaceSelector:
matchLabels:
name: bob-system
egress:
- to:
- namespaceSelector:
matchLabels:
name: kube-system
Why: Network isolation prevents unauthorized access and ensures Bob operates in a secure, isolated environment.
7. Start Bob Service
Deploy Bob using Kubernetes:
kubectl apply -f bob-config.yaml
kubectl apply -f bob-deployment.yaml
kubectl apply -f network-policy.yaml
Verify the deployment:
kubectl get pods -n bob-system
Why: This step ensures Bob is running correctly and all components are properly initialized.
8. Test Bob Integration
Create a simple test to verify Bob's functionality:
curl -X POST http://bob-service.bob-system.svc.cluster.local:8080/api/v1/health
Expected response:
{
"status": "healthy",
"model": "nemotron-4-340b",
"version": "1.0.0"
}
Why: Testing confirms Bob is functioning correctly and ready to assist with agentic software development tasks.
Summary
In this tutorial, we've successfully deployed IBM Bob in a self-hosted environment with full network isolation. We've configured Bob to use the NVIDIA Nemotron model, set up proper resource allocation, and ensured network security through Kubernetes policies. This deployment allows enterprises to leverage AI-powered development capabilities while maintaining complete control over their code and data. Bob is now ready to assist with agentic software development tasks without requiring any code to leave the organization's network.