IBM Brings Bob to Self-Hosted and Air-Gapped Environments: Agentic Software Development Without Moving Your Code
Back to Tutorials
aiTutorialintermediate

IBM Brings Bob to Self-Hosted and Air-Gapped Environments: Agentic Software Development Without Moving Your Code

October 3, 202629 views4 min read

Learn to deploy IBM Bob, an agentic software development platform, in a self-hosted environment with full network isolation using NVIDIA Nemotron models.

Introduction

In this tutorial, we'll explore how to set up and deploy IBM Bob, an agentic software development platform, in a self-hosted environment. This approach allows enterprises to maintain full control over their code while leveraging AI-powered development capabilities. We'll focus on deploying Bob in a private cloud environment using NVIDIA's Nemotron model for complete isolation, ensuring your code never leaves your network.

Prerequisites

  • Basic understanding of containerization with Docker
  • Access to a private cloud or on-premises environment with Docker installed
  • NVIDIA GPU with CUDA support (for Nemotron model)
  • Basic knowledge of Kubernetes or container orchestration
  • Access to IBM Cloud account with appropriate permissions
  • Network connectivity to download required Docker images

Step-by-Step Instructions

1. Prepare Your Environment

Before deploying Bob, ensure your environment meets the requirements. First, verify that Docker is installed and running:

docker --version

Also, confirm your NVIDIA GPU drivers are properly installed:

nvidia-smi

Why: These checks ensure your hardware can support the AI model requirements and that Docker is properly configured for containerized deployment.

2. Create Deployment Configuration

Create a configuration file that defines how Bob will be deployed. This file will specify the model to use, network settings, and resource allocation:

mkdir bob-deployment
 cd bob-deployment

Then create a bob-config.yaml file:

apiVersion: v1
kind: ConfigMap
metadata:
  name: bob-config
  namespace: bob-system
  data:
    model: "nemotron-4-340b"
    model-path: "/models/nemotron"
    gpu-cores: "4"
    memory-gb: "16"
    network-isolation: "true"

Why: This configuration file centralizes all deployment settings, making it easier to manage and modify deployment parameters without changing code.

3. Set Up Required Directories

Create the necessary directory structure for Bob's deployment:

mkdir -p /opt/bob/models
mkdir -p /opt/bob/logs
mkdir -p /opt/bob/data

Why: Organizing files into dedicated directories helps maintain system cleanliness and makes it easier to manage Bob's resources and data.

4. Pull and Prepare Docker Images

Download the required Docker images for Bob's deployment:

docker pull ibm/bob-platform:latest
docker pull nvidia/cuda:12.2.0-devel-ubuntu20.04

Also, pull the Nemotron model:

docker pull ibm/nemotron-4-340b:latest

Why: Preparing Docker images ensures all necessary components are available locally, which is essential for air-gapped environments where internet access is limited.

5. Deploy Bob Using Kubernetes

Create a Kubernetes deployment manifest for Bob:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: bob-deployment
  namespace: bob-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: bob
  template:
    metadata:
      labels:
        app: bob
    spec:
      containers:
      - name: bob-platform
        image: ibm/bob-platform:latest
        ports:
        - containerPort: 8080
        volumeMounts:
        - name: model-volume
          mountPath: /models
        - name: data-volume
          mountPath: /data
        resources:
          requests:
            memory: "16Gi"
            cpu: "4"
          limits:
            memory: "16Gi"
            cpu: "4"
      volumes:
      - name: model-volume
        hostPath:
          path: /opt/bob/models
      - name: data-volume
        hostPath:
          path: /opt/bob/data
      nodeSelector:
        kubernetes.io/hostname: your-node-name

Why: This Kubernetes manifest defines how Bob will be deployed, including resource allocation, volume mounts, and node selection for optimal performance.

6. Configure Network Isolation

Set up network isolation by creating a dedicated namespace and network policies:

kubectl create namespace bob-system
kubectl apply -f network-policy.yaml

Where network-policy.yaml defines:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: bob-isolation
  namespace: bob-system
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          name: bob-system
  egress:
  - to:
    - namespaceSelector:
        matchLabels:
          name: kube-system

Why: Network isolation prevents unauthorized access and ensures Bob operates in a secure, isolated environment.

7. Start Bob Service

Deploy Bob using Kubernetes:

kubectl apply -f bob-config.yaml
kubectl apply -f bob-deployment.yaml
kubectl apply -f network-policy.yaml

Verify the deployment:

kubectl get pods -n bob-system

Why: This step ensures Bob is running correctly and all components are properly initialized.

8. Test Bob Integration

Create a simple test to verify Bob's functionality:

curl -X POST http://bob-service.bob-system.svc.cluster.local:8080/api/v1/health

Expected response:

{
  "status": "healthy",
  "model": "nemotron-4-340b",
  "version": "1.0.0"
}

Why: Testing confirms Bob is functioning correctly and ready to assist with agentic software development tasks.

Summary

In this tutorial, we've successfully deployed IBM Bob in a self-hosted environment with full network isolation. We've configured Bob to use the NVIDIA Nemotron model, set up proper resource allocation, and ensured network security through Kubernetes policies. This deployment allows enterprises to leverage AI-powered development capabilities while maintaining complete control over their code and data. Bob is now ready to assist with agentic software development tasks without requiring any code to leave the organization's network.

Source: MarkTechPost

Related Articles