Tag
5 articles
This article explains how AI startups scale their infrastructure on AWS, covering GPU computing, model optimization, and orchestration techniques.
NVIDIA introduces Dynamo Snapshot, a CRIU-based system that accelerates AI inference on Kubernetes by enabling fast startup and restoration of vLLM workers.
Learn how to deploy and manage AI models using Docker and Kubernetes, similar to practices used by companies like Anthropic. This tutorial covers containerization, orchestration, and scaling techniques essential for production AI services.
LiteLLM, a popular open-source AI proxy, has been compromised by malware that steals credentials and spreads across Kubernetes clusters. NVIDIA AI Director Jim Fan warns this marks a new class of attacks targeting AI infrastructure.
Major tech companies including IBM Cloud, Nutanix, SUSE, and OVHcloud have independently chosen Traefik as the replacement for the retired Ingress NGINX in Kubernetes environments.
