Tag

#inference

7 articles

NVIDIA Releases AITune: An Open-Source Inference Toolkit That Automatically Finds the Fastest Inference Backend for Any PyTorch Model

NVIDIA releases AITune, an open-source toolkit that automatically identifies the fastest inference backend for PyTorch models, streamlining deployment and enhancing performance.

Apr 104

Sigmoid vs ReLU Activation Functions: The Inference Cost of Losing Geometric Context

This article explains how sigmoid and ReLU activation functions affect geometric context preservation in neural networks, and why this matters for inference accuracy.

Apr 815

tech

AI chip startup Rebellions raises $400 million at $2.3B valuation in pre-IPO round

AI chip startup Rebellions raises $400 million at $2.3B valuation, positioning itself as a challenger to Nvidia's dominance in AI inference hardware.

Mar 3035

tech

GTC 2026: With Groq 3 LPX, Nvidia adds dedicated inference hardware to its platform for the first time

Learn how to deploy machine learning models on Nvidia's new Vera Rubin platform with dedicated Groq 3 LPX inference chips using Docker containers and ONNX export.

Mar 1742

tech

Meta unveils four generations of custom AI chips to cut inference costs for billions of users

Meta has unveiled four new generations of custom AI chips aimed at reducing inference costs and decreasing reliance on external GPU suppliers like Nvidia and AMD.

Mar 1246

DeepSeek Signals Next-Gen R2 Model, Unveils Novel Approach to Scaling Inference with SPCT

Learn about SPCT (Sparse Prompt Compression Technique), a new method developed by DeepSeek AI that improves the scalability of reward models during inference, making AI systems more efficient and cost-effective.

Feb 2775

Taalas is replacing programmable GPUs with hardwired AI chips to achieve 17,000 tokens per second for ubiquitous inference

Toronto startup Taalas is pioneering hardwired AI chips that can process 17,000 tokens per second, challenging the dominance of programmable GPUs in AI inference.

Feb 2344