Tag

#local inference

4 articles

Deploying a 1-Bit Bonsai-27B Model with PrismML llama.cpp and OpenAI-Compatible Local Inference Workflows

Developers have successfully deployed the 1-bit Bonsai-27B language model using PrismML’s fork of llama.cpp, enabling efficient local inference with OpenAI-compatible workflows.

Jul 2759

Best Local LLMs You Can Run on a Single 24GB GPU in 2026: Qwen, Gemma, Mistral, DeepSeek Compared

This article explains how large language models can be optimized to run locally on 24GB GPUs using quantization and architectural efficiency. It explores the technical strategies behind models like Qwen3.6, Mistral Small, and DeepSeek-R1-Distill.

Jul 1960

Google quietly launched an AI dictation app that works offline

Learn to build an offline speech-to-text application using Google's Gemma AI models with real-time audio capture and local inference capabilities.

Apr 6161

tech

Qualcomm shrinks AI reasoning chains by 2.4x to fit thinking models on smartphones

Learn how Qualcomm is shrinking AI reasoning models to fit smartphones, making them faster, more private, and more reliable for everyday use.

Mar 20125