Tag

#GGUF

2 articles

A Coding Tutorial for Running PrismML Bonsai 1-Bit LLM on CUDA with GGUF, Benchmarking, Chat, JSON, and RAG

Learn how to run a tiny but powerful AI model called Bonsai 1-bit LLM on your computer using CUDA and GGUF technology.

Apr 1838

tools

A Coding Implementation to Run Qwen3.5 Reasoning Models Distilled with Claude-Style Thinking Using GGUF and 4-Bit Quantization

A new tutorial shows how to run Qwen3.5 reasoning models with Claude-style thinking using GGUF and 4-bit quantization, enabling flexible deployment across different hardware setups.

Mar 2672