Tag
17 articles
This article explains Baidu's Unlimited OCR 2, an advanced OCR model that can process hundreds of pages in a single request using a novel SR-SWA attention mechanism. It addresses computational bottlenecks in long-document processing and demonstrates innovations in attention-based architectures.
Learn to implement a token pruning technique similar to Stanford and UC Berkeley's TokenGate that can accelerate long-context language model prefill operations by up to 11.3x while maintaining accuracy.
Learn how VC-Attention, a new AI technique from Nunchux AI, speeds up video generation by solving key performance bottlenecks in Diffusion Transformers.
Learn to use NVIDIA's cuDNN Frontend Graph API to optimize scaled dot-product attention computations, enabling kernel fusion, autotuning, and plan reuse for enhanced deep learning performance.
Learn to implement a Recurrent Looped Transformer (RLT) architecture that maintains decoder state across tokens, enabling unbounded temporal depth for long sequence modeling.
Learn how Kimi K3, a new AI model from Moonshot AI, uses Mixture of Experts and advanced attention methods to become more efficient and powerful than previous models.
Learn how Baidu's Unlimited OCR achieves efficient processing of dozens of document pages in a single pass by mimicking human memory and forgetting mechanisms.
A Miami startup claims to have cracked a key mathematical bottleneck in AI, making large language models faster and more energy-efficient. Independent tests back up their bold claims.
Learn to implement a simplified version of MiniMax's Sparse Attention mechanism that reduces computational complexity in attention operations while maintaining performance.
Learn how Parallax, a new AI technique, improves language understanding by combining traditional attention methods with smart error corrections.
Learn to implement and experiment with fundamental AI concepts including neural networks, transformers, and attention mechanisms through hands-on coding exercises.
NVIDIA's Gated DeltaNet-2 decouples erase and write operations in linear attention, outperforming models like Mamba-2 and KDA in long-context tasks.