Tag

#mixture-of-experts

3 articles

Meet ‘North Mini Code’: Cohere’s 30B Open-Weight Mixture-of-Experts Model With 3B Active Parameters for Agentic Coding

Learn how Cohere's North Mini Code uses mixture-of-experts architecture to enable efficient, large-scale coding assistance with 30B parameters and 3B active parameters.

Jun 109

Cohere Releases Command A+: A 218B Sparse MoE Model for Agentic Workflows That Runs on as Few as Two H100 GPUs

Learn to deploy and use Cohere's Command A+ 218B parameter model for agentic workflows, optimized to run efficiently on just two H100 GPUs with W4A4 quantization.

May 2128

Researchers train AI model that hits near-full performance with just 12.5 percent of its experts

Researchers at the Allen Institute for AI and UC Berkeley have developed EMO, a mixture-of-experts model that maintains near-full performance using only 12.5% of its experts, making it more practical for memory-constrained settings.

May 1520