#sparse activation

Thinking Machines Lab Releases Inkling-Small: A 276B Total, 12B Active Open Weights Multimodal MoE Model

This article explains the advanced concepts behind Inkling-Small, a 276 billion parameter multimodal MoE model that achieves efficiency through sparse activation and open weights.