Snowflake Arctic

Snowflake Arctic arrived April 24, 2024 as an enterprise-first open model: a dense-MoE hybrid transformer with 480 billion total parameters and only 17 billion active per token, using top-2 gating across 128 fine-grained experts. It is tuned for the unglamorous work enterprises actually pay for — SQL generation, code, and instruction following — and Snowflake reported training it on a roughly $2M budget. The release is genuinely open under Apache 2.0 with ungated weights, plus a series of cookbooks on training MoE models. The tradeoffs are real: a 4,096-token context window that feels short next to modern models, English-centric training, and inference that needs trust_remote_code plus DeepSpeed. It has since been eclipsed on raw benchmarks but remains a clean example of efficient MoE design.

Team-reported: competitive with Llama 3 70B, Mixtral 8x22B, and DBRX on enterprise tasks (team-reported)