OLMoE
Fully open MoE: 7B total, 1B active, with data and code.
About
OLMoE-1B-7B is a fully open Mixture-of-Experts language model from the Allen Institute for AI, released in September 2024. With 7 billion total parameters and only 1B active per token across 64 experts with top-8 routing, it outperforms all models with similar active parameter counts, even surpassing larger models like Llama-2-13B-Chat.
What sets OLMoE apart is its radical openness: AI2 published not just weights but the pretraining data mix, training code, intermediate checkpoints, and experiment logs. A January 2025 refresh (OLMoE-1B-7B-0125) improved on the original with 5T training tokens.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 6/10
Scored 6/10 (metrics tier, metrics-v1). Evidence: Verified via vendor site/docs/GitHub and reputable coverage, wave 66; Trending score 64/100 from researcher assessment.