OpenELM-3B-Instruct

OpenELM-3B-Instruct is the largest instruction-tuned model in Apple's OpenELM family, released on April 24, 2024. OpenELM uses a layer-wise scaling strategy to allocate parameters efficiently within each transformer layer, and the 3B model was pre-trained on roughly 1.8 trillion tokens from RefinedWeb, deduplicated PILE, RedPajama, and Dolma.

Apple released eight OpenELM models in total, four pre-trained and four instruction-tuned from 270M to 3B, framing the release as a reproducibility and transparency contribution to open research. Weights are available under Apple's Sample Code License, which carries restrictions beyond typical open source licenses.