MPT-7B

MPT-7B, released by MosaicML on May 5, 2023, was a watershed for open AI: a 7B transformer trained from scratch on ~1T tokens of text and code that matched Meta's LLaMA-7B quality — and unlike LLaMA, shipped under Apache 2.0, fully cleared for commercial use. Training took 9.5 days on the MosaicML platform at a disclosed cost of roughly $200,000, with the complete training recipe open-sourced via llm-foundry.

MosaicML released three fine-tuned siblings alongside the base model: MPT-7B-Instruct, MPT-7B-Chat, and MPT-7B-StoryWriter-65k+ — the last demonstrating 65K-token context via ALiBi extrapolation (80K demonstrated on a single A100), a striking long-context proof point for mid-2023.

License nuance: the base model is Apache 2.0, but the fine-tuned variants carry different terms (Instruct CC-BY-SA, Chat CC-BY-NC-SA) — verify per checkpoint. Legacy now, but historically pivotal in the commercial-open-model movement.