Zephyr 7B

Zephyr 7B is a chat model from Hugging Face's H4 team, released October 25, 2023. It starts from Mistral-7B and aligns it entirely through distillation: supervised fine-tuning on the UltraChat dataset, then distilled direct preference optimization (dDPO) on AI-generated feedback from the UltraFeedback dataset — no human annotation required.

The result was striking: a 7B model scoring 7.34 on MT-Bench, competitive with 70B-parameter chat models aligned with expensive human feedback, and trainable in a matter of hours on 16 A100 GPUs. The accompanying technical report and the open alignment-handbook repository made it a template for cheap, reproducible alignment.

The model card warns honestly that removing the source datasets' built-in alignment boosted helpfulness at the cost of safety — it can generate problematic text when prompted, and was intended for research. The weights are published under a permissive MIT license.