Phi-3.5-mini-instruct

Phi-3.5-mini-instruct is Microsoft's 3.8-billion-parameter small language model, released on August 20, 2024 as part of the Phi-3.5 series alongside the MoE and vision variants. It is a lightweight, state-of-the-art open model trained on synthetic and filtered web data with a focus on reasoning-dense content.

It supports a 128K token context length and underwent supervised fine-tuning, proximal policy optimization and direct preference optimization. Microsoft released it under the MIT license on Hugging Face, and benchmarks showed it beating Llama-3.1-8B and Mistral-7B on long-context code understanding.