Phi-3.5

Phi-3.5 is Microsoft's August 2024 small language model series, building on the Phi-3 collection with three open-source models: Phi-3.5-mini (3.82B), Phi-3.5-vision (4.15B, handling image and video analysis), and Phi-3.5-MoE, a 41.9B-parameter mixture-of-experts model that activates only ~6.6B parameters per token. All three support a 128K-token context length. Microsoft trained the MoE on 4.9 trillion tokens (10% multilingual) with synthetic textbook-style data, and reported it beating GPT-4o mini on some reasoning benchmarks. The vision model was reported to beat GPT-4o on MMMU, MMBench, and TextVQA in Microsoft's testing.\n\nThe honest picture: Microsoft's benchmark claims are self-reported and sit in the dense mid-2024 era — Phi-3.5-mini famously beat Llama-3.1-8B on RepoQA, but the Phi line has since been superseded by Microsoft's MAI family announced in mid-2026. Still, Phi-3.5-MoE remains an interesting efficient MoE design, and all three are permissively open for local deployment.