Phi-4-multimodal-instruct

Phi-4-multimodal-instruct is Microsoft's 5.6-billion-parameter multimodal model, released in February 2025 and published under the MIT License. It builds on the Phi-4-Mini-Instruct backbone and adds vision and speech encoders with modality-specific adapters, accepting text, images, and audio input and generating text, with a 128K-token context window.

Microsoft trained it on roughly 5 trillion text tokens, 1.1 trillion image-text tokens, and 2.3 million hours of speech. It was reported as the first open-weights model with speech summarization capability and topped the Hugging Face OpenASR leaderboard at release. It is aimed at developers who need multimodal understanding in a model small enough to self-host.