Mini-Omni
Open-source speech model that hears, talks, and thinks in streaming.
About
Mini-Omni (arXiv:2408.16725, August 2024) is an open-source multimodal language model that does real-time end-to-end speech input with streaming audio output — no separate ASR or TTS models required. Its headline capability is 'talking while thinking': generating text and audio responses simultaneously rather than in sequence.
The project shipped with the VoiceAssistant-400K dataset and a live Gradio demo, followed by Mini-Omni2 with vision support in October 2024. It became a reference implementation for researchers building GPT-4o-style voice interaction on open models.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 7/10
Scored 7/10 (metrics tier, metrics-v1). Evidence: Verified via vendor site/docs/GitHub and reputable coverage, wave 74; Trending score 66/100 from researcher assessment.