Step-Audio

Step-Audio is StepFun's unified speech-text multimodal model: a 130B-parameter system that ingests raw speech tokens, reasons in a joint audio-text space, and emits speech directly, keeping prosody, emotion, accent, and even laughter intact instead of flattening everything through a text transcript. The Step-Audio-Chat version is open-sourced with inference code, and the team also distilled a lightweight Step-Audio-TTS-3B that claims state-of-the-art character error rates on the SEED TTS benchmark and can generate rap and humming. It supports tool calling and role-play for agentic voice applications. The full 130B model demands serious hardware, and as with any large open voice model, cloning capabilities need consent safeguards in production.

Team-reported: state-of-the-art on the StepEval-Audio-360 human evaluation benchmark; Step-Audio-TTS-3B reports SOTA CER on SEED (paper arXiv 2502.11946)