Step3

Step3 is StepFun's multimodal reasoning model, notable less for raw benchmark scores than for its model-system co-design aimed at minimizing decoding cost. It is a 321B-parameter mixture-of-experts vision-language model with 38B active parameters per token, built around two ideas: Multi-Matrix Factorization Attention, which shrinks KV cache and compute, and Attention-FFN Disaggregation, which splits attention and feed-forward layers into specialized inference subsystems. The paper reports decoding throughput up to 4,039 tokens per second per GPU on Hopper hardware, ahead of DeepSeek-V3 in the same setup. Context is 64K tokens, with image and text input and text output. Checkpoints are on Hugging Face in bf16 and block-fp8, with vLLM and SGLang as the recommended engines, and an OpenAI-compatible API on StepFun's platform. StepFun later released smaller siblings including Step3-VL-10B and Step 3.7 Flash. The 321B size means self-hosting needs a serious GPU cluster; most users will hit the API.

Paper-reported: 4,039 tok/s/GPU decoding throughput vs 2,324 for DeepSeek-V3 (same setup); vision-language evals in model card