DeepSeek-V2

DeepSeek-V2, released in May 2024, is a 236-billion-parameter mixture-of-experts language model with 21 billion active parameters per token. Its headline contribution was Multi-head Latent Attention (MLA), which compresses the key-value cache into a low-dimensional latent vector and reconstructs keys and values on the fly, cutting KV cache memory by about 93% versus standard multi-head attention.

Trained on 8.1 trillion tokens with a 128K context window, V2 matched or beat the earlier dense 67B model on benchmarks while serving far more tokens per second on the same hardware. DeepSeek priced its API at roughly $0.14 per million input tokens, a move widely reported as kicking off an industry price war. MLA and the DeepSeekMoE architecture carried forward into V3 and R1, making V2 the architectural foundation of the company's later frontier releases. Note the weights ship under the custom DeepSeek Model License, not an OSI-approved open license.