Mamba-Codestral-7B

Codestral Mamba 7B is Mistral's 2024 experiment in post-Transformer architectures: a code model built entirely on Mamba-2, the selective state-space architecture, with no attention layers at all. Because inference cost per token stays constant instead of growing with context, Mistral tested it on inputs up to 256,000 tokens, double what GPT-4o offered at the time, and positioned it as a fast local coding assistant rather than a heavyweight reasoning model. It shipped on la Plateforme as codestral-mamba-2407 alongside raw weights on Hugging Face.

Mistral reported it beating CodeLlama 7B, CodeGemma 7B and DeepSeek-Coder on HumanEval, and the weights are Apache 2.0, which made it one of the few permissively licensed SSM code models. It never got a successor, and Mistral's attention moved back to Transformer-based releases, so consider it a finished artifact rather than a living line. Still, with over 20,000 downloads and GGUF/MLX community builds, it remains the easiest way to run a pure state-space code model locally.

• Mistral-reported: beats CodeLlama 7B, CodeGemma 7B and DeepSeek-Coder on HumanEval (provider claim via VentureBeat, Jul 2024)
• Mistral positions it on par with SOTA Transformer-based code models (provider framing)
• No independent benchmark table verified