RecurrentGemma-9B

RecurrentGemma-9B is Google DeepMind's open model built on the Griffin architecture, a hybrid that replaces most transformer self-attention layers with gated linear recurrences while keeping a few local sliding-window attention layers. The result is a 9B model whose inference memory stays constant no matter how long the generated sequence gets.

It ships under the Gemma Terms of Use rather than a standard open-source license, and Google positioned it as an architecture experiment: it scores somewhat below same-size transformers on benchmarks like MMLU but demonstrates that recurrent designs can stay competitive while being far cheaper to serve.