RWKV

RWKV (pronounced 'RwaKuv', from Receptance-Weight-Key-Value) is an open-source recurrent neural network architecture that trains like a GPT transformer in parallel yet infers like an RNN with constant memory and no KV cache. Created by Bo Peng (BlinkDL) and adopted as a Linux Foundation AI & Data project, it was the first purely recurrent architecture demonstrated to match transformer perplexity at matched compute budgets (EMNLP 2023).

The family has iterated through RWKV-4, RWKV-5 'Eagle' (matrix-valued states), RWKV-6 'Finch' (data-dependent decay, up to 14B), and RWKV-7 'Goose' (dynamic state evolution with proven expressivity beyond the TC0 barrier that limits attention). With 400+ community projects, a pip package, and strong edge-deployment ergonomics — a 7B model runs on 8GB VRAM — RWKV remains the most commercially deployed attention-free LLM family.