LeoLM

LeoLM, short for Linguistically Enhanced Open Language Model, is a suite of German-first foundation models from LAION in collaboration with HessianAI. The team continued pretraining Llama-2 and Mistral 7B on a large German-language corpus using HessianAI's supercomputer, producing 7B and 13B variants with 8k context windows plus chat-tuned versions evaluated on a German adaptation of MT-Bench. The Mistral-based variant ships under Apache 2.0 while the Llama-2-based ones carry the Llama 2 community license.

Released in late 2023, LeoLM was the first openly available German foundation model meeting contemporary standards, and it demonstrated that large-scale continued pretraining could add a language without destroying the base model's existing capabilities. Chat variants score well on German writing and humanities tasks while lagging on math and reasoning, a tradeoff the authors document openly.