FLOR 6.3B

FLOR-6.3B is a 6.3-billion-parameter transformer language model from the Barcelona Supercomputing Center's Projecte Aina, built to serve Catalan, Spanish and English well instead of treating them as afterthoughts. It is the flagship of the FLOR family, a set of models from 760M to 6.3B parameters that adapt public BLOOM checkpoints through vocabulary surgery: the 250K-token BLOOM vocabulary is replaced with a compact 50,257-subword tokenizer trained on the target languages, then the model is continually pretrained on 140 billion Catalan, Spanish and English tokens. The full FLOR-6.3B training run completed in about 2.5 days on 16 Cerebras CS-2 wafer-scale systems on Condor Galaxy 1.

FLOR is open source and usable for research and commercial applications. The models are available on Hugging Face (projecte-aina/FLOR-6.3B) and in the Azure model registry, with an Instructed variant fine-tuned on the InstruCat Catalan set, the MENTOR_ES and MENTOR_CA instruction sets and a Dolly subset. As a 2024-era 6.3B model it trails modern frontier models on general benchmarks, but it remains a landmark example of sovereign, language-first model development and a practical base for Catalan and Spanish fine-tuning.