GlórIA 1.3B

GlórIA 1.3B is a generative large language model focused on European Portuguese, developed by Ricardo Lopes, João Magalhães, and David Semedo in the NOVA LINCS Multimodal Systems Group. The paper was accepted at PROPOR 2024 and the preprint appeared on arXiv in February 2024.

The model follows the GPT-Neo 1.3B architecture (a 2.7B variant was also trained) and was pre-trained on a 35-billion-token corpus assembled from Portuguese Wikipedia, news, dialogs, and web pages, partly in collaboration with Arquivo.pt, the Portuguese web archive.

The authors introduced CALAME-PT, the first Portuguese zero-shot language-modeling benchmark, on which GlórIA significantly outperformed existing open Portuguese decoder models. Pretrained weights, benchmark, and code are publicly available.