Polyglot-Ko-12.8B

Polyglot-Ko-12.8B is the flagship of EleutherAI's Polyglot-Ko series, a family of Korean autoregressive language models described in arXiv 2306.02254. It was trained on 167 billion tokens over 301,000 steps using 256 A100 GPUs with the GPT-NeoX framework, drawing on 863GB of curated Korean data from TUNiB spanning blogs, news, patents, Q&A, fiction, and Wikipedia.

The project applied PII masking for bank accounts, resident registration numbers, and phone numbers during preprocessing. All Polyglot-Ko models ship under Apache 2.0, a deliberate choice to serve Korean companies and researchers with a fully open foundation model. It is evaluated on the KOBEST Korean benchmark suite.