TinySwallow-1.5B

TinySwallow-1.5B is a Japanese compact language model from Sakana AI and the Swallow Team (Tokyo Tech), released in January 2025 under Apache 2.0. It was created with TAID (Temporally Adaptive Interpolated Distillation), a new knowledge-distillation method that interpolates teacher and student logits over training time, using Qwen2.5-32B-Instruct as teacher and Qwen2.5-1.5B-Instruct as student, followed by further pretraining on Japanese text.

The base model carries Apache 2.0, though the instruct variant's card notes it was trained on Gemma-family data, so Gemma Terms of Use also apply there (flagged for compliance). An instruct version, TinySwallow-1.5B-Instruct, is available for chat use, with GGUF and MLC conversions for local deployment.