Fox-1-1.6B

Fox-1 is TensorOpera's open-source small language model with 1.6 billion parameters, trained from scratch on 3 trillion tokens of text and code using a three-stage data curriculum. Its architecture is notably deeper than competing small models, with grouped-query attention and an 8K context length, and the company reports benchmark results on par with or better than Gemma-2B and Qwen1.5-1.8B despite the smaller size.

The model is released under Apache 2.0 and ships in base and instruction-tuned variants, aimed at developers who need a capable model for edge devices and cost-sensitive cloud serving. TensorOpera also published a technical report documenting the training recipe.