Octopus v2

Octopus v2 was released by Nexa AI in 2024 (arXiv 2404.01744) as a 2-billion-parameter language model specialized for function calling on edge devices. Its key innovation is the 'functional token' strategy: instead of stuffing tens of thousands of tokens of API descriptions into context via RAG, the model learns mappings from functional tokens to function descriptions, achieving 98-100% function-call accuracy comparable to GPT-4 while running in 1.1-1.7 seconds on a standard Android phone.

Benchmarks claimed 36x faster inference than a Llama-7B-plus-RAG pipeline on a single A100. Note the license caveat: the reference implementation loads it as a Gemma-family model, so check the Hugging Face model card for the exact license terms before commercial use.