Guidance

Guidance is Microsoft's programming paradigm for controlling language models: instead of hoping the model emits valid JSON, you constrain generation itself — regexes, context-free grammars, and structured gen() calls guarantee output syntax while interleaving Python control flow (conditionals, loops, tool calls) with generation. It supports many backends (Transformers, llama.cpp, OpenAI-compatible APIs) and is designed to cut latency and cost versus repeated prompting or fine-tuning for structured tasks.

```bash

pip install guidance

```

```python

from guidance import system, user, assistant, gen

from guidance.models import Transformers

lm = Transformers("microsoft/Phi-4-mini-instruct")

with system():

lm += "You are a helpful assistant"

with user():

lm += "How old are you?"

with assistant():

lm += gen("age", regex=r"\d+", max_tokens=10)

print(lm["age"])

```

Backends: Transformers, LlamaCpp, OpenAI, and more — pick per model. Docs in the repo's docs/ folder.