VALL-E

VALL-E, announced by Microsoft Research on January 5, 2023, was the first text-to-speech system to treat synthesis as a language-modeling problem over discrete neural audio codec tokens. Trained on 60,000 hours of English speech from LibriLight, it can clone an unseen speaker's voice — timbre, emotion, pacing, and acoustic environment — from a single 3-second enrollment clip, with no per-speaker fine-tuning, mirroring GPT-3's few-shot behavior.

Its significance was as much sociotechnical as technical: Microsoft deliberately withheld all model weights, training code, and public API access, citing deepfake, fraud, and voice-authentication bypass risks — a landmark early example of non-release as a safety policy. Follow-up systems (VALL-E X, VALL-E R, VALL-E 2) were likewise kept private.

The approach spawned an entire ecosystem of open neural-codec TTS models; VALL-E itself remains unavailable except as a research reference.