AudioPaLM

AudioPaLM is a large language model introduced by Google in June 2023 that can both understand and generate speech. It combines the text capabilities of PaLM-2 with the audio generation of AudioLM, producing a unified model that speaks, listens, and translates across languages.

The model demonstrated speech-to-speech translation by preserving the speaker's voice across languages, as well as voice transfer and text-to-speech generation from text tokens alone. It set a new bar for zero-shot multilingual voice cloning quality at the time.

AudioPaLM was released as a research demonstration rather than a product, with no public weights. Its techniques informed later Google speech products and open speech research.