F5-TTS
Flow-matching zero-shot TTS — clone any voice from seconds of audio, MIT-licensed, with Gradio app, Docker, and Triton runtime.
About
F5-TTS is an open-source text-to-speech system based on flow matching with a Diffusion Transformer and ConvNeXt V2 text refinement, from Yushen Chen's team at SJTU (X-LANCE lab), paper arXiv:2410.06885. It performs zero-shot voice cloning: given a short reference clip and its transcript, it synthesizes new speech in that voice without any fine-tuning. The repo ships a pip package, CLI, Gradio web app (chunk inference, multi-style/multi-speaker generation, Qwen2.5-3B-powered voice chat), Docker images, an E2-TTS reproduction, fine-tuning scripts, and a Triton + TensorRT-LLM production runtime. Sway Sampling, an inference-time flow-step strategy, improves quality at low step counts. MIT-licensed and actively developed.
What it can do
- Zero-shot voice cloning via flow matching (Diffusion Transformer + ConvNeXt V2 text encoder) — reference audio + transcript in, cloned voice out, no fine-tuning needed
- Sway Sampling: inference-time flow-step sampling strategy that improves quality at low NFE (number of function evaluations)
- Gradio web app: basic TTS with chunk inference, multi-style/multi-speaker generation, voice chat demo powered by Qwen2.5-3B-Instruct
- CLI (`f5-tts_infer-cli`), pip package, and official Docker images (ghcr.io/swivid/f5-tts)
- Production runtime with Triton + TensorRT-LLM — vendor benchmark: RTF 0.0394 (avg 253ms latency, concurrency 2) on a single L20 GPU
- Fine-tuning and training scripts included; E2-TTS (Flat-UNet Transformer) reproduction bundled in the same repo
- Custom inference configs for extended language support (community models beyond EN/ZH)
Setup
Inference-only install (Python 3.10+):
```bash
pip install f5-tts
```
Install PyTorch matched to your device first (e.g. CUDA: pip install torch==2.8.0+cu128 torchaudio==2.8.0+cu128 --extra-index-url https://download.pytorch.org/whl/cu128; Apple Silicon: plain pip install torch torchaudio), plus ffmpeg.
Launch the Gradio web UI:
```bash
f5-tts_infer-gradio
# or: f5-tts_infer-gradio --port 7860 --host 0.0.0.0
```
Docker (GPU):
```bash
docker container run --rm -it --gpus=all -p 7860:7860 ghcr.io/swivid/f5-tts:main f5-tts_infer-gradio --host 0.0.0.0
```
For training/fine-tuning, clone and install editable: git clone https://github.com/SWivid/F5-TTS.git && cd F5-TTS && pip install -e .. Model weights download from Hugging Face on first run. Read src/f5_tts/infer guidance before tuning inference settings.
Prerequisites
Python 3.10+. PyTorch matched to your hardware (CUDA/ROCm/XPU/Apple Silicon builds documented in README). ffmpeg. GPU strongly recommended — CPU inference is far slower than real-time. Weights download from Hugging Face on first run. No API key. For the Triton/TensorRT-LLM production runtime, see src/f5_tts/runtime/triton_trtllm/README.md.
Use cases
Works with
Trust score — 9/10
Scored 9/10 (verified tier, metrics-v1). Evidence: 15,386 GitHub stars (log-scaled vs catalog peers); blended with ~45,839 monthly package downloads; last push 2d ago; 10 releases/12mo; 62 open issues. Manual review: Triton RTF benchmark is repo's own measurement; '3 seconds of reference audio' community-reported; cloning-misuse risk inherent. License: MIT.
★ 15,386 GitHub stars
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 7.6/10
- Maintenance · recency, releases, issue load
- 9.2/10
- Openness · license + docs
- 10.0/10
- Manual review · sanity read + red-flag check
- 9.0/10
- · 15,386 GitHub stars (log-scaled vs catalog peers)
- · blended with ~45,839 monthly package downloads
- · last push 2d ago; 10 releases/12mo; 62 open issues
- · manual review: Triton RTF benchmark is repo's own measurement; '3 seconds of reference audio' community-reported; cloning-misuse risk inherent
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.