MuseTalk
Tencent's real-time lip-sync model — 30fps+ talking faces at 256x256, open weights.
About
MuseTalk, from Tencent Music Entertainment's Lyra Lab, is a real-time high-quality lip-synchronization model that animates a face from audio alone. It generates lip-sync targets in a VAE latent space with a multi-scale U-Net fusing audio and visual features, hitting 30+ fps at 256x256 on a V100 with negligible startup latency — genuinely usable for live streaming and visual dubbing.
It supports Chinese, English, and Japanese audio and pairs with MuseV for full virtual-human video. Limitations: output resolution is capped at 256x256 for the face region (fine for stream overlays, soft for close-ups), and like all dubbing models it can be misused for impersonation — deploy with consent and disclosure controls.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 7/10
Scored 7/10 (metrics tier, metrics-v1). Evidence: Verified via vendor site/docs/GitHub and reputable coverage, wave 65; Trending score 72/100 from researcher assessment.