lipku/LiveTalking

Health v2 · maintenance only

Adoption not part of the score

9238 stars · 1459 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

LiveTalking is an open-source real-time interactive streaming digital human engine that drives a talking-head avatar from text or audio with lip-sync, optionally combined with LLM and TTS for conversational avatars. It supports multiple models (Wav2Lip, MuseTalk, ER-NeRF, Ultralight-Digital-Human), voice cloning, interruption, and outputs via WebRTC, RTMP, or virtual camera.

Use cases

• build a real-time talking avatar that lip-syncs to speech
• create a 24/7 AI virtual streamer for live commerce
• deploy an AI digital human customer service agent
• generate talking-head videos from scripts via API
• build an interactive virtual presenter for education or kiosks
• integrate an LLM-powered avatar with voice conversation and interruption

When to choose

• you need a self-hosted, commercially usable (Apache-2.0) real-time digital human with low latency
• you want to switch between multiple lip-sync models like Wav2Lip, MuseTalk, or ER-NeRF
• you need streaming output over WebRTC/RTMP or a virtual camera
• you want voice cloning, interruption support, and HTTP API integration with LLM/TTS pipelines

When to avoid

• you only need offline video generation of talking heads without real-time streaming
• you lack a CUDA-capable GPU, since inference requires GPU acceleration
• you need enterprise features like transparent backgrounds, multi-avatar scenes, or wake-word interruption, which are in the paid commercial edition
• you want a no-code hosted SaaS rather than a deployable engine

Facets

framework · maturity active

tts speech-recognition llm-inference video-processing audio-processing machine-learning streaming api-framework chatbot artificial-intelligence computer-vision deep-learning web-development media self-hosted windows python self-hosted digital-human talking-head lip-sync wav2lip musetalk nerf webrtc rtmp voice-cloning virtual-streamer realtime-streaming avatar video audio linux macos docker gpu web-server

Member repositories

For agents

markdown · JSON · MCP: product_card(name="lipku/LiveTalking")