jd-opensource/JoyAI-VL-Interaction

Health v2 · maintenance only

Flags: no_releases young

Adoption not part of the score

1806 stars · 180 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

JoyAI-VL-Interaction is an open 8B-scale vision-language interaction model with a complete deployable real-time streaming system, including the model weights, training recipe, time-aligned interaction dataset, and vLLM-based inference stack. It watches live video from a webcam or livestream and proactively speaks up in real time (sub-second latency) when a moment matters, rather than waiting to be asked.

Use cases

• build an always-present AI assistant that watches a live camera feed
• real-time video call assistant that reacts without being polled
• monitor a livestream and get proactive commentary on important moments
• self-host a vision-language model for streaming video understanding
• train or fine-tune a proactive video-language interaction model
• deploy a quantized 8B VLM with vLLM on CUDA GPUs

When to choose

• you need a fully open (weights, data, training recipe) real-time video-language model
• you want proactive, time-aware interaction rather than turn-based Q&A
• you need sub-second latency streaming inference on standard GPU infrastructure
• you want quantized checkpoints (INT4/INT8/FP8/NVFP4) for efficient deployment

When to avoid

• you only need offline video analysis or batch video captioning
• you lack CUDA GPUs or cannot run 8B-scale models
• you need a turn-based chatbot rather than a proactive streaming assistant
• you need a small model for edge or CPU-only devices

Facets

framework · maturity active

llm-inference machine-learning video-processing speech-recognition streaming rag artificial-intelligence large-language-models computer-vision deep-learning python cloud vision-language-model real-time-interaction vllm 8b-model proactive-ai streaming-video self-hosted-ai quantization video real-time gpu linux docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="jd-opensource/JoyAI-VL-Interaction")