mit-han-lab/streaming-vlm

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1072 stars · 66 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

StreamingVLM is a vision-language model framework from MIT Han Lab for real-time understanding of effectively infinite video streams. It maintains a compact KV cache with training aligned to streaming inference, avoiding quadratic cost and sliding-window pitfalls.

Use cases

• understand infinite live video streams in real time
• run video question answering on long videos
• process hours-long video without quadratic attention cost
• benchmark long video understanding with OVOBench
• fine-tune a VLM for streaming video inference
• achieve real-time FPS video captioning on a single GPU

When to choose

• you need stable real-time comprehension of very long or endless video feeds
• you want a research-grade streaming VLM with training and evaluation scripts
• you have GPU hardware like an H100 and need high-FPS video understanding

When to avoid

• you only need short-clip video analysis with standard VLMs
• you lack GPU resources for inference or fine-tuning
• you need a production-ready turnkey product rather than research code

Facets

library · maturity active

machine-learning video-processing llm-inference rag computer-vision large-language-models artificial-intelligence python vision-language-model streaming-video kv-cache real-time-inference video-understanding research-code video gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="mit-han-lab/streaming-vlm")