PKU-YuanGroup/Video-LLaVA

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

3500 stars · 255 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

Video-LLaVA is a large vision-language model that aligns image and video representations into a unified visual space before projection into the language model, enabling joint image and video understanding. It is an EMNLP 2024 research codebase with pretrained checkpoints, training scripts, and demo spaces.

Use cases

• answer questions about videos with an llm
• video question answering model
• chat with images and videos using a vision-language model
• run multimodal llm inference on video
• fine-tune a video-language model with instruction tuning
• unified image and video understanding model

When to choose

• you need a single model that handles both image and video inputs
• you want an open-source research baseline for video-language alignment
• you need video question answering or captioning with an LLM

When to avoid

• you need production-grade, commercially supported multimodal APIs
• you lack a GPU for inference or training
• you only need text-only LLM capabilities

Facets

library · maturity active

machine-learning deep-learning llm-inference video-processing image-processing nlp large-language-models computer-vision artificial-intelligence python multimodal vision-language-model video-understanding instruction-tuning video-qa research video natural-language-processing gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="PKU-YuanGroup/Video-LLaVA")