DAMO-NLP-SG/Video-LLaMA

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

3139 stars · 287 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Video-LLaMA is an instruction-tuned audio-visual language model that extends LLaMA with video and audio understanding via cross-modal pretraining (BLIP-2 style Q-former). It provides pretrained and finetuned checkpoints plus inference code for multimodal video chat.

Use cases

• chat with a video using an LLM
• understand video content with a multimodal model
• ask questions about a video's audio and visuals
• build a video question-answering demo
• finetune a video-language model on custom data
• summarize videos with a large language model

When to choose

• you need open checkpoints for video+audio instruction-following chat
• you want a research baseline for video-language understanding
• you want to customize a pretrained video LLM with your own data

When to avoid

• you need the strongest or easiest-to-use codebase - the authors recommend VideoLLaMA2
• you need production-grade, actively maintained software
• you lack GPU resources for 7B/13B multimodal inference
• you need Chinese-language chat support

Facets

library · maturity maintenance

machine-learning deep-learning llm-inference video-processing audio-processing chatbot large-language-models computer-vision artificial-intelligence python multimodal video-understanding audio-visual instruction-tuning llama blip2 vision-language-model research-model video natural-language-processing gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="DAMO-NLP-SG/Video-LLaMA")