salesforce/LAVIS

Health v2 · maintenance only

Adoption not part of the score

11262 stars · 1108 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

LAVIS is a Python library from Salesforce AI Research providing a unified toolkit for language-vision (multimodal) intelligence, including pretrained models like BLIP-2, InstructBLIP, and BLIP-Diffusion. It supports tasks such as image captioning, visual question answering, retrieval, and text-to-image generation with datasets, benchmarks, and training utilities.

Use cases

• generate captions for images
• build a visual question answering model
• run zero-shot vision-language inference with BLIP-2
• do subject-driven text-to-image generation
• pretrain or fine-tune a vision-language transformer
• retrieve images using text queries
• evaluate models on multimodal benchmarks

When to choose

• you need state-of-the-art pretrained vision-language models in one library
• you want to fine-tune or instruction-tune multimodal models like InstructBLIP
• you're doing research on image-text tasks like captioning, VQA, or retrieval

When to avoid

• you only need single-modality NLP or computer vision without cross-modal tasks
• you need a production inference server rather than a research library
• you work outside PyTorch/Python ecosystems

Facets

library · maturity active

machine-learning deep-learning image-processing nlp llm-inference stable-diffusion deep-learning computer-vision large-language-models machine-learning artificial-intelligence python cross-platform multimodal vision-language image-captioning visual-question-answering blip pretrained-models research-library natural-language-processing gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="salesforce/LAVIS")