dandelin/ViLT

Health v2 · maintenance only

Adoption not part of the score

1538 stars · 229 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Official PyTorch code for the ICML 2021 paper ViLT, a vision-and-language transformer that performs multimodal pre-training without convolutional image feature extractors or region supervision. It ships pretrained checkpoints and finetuned models for VQA, NLVR2, and image-text retrieval, plus Gradio demos.

Use cases

• run visual question answering on images
• answer questions about an image with a pretrained model
• retrieve images from text descriptions
• finetune a vision-language transformer on my own dataset
• reproduce the ViLT paper results
• visualize masked language model attention over images

When to choose

• you need a lightweight multimodal transformer without heavy object-detection feature extraction
• you want pretrained VQA or image-text retrieval models for research
• you want to reproduce or extend the ViLT paper

When to avoid

• you need production-grade multimodal APIs with active support
• you want the latest state-of-the-art vision-language models
• you need a maintained library rather than research code

Facets

library · maturity maintenance

machine-learning deep-learning nlp image-processing artificial-intelligence machine-learning computer-vision python vision-and-language transformer vqa visual-question-answering image-text-retrieval masked-language-modeling pretrained-models research-code natural-language-processing gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="dandelin/ViLT")