lucidrains/flamingo-pytorch

Health v2 · maintenance only

Adoption not part of the score

1269 stars · 66 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A PyTorch implementation of DeepMind's Flamingo visual language model architecture, providing the Perceiver Resampler and Gated Cross-Attention blocks for few-shot visual question answering. It is a research building block meant to be inserted into a large language model so text can attend to image embeddings.

Use cases

• implement the Flamingo architecture in PyTorch
• add visual question answering to a large language model
• resample image embeddings with a perceiver resampler
• build a multimodal vision-language model from scratch
• insert gated cross attention between text and media embeddings
• experiment with few-shot visual language model training

When to choose

• you want a lightweight, pip-installable PyTorch implementation of Flamingo's core components
• you are building a custom multimodal LLM and need the perceiver resampler or gated cross-attention blocks
• you want readable research code to study or adapt the Flamingo architecture

When to avoid

• you need a fully trained, ready-to-use visual question answering model with pretrained weights
• you want a complete production multimodal pipeline rather than architectural components
• you need a framework-agnostic or non-PyTorch solution

Facets

library · maturity active

deep-learning machine-learning llm-training transformers artificial-intelligence deep-learning large-language-models computer-vision python flamingo visual-question-answering perceiver-resampler gated-cross-attention vision-language-model research-implementation pytorch multimodal natural-language-processing

Member repositories

For agents

markdown · JSON · MCP: product_card(name="lucidrains/flamingo-pytorch")