EvolvingLMMs-Lab/Otter

Health v2 · maintenance only

Adoption not part of the score

3436 stars · 209 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

Otter is a multi-modal vision-language model built on OpenFlamingo, instruction-tuned on the MIMIC-IT dataset with image and video understanding checkpoints. It includes training/fine-tuning scripts (e.g., OtterHD based on Fuyu-8B) and evaluation tooling for multimodal benchmarks.

Use cases

• run a multimodal chatbot that understands images and video
• fine-tune a vision-language model on custom instruction data
• evaluate multimodal LLMs on benchmarks like MagnifierBench
• do in-context learning with interleaved image-text prompts
• fine-tune Fuyu-8B for high-resolution image understanding
• research instruction tuning for open multimodal foundation models

When to choose

• you need an open-source multimodal model with released checkpoints you can run or fine-tune
• you want a research baseline for vision-language instruction tuning
• you need image or video understanding with in-context learning examples

When to avoid

• you need a production-ready, polished product rather than research code
• you lack GPU resources for large 7B-8B parameter models
• you need the latest multimodal models rather than a 2023-era research artifact

Facets

library · maturity maintenance

machine-learning deep-learning llm-training nlp image-processing video-processing chatbot large-language-models deep-learning artificial-intelligence computer-vision python multimodal vision-language-model openflamingo instruction-tuning in-context-learning foundation-models mimic-it fuyu natural-language-processing gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="EvolvingLMMs-Lab/Otter")