open-mmlab/Multimodal-GPT

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1512 stars · 128 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Multimodal-GPT is an open-source project for training a multimodal chatbot that combines vision and language instructions, built on OpenFlamingo. It jointly fine-tunes visual and language instruction data (VQA, captioning, visual reasoning, OCR, visual dialogue) using parameter-efficient LoRA tuning.

Use cases

• train a multimodal chatbot that understands images and text
• fine-tune a vision-language model with LoRA
• build a visual question answering assistant
• jointly train on visual and language instruction data
• experiment with OpenFlamingo-based multimodal models
• create a GPT-4-style assistant that can see images

When to choose

• you want to train or fine-tune an open multimodal vision-language chatbot
• you need LoRA-based parameter-efficient tuning of a Flamingo-style model
• you want to combine VQA, captioning, and dialogue instruction data in joint training

When to avoid

• you need a production-ready multimodal assistant with active maintenance
• you want inference-only usage without training infrastructure
• you need the latest multimodal LLM capabilities, as the project has not been updated since mid-2023

Facets

library · maturity maintenance

machine-learning deep-learning llm-training chatbot nlp computer-vision artificial-intelligence large-language-models computer-vision machine-learning python multimodal vision-language flamingo lora visual-instruction-tuning openmmlab natural-language-processing gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="open-mmlab/Multimodal-GPT")