MiniGPT-4

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

25605 stars · 2861 forks observed · 2026-10-02

What it is AI-extracted, prompt v1, taxonomy v1, 2026-10-12, confidence 0.97

MiniGPT-4 is an open-source Python library that aligns a frozen visual encoder with a frozen large language model to enable multimodal image understanding and generation. It provides pretrained models and scripts for fine‑tuning on custom image‑text data.

Use cases

• Generate detailed captions for images using a large language model
• Create textual descriptions or stories based on visual inputs
• Build interactive AI assistants that can answer questions about images
• Develop applications that convert hand‑written sketches into web pages
• Fine‑tune a vision‑language model on a specific domain dataset

When to choose

• You need a lightweight multimodal model that leverages a frozen LLM and visual encoder
• You want to experiment with vision‑language tasks without training a full model from scratch
• Your project runs on Python and you have access to GPU resources for inference

When to avoid

• You require real‑time inference on CPU‑only environments
• You need a fully trainable end‑to‑end vision‑language model from the ground up
• Your use case demands support for modalities beyond images, such as video or audio

Facets

library · maturity active

llm-inference computer-vision nlp parser artificial-intelligence computer-vision python windows vision-language multimodal vicuna vit q-former projection-layer fine-tuning natural-language-processing linux macos

Member repositories

For agents

markdown · JSON · MCP: product_card(name="Vision-CAIR/MiniGPT-4")