# PKU-YuanGroup/MoE-LLaVA

【TMM 2025🔥】 Mixture-of-Experts for Large Vision-Language Models

Repository: https://github.com/PKU-YuanGroup/MoE-LLaVA
Canonical: https://ross.abutalabs.com/products/moe-llava
Homepage: https://arxiv.org/abs/2401.15947
Language: Python
License: Apache-2.0
License Family: permissive
Topics: large-vision-language-model, mixture-of-experts, moe, multi-modal
Last push: 2025-07-15T07:59:33+00:00

## Health v2 (maintenance only)
Score: 31/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 31, release rhythm 8, longevity 70
- inputs: {"age_days": 993, "days_push": 414, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 2322, forks 139 (observed 2026-08-28T04:06:37.225046+00:00)

## What it is
MoE-LLaVA is an open-source Mixture-of-Experts based sparse large vision-language model, released with the MoE-Tuning training strategy from a TMM 2025 paper. It activates only top-k experts per token, achieving LLaVA-1.5-7B-level performance with roughly 3B activated parameters.

## Use cases
- run a sparse vision-language model for image captioning
- build a multimodal chatbot that answers questions about images
- research mixture-of-experts training for LVLMs
- reduce inference cost of vision-language models via sparsity
- benchmark visual understanding and object hallucination
- fine-tune a MoE-based multi-modal model on custom data

## When to choose
- you need an efficient sparse LVLM with low activated parameter count
- you are researching MoE architectures for multi-modal models
- you want a reproducible baseline from a peer-reviewed paper
- you need image understanding with strong hallucination resistance

## When to avoid
- you need a production-grade, commercially supported multimodal API
- you lack GPU resources for training or inference
- you need video or audio understanding rather than images
- you want a plug-and-play model with broad ecosystem tooling

## Facets
- artifact type: library
- maturity: active
- function: machine-learning, deep-learning, llm-inference, nlp, computer-vision
- domain: large-language-models, computer-vision, deep-learning, artificial-intelligence
- platform: python
- tags: mixture-of-experts, vision-language-model, multimodal, sparse-model, moe-tuning, research-code, gpu, linux

## Member repositories
- PKU-YuanGroup/MoE-LLaVA (main) score 31

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:06:37.225046+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T02:38:36.911521+00:00, confidence not recorded.
  - readme: https://github.com/PKU-YuanGroup/MoE-LLaVA (fetched 2026-08-28T04:06:37.225046+00:00, sha 3450b89e9886)
  - homepage: https://arxiv.org/abs/2401.15947 (fetched 2026-08-29T10:18:57.521515+00:00, sha c8ae5ccd9163)
  - site_page: https://info.arxiv.org/about/donate.html (fetched 2026-08-29T10:18:57.534312+00:00, sha cca9c3a11c56)
  - site_page: https://info.arxiv.org/about/ourmembers.html (fetched 2026-08-29T10:18:57.537867+00:00, sha 47cbc55ff1de)
  - site_page: https://info.arxiv.org/about (fetched 2026-08-29T10:18:57.539754+00:00, sha a1f16f915a9a)
  - site_page: https://info.arxiv.org/labs/index.html (fetched 2026-08-29T10:18:57.536233+00:00, sha b14a8d05a0ec)
- Data as of 2026-08-30T08:39:29.467469+00:00.