lyuchenyang/Macaw-LLM

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1591 stars · 130 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Macaw-LLM is a multi-modal language modeling framework that integrates image, video, audio, and text data, built on CLIP, Whisper, and LLaMA. It provides code, datasets, and pretrained models for research into multi-modal instruction-following language models.

Use cases

• build a multimodal LLM that understands images, video, audio and text
• experiment with combining CLIP, Whisper and LLaMA into one model
• train a language model on multi-modal instruction data
• research multi-modal alignment strategies for LLMs
• run a demo answering questions about images or audio
• fine-tune a multimodal model on custom datasets

When to choose

• you need a research codebase for multi-modal language modeling across image, video, audio and text
• you want to build on CLIP, Whisper, and LLaMA with an existing alignment pipeline
• you are exploring multi-modal instruction tuning and need a starting point

When to avoid

• you need a production-ready, well-maintained multimodal assistant
• you only need single-modality text generation
• you require commercial support or extensive documentation

Facets

library · maturity experimental

machine-learning deep-learning llm-training nlp image-processing audio-processing video-processing large-language-models machine-learning deep-learning artificial-intelligence python multimodal llm clip whisper llama research natural-language-processing

Member repositories

For agents

markdown · JSON · MCP: product_card(name="lyuchenyang/Macaw-LLM")