amazon-science/mm-cot

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

3985 stars · 333 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

Official PyTorch implementation of the paper 'Multimodal Chain-of-Thought Reasoning in Language Models', which adds vision features to a two-stage rationale-generation and answer-inference framework. It achieves state-of-the-art ScienceQA results with a sub-1B parameter model.

Use cases

• reproduce multimodal chain-of-thought reasoning results
• train a model to answer science questions with images and text
• generate rationales before answering visual QA questions
• extract vision features like CLIP or ViT for ScienceQA
• experiment with two-stage CoT training frameworks
• reduce hallucination in multimodal LLM reasoning

When to choose

• you want to reproduce or extend the Multimodal-CoT paper
• you need a two-stage rationale-then-answer pipeline for vision-language QA
• you're researching chain-of-thought reasoning with image inputs on ScienceQA or A-OKVQA

When to avoid

• you need a production-ready multimodal inference service
• you want a general-purpose LLM framework rather than a research codebase
• you don't have GPU resources for training
• you need a maintained library with API stability guarantees

Facets

library · maturity maintenance

machine-learning deep-learning llm-training nlp image-processing large-language-models machine-learning computer-vision artificial-intelligence python chain-of-thought multimodal vision-language research-code scienceqa rationale-generation paper-implementation research gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="amazon-science/mm-cot")