om-ai-lab/VLM-R1

Health v2 · maintenance only

Adoption not part of the score

6015 stars · 383 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

VLM-R1 is a framework for training R1-style large vision-language models using reinforcement learning (GRPO) on top of Qwen2.5-VL. It provides training scripts for full and LoRA fine-tuning, multi-node training, and released checkpoints for tasks like referring expression comprehension and open-vocabulary detection.

Use cases

• fine-tune a vision-language model with GRPO reinforcement learning
• train a VLM for referring expression comprehension
• improve out-of-domain generalization of visual reasoning models
• run LoRA fine-tuning on Qwen2.5-VL
• reproduce R1-style RL training for multimodal models
• train an open-vocabulary detection model

When to choose

• you want to apply RL post-training (GRPO) to vision-language models
• you need better out-of-domain generalization than SFT provides
• you want proven checkpoints for REC or open-vocabulary detection

When to avoid

• you only need to run inference with a VLM without training
• you lack multi-GPU resources for RL fine-tuning
• you need simple supervised fine-tuning only

Facets

library · maturity active

reinforcement-learning llm-training machine-learning computer-vision deep-learning large-language-models computer-vision machine-learning python vlm grpo deepseek-r1 qwen multimodal fine-tuning visual-reasoning referring-expression-comprehension gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="om-ai-lab/VLM-R1")