# IDEA-Research/Rex-Omni

[CVPR2026] Detect Anything via Next Point Prediction

Repository: https://github.com/IDEA-Research/Rex-Omni
Canonical: https://ross.abutalabs.com/products/rex-omni
Homepage: https://rex-omni.github.io/
Language: Jupyter Notebook
License: NOASSERTION
License Family: other
Topics: mllm, object-detection, open-set
Last push: 2026-02-22T02:59:06+00:00

## Health v2 (maintenance only)
Score: 47/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 68, release rhythm 35, longevity 23
- inputs: {"age_days": 334, "days_push": 192, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases, no_license
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1561, forks 111 (observed 2026-08-28T04:05:03.838916+00:00)

## What it is
Rex-Omni is a 3B-parameter multimodal large language model that unifies object detection, OCR, pointing, keypoint detection, and visual prompting as a next-point-prediction task. It ships with a Python wrapper for inference, fine-tuning (SFT and GRPO), evaluation code, and an AWQ-quantized checkpoint.

## Use cases
- detect objects in images by category name
- open-set object detection with text prompts
- locate objects from referring expressions
- point to objects described in natural language
- OCR with bounding boxes and polygons
- keypoint detection for humans and animals
- detect objects matching a visual prompt box
- fine-tune a detection model on custom pointing datasets

## When to choose
- you need one unified model for multiple visual grounding tasks
- you want open-vocabulary detection driven by natural language
- you need a lightweight 3B model with quantized weights for GPU inference
- you want to fine-tune detection or pointing behavior with SFT or RL

## When to avoid
- you need real-time detection on CPU-only hardware
- you need a mature production detector with a permissive license
- you need video or 3D perception rather than single-image tasks

## Facets
- artifact type: library
- maturity: active
- function: computer-vision, ocr, machine-learning, llm-inference
- domain: computer-vision, image-processing, artificial-intelligence, large-language-models
- platform: python, cross-platform
- tags: mllm, open-set-detection, next-point-prediction, visual-grounding, keypoint-detection, visual-prompting, grpo, fine-tuning, awq-quantization, natural-language-processing, gpu

## Member repositories
- IDEA-Research/Rex-Omni (main) score 47

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:05:03.838916+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T04:29:43.103056+00:00, confidence not recorded.
  - readme: https://github.com/IDEA-Research/Rex-Omni (fetched 2026-08-28T04:05:03.838916+00:00, sha b91000771437)
  - homepage: https://rex-omni.github.io/ (fetched 2026-08-29T11:29:22.595666+00:00, sha f08933d2ed0b)
- Data as of 2026-08-30T08:39:29.467469+00:00.