apple/ml-ferret

Health v2 · maintenance only

Flags: no_releases no_license

Adoption not part of the score

8655 stars · 518 forks observed · 2026-10-07

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

Apple's Ferret, an end-to-end multimodal large language model (MLLM) that accepts any-form referring and grounds anything in its responses, with fine-grained open-vocabulary referring and grounding via hybrid region representation. Includes Ferret-UI, a UI-centric MLLM for referring, grounding, and reasoning on mobile interfaces, plus training/evaluation code and checkpoints.

Use cases

• ground objects in images with natural language referring
• build a multimodal chatbot that understands image regions
• run referring and grounding reasoning on mobile UI screenshots
• fine-tune a vision-language model with the GRIT instruction dataset
• evaluate MLLMs on Ferret-Bench multimodal benchmarks

When to choose

• you need fine-grained region-level referring and grounding in an MLLM
• you want a UI-understanding model for app screenshots
• you're doing research on multimodal instruction tuning

When to avoid

• you need a production-ready commercial product (research-only license)
• you just need a general-purpose VLM without grounding capabilities
• you lack GPU resources for 7B/13B model inference

Facets

library · maturity maintenance

machine-learning deep-learning nlp computer-vision llm-inference llm-training large-language-models computer-vision artificial-intelligence image-processing python multimodal vision-language-model visual-grounding referring-expression research-code ferret-ui instruction-tuning natural-language-processing linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="apple-aiml-research/ml-ferret")