OFA-Sys/ONE-PEACE

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1060 stars · 71 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

ONE-PEACE is a general multimodal representation model that jointly encodes vision, audio, and language modalities without initializing from pretrained vision or language models. The repository provides pretrained checkpoints, fine-tuning and inference scripts, embedding and visual grounding APIs, and a multimodal retrieval demo.

Use cases

• extract joint embeddings for images, audio, and text
• retrieve images using audio, text, or combined audio+text+image queries
• fine-tune a multimodal model on vision-language tasks like VQA and captioning
• fine-tune on audio classification and audio-language tasks
• locate objects in images with visual grounding
• pretrain a modality-agnostic transformer from scratch

When to choose

• you need a single model embedding multiple modalities in one shared space
• you want strong zero-shot cross-modal retrieval including unpaired modality combinations
• you need a research foundation model for multimodal representation learning

When to avoid

• you need a lightweight production embedding service with minimal dependencies
• you only need unimodal text or image models
• you require active community support and frequent updates

Facets

library · maturity maintenance

machine-learning deep-learning nlp image-processing audio-processing search-engine machine-learning deep-learning artificial-intelligence computer-vision python multimodal representation-learning foundation-model vision-language contrastive-learning zero-shot-retrieval embeddings natural-language-processing audio

Member repositories

For agents

markdown · JSON · MCP: product_card(name="OFA-Sys/ONE-PEACE")