arcee-ai/mergekit

Health v2 · maintenance only

Adoption not part of the score

7310 stars · 785 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

mergekit is a Python toolkit for merging pre-trained large language models directly in weight space, supporting many merge methods (SLERP, TIES, DARE, MoE merging, evolutionary merges) with an out-of-core approach that runs on CPU or minimal GPU memory. It includes companion tools for LoRA extraction, raw PyTorch merging, tokenizer transplantation, and multi-stage merges.

Use cases

• merge two fine-tuned llama models into one
• combine strengths of multiple LLMs without training
• create a mixture-of-experts model from existing checkpoints
• merge language models on a low-VRAM GPU
• extract a LoRA adapter from a merged model
• transfer capabilities between models without training data
• transplant tokenizer between language models

When to choose

• you want to combine pretrained LLM checkpoints without additional training
• you have limited GPU memory and need out-of-core merging
• you want to experiment with many merge algorithms (SLERP, TIES, DARE, evolutionary)
• you work with Llama, Mistral, GPT-NeoX, or similar architectures

When to avoid

• you need to train or fine-tune a model from scratch rather than merge existing ones
• your models have incompatible architectures or vastly different tokenizers without using the specialized tools
• you need a managed GUI service rather than a config-driven CLI toolkit

Facets

library · maturity active

machine-learning llm-training sdk cli large-language-models machine-learning deep-learning developer-tools python cli cross-platform model-merging llm model-fusion moe lora weight-space-merging huggingface gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="arcee-ai/mergekit")