Skip to content

Orchestra-Research/AI-Research-SKILLs / 19-emerging-techniques/model-merging

model-merging

Verified

Merge multiple fine-tuned models using mergekit to combine capabilities without retraining. Use when creating specialized models by blending domain-specific expertise (math + coding + chat), improving performance beyond single models, or experimenting rapidly with model variants. Covers SLERP, TIES-Merging, DARE, Task Arithmetic, linear merging, and production deployment strategies.

FileKindSize
SKILL.mdskill md12.5 KB
references/coefficient-tuning.mdmarkdown14.3 KB
references/evaluation.mdmarkdown11.4 KB
references/examples.mdmarkdown8.6 KB
references/methods.mdmarkdown9.3 KB

references/examples.md

View on GitHub
# Model Merging Examples

Real-world merge configurations from successful models on HuggingFace and research papers.

## Table of Contents
- Successful Merges
- Mixtral-based Merges
- Llama-based Merges
- Task-Specific Merges
- Production Examples

## Successful Merges

### Marcoro14-7B-slerp

**Achievement**: #1 on Open LLM Leaderboard (February 2024)
**Method**: SLERP
**Source**: HuggingFace

```yaml
# marcoro14-7b-slerp.yml
merge_method: slerp
slices:
  - sources:
      - model: AIDC-ai-business/Marcoroni-7B-v3
        layer_range: [0, 32]
      - model: EmbeddedLLM/Mistral-7B-Merge-14-v0.1
        layer_range: [0, 32]
parameters:
  t: 0.5  # Equal blend
dtype: bfloat16
```

**Results**:
- Average: 74.32 on Open LLM Leaderboard
- Strong across all tasks
- Smooth capability combination

### goliath-120b (Mixtral MoE)

**Method**: Linear + SLERP
**Achievement**: Top-performing 120B model

```yaml
# goliath-120b.yml
merge_method: slerp
slices:
  - sources:
      - model: alpindale/c4ai-command-r-plus-GPTQ
        layer_range: [0, 40]
      - model: CohereForAI/c4ai-command-r-v01
        layer_range: [0, 40]
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]  # Layer-specific blending
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5  # Default
dtype: float16
```

## Mixtral-based Merges

### Math + Code Specialist

**Goal**: Combine mathematical reasoning with code generation

```yaml
# math-code-mixtral.yml
merge_method: task_arithmetic
base_model: mistralai/Mixtral-8x7B-v0.1
models:
  - model: WizardLM/WizardMath-7B-V1.1
    parameters:
      weight: 0.6  # Emphasize math
  - model: ajibawa-2023/Code-Mixtral-8x7B
    parameters:
      weight: 0.4  # Add code
dtype: bfloat16
```

**Expected capabilities**:
- Strong mathematical reasoning
- Code generation and understanding
- Technical problem-solving

### Chat + Roleplay Merge

```yaml
# chat-roleplay.yml
merge_method: slerp
slices:
  - sources:
      - model: teknium/OpenHermes-2.5-Mistral-7B
        layer_range: [0, 32]
      - model: Undi95/MLewd-ReMM-L2-Chat-20B-Part1
        layer_range: [0, 32]
parameters:
  t: 0.5
dtype: bfloat16
```

### Multi-Task TIES Merge

```yaml
# multi-task-mixtral.yml
merge_method: ties
base_model: mistralai/Mixtral-8x7B-v0.1
models:
  - model: WizardLM/WizardMath-7B-V1.1
    parameters:
      density: 0.5
      weight: 1.0
  - model: teknium/OpenHermes-2.5-Mistral-7B
    parameters:
      density: 0.5
      weight: 1.0
  - model: ajibawa-2023/Code-Mixtral-8x7B
    parameters:
      density: 0.5
      weight: 1.0
parameters:
  normalize: true
dtype: bfloat16
```

## Llama-based Merges

### Platypus-Hermes Merge

**Models**: Garage-bAInd/Platypus2-13B + WizardLM/WizardLM-13B-V1.2

```yaml
# platypus-hermes-13b.yml
merge_method: linear
models:
  - model: garage-bAInd/Platypus2-13B
    parameters:
      weight: 0.5
  - model: WizardLM/WizardLM-13B-V1.2
    parameters:
      weight: 0.3
  - model: psmathur/orca_mini_v3_13b
    parameters:
      weight: 0.2
dtype: float16
```

### DARE-TIES Llama Merge

**Source**: DARE paper (arXiv 2311.03099)

```yaml
# dare-ties-llama.yml
merge_method: dare_ties
base_model: meta-llama/Llama-2-7b-hf
models:
  - model: WizardLM/WizardLM-7B-V1.0
    parameters:
      density: 0.5   # Keep top 50%
      weight: 0.6
      dare:
        drop_rate: 0.9  # Drop 90% of deltas
  - model: garage-bAInd/Platypus-7B
    parameters:
      density: 0.5
      weight: 0.4
      dare:
        drop_rate: 0.9
parameters:
  int8_mask: true
dtype: bfloat16
```

## Task-Specific Merges

### Medical Domain

**Goal**: Create medical specialist model

```yaml
# medical-specialist.yml
merge_method: task_arithmetic
base_model: mistralai/Mistral-7B-v0.1
models:
  - model: medalpaca/medalpaca-7b
    parameters:
      weight: 0.7  # Strong medical knowledge
  - model: teknium/OpenHermes-2.5-Mistral-7B
    parameters:
      weight: 0.3  # Add general chat ability
dtype: bfloat16
```

### Legal Assistant

```yaml
# legal-assistant.yml
merge_method: slerp
slices:
  - sources:
      - model: law-ai/legal-bert-7b
        layer_range: [0, 32]
      - model: teknium/OpenHermes-2.5-Mistral-7B
        layer_range: [0, 32]
parameters:
  t:
    - filter: self_attn
      value: 0.7  # Emphasize legal model in attention
    - filter: mlp
      value: 0.3  # More general chat in MLPs
    - value: 0.5
dtype: bfloat16
```

### Multilingual Merge

```yaml
# multilingual-merge.yml
merge_method: linear
models:
  - model: mistralai/Mistral-7B-v0.1
    parameters:
      weight: 0.4  # English
  - model: CohereForAI/aya-23-7B
    parameters:
      weight: 0.3  # Multilingual
  - model: Qwen/Qwen3-7B
    parameters:
      weight: 0.3  # Asian languages
dtype: bfloat16
```

## Production Examples

### Gradual Merge (Safer)

**Strategy**: Merge incrementally, test at each step

```yaml
# Step 1: Merge two models
# step1.yml
merge_method: slerp
slices:
  - sources:
      - model: base_model
        layer_range: [0, 32]
      - model: specialist_1
        layer_range: [0, 32]
parameters:
  t: 0.3  # Conservative blend
dtype: bfloat16
```

```yaml
# Step 2: Add third model to result
# step2.yml
merge_method: slerp
slices:
  - sources:
      - model: ./merged_step1  # Previous merge
        layer_range: [0, 32]
      - model: specialist_2
        layer_range: [0, 32]
parameters:
  t: 0.3  # Conservative
dtype: bfloat16
```

**Benefits**:
- Test after each merge
- Easier to debug
- Can stop if quality degrades

### A/B Testing Setup

```yaml
# variant_a.yml - Conservative
merge_method: slerp
slices:
  - sources:
      - model: base_model
        layer_range: [0, 32]
      - model: specialist
        layer_range: [0, 32]
parameters:
  t: 0.3  # 30% specialist
dtype: bfloat16
```

```yaml
# variant_b.yml - Aggressive
merge_method: slerp
slices:
  - sources:
      - model: base_model
        layer_range: [0, 32]
      - model: specialist
        layer_range: [0, 32]
parameters:
  t: 0.7  # 70% specialist
dtype: bfloat16
```

**Test both**, choose best performer

### Frankenmerge (Experimental)

**Warning**: Experimental, may not work

```yaml
# frankenmerge.yml
merge_method: passthrough
slices:
  # First 8 layers from model A
  - sources:
      - model: model_a
        layer_range: [0, 8]

  # Middle 16 layers from model B
  - sources:
      - model: model_b
        layer_range: [8, 24]

  # Last 8 layers from model C
  - sources:
      - model: model_c
        layer_range: [24, 32]
dtype: bfloat16
```

**Use case**: Create models with non-standard layer counts

### MoE from Merges

```yaml
# moe-from-merges.yml
merge_method: moe
base_model: mistralai/Mistral-7B-v0.1
experts:
  - source_model: WizardLM/WizardMath-7B-V1.1
    positive_prompts:
      - "math"
      - "calculate"
      - "solve"
      - "equation"

  - source_model: ajibawa-2023/Code-Mistral-7B
    positive_prompts:
      - "code"
      - "python"
      - "function"
      - "programming"

  - source_model: teknium/OpenHermes-2.5-Mistral-7B
    positive_prompts:
      - "chat"
      - "conversation"
      - "help"
      - "question"
dtype: bfloat16
```

**Result**: Dynamic expert selection based on prompt

## Command-Line Examples

### Basic Merge

```bash
# Simple two-model SLERP
mergekit-yaml config.yml ./output-model \
  --cuda \
  --lazy-unpickle
```

### Large Model Merge (Low VRAM)

```bash
# Merge on CPU (slow but works with 8GB VRAM)
mergekit-yaml config.yml ./output-model \
  --allow-crimes \  # Enable CPU offloading
  --low-cpu-memory
```

### Merge and Upload

```bash
# Merge and push to HuggingFace
mergekit-yaml config.yml ./merged-model --cuda

cd merged-model
python << EOF
from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("./")
tokenizer = AutoTokenizer.from_pretrained("./")

model.push_to_hub("username/my-merged-model")
tokenizer.push_to_hub("username/my-merged-model")
EOF
```

### Batch Merging

```bash
# Merge multiple configs
for config in configs/*.yml; do
  output="./output/$(basename $config .yml)"
  mergekit-yaml $config $output --cuda
done
```

## Tips from Successful Merges

1. **Start Conservative**: Use t=0.3-0.5 for SLERP, test before going higher
2. **Match Architectures**: Only merge models with same base architecture
3. **Test Extensively**: Benchmark on multiple tasks before deploying
4. **Layer-Specific Merging**: Different t values for attention vs MLP often works better
5. **DARE for Many Models**: When merging 3+ models, DARE-TIES often best
6. **Gradual Merging**: For production, merge incrementally and test

## Resources

- **HuggingFace Models**: Browse merged models for inspiration
- **Open LLM Leaderboard**: See top-performing merges
- **mergekit Examples**: https://github.com/arcee-ai/mergekit/tree/main/examples
writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 322kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall