ericrisco/rsc-harness / skills/unsloth
unsloth
VerifiedUse when fine-tuning an open-weight LLM fast on ONE GPU with low VRAM — Unsloth's fast model loaders with 4-bit QLoRA and the trl trainer, response-only loss masking so the prompt is not trained on, GRPO reasoning fine-tunes, and export to merged 16-bit, GGUF or the Hub. NOT whether, why or which method to fine-tune (that is `finetuning`), NOT running the exported GGUF locally (that is `ollama`), NOT serving-engine flags and throughput (that is `vllm`), NOT building the JSONL dataset (that is `training-data`).
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 14.0 KB |
| evals/README.md | markdown | 756 B |
| evals/cases.yaml | config | 3.8 KB |
| references/export.md | markdown | 3.5 KB |
| references/grpo.md | markdown | 3.7 KB |
| references/masking-and-templates.md | markdown | 4.2 KB |
references/export.md
View on GitHub# Export: from trained LoRA to a runnable artifact
After `trainer.train()` you hold LoRA adapters on top of a (possibly 4-bit) base. Export depends on
where it runs next. Tracks docs.unsloth.ai inference-and-deployment pages (accessed 2026-07).
## The matrix
| Target | Method | Notes |
| -------------------------- | ------------------------------------------------------------- | ----- |
| vLLM / re-host / portable | `save_pretrained_merged(dir, tok, save_method="merged_16bit")`| Base + LoRA merged at 16-bit. The high-quality keeper. |
| Small, hot-swappable | `save_pretrained_merged(dir, tok, save_method="lora")` | Adapters only (~MBs). Load onto the base at run time. |
| Ollama / llama.cpp | `save_pretrained_gguf(dir, tok, quantization_method="q4_k_m")`| Converts + quantizes to GGUF. |
| DPO / niche 4-bit inference| `save_method="merged_4bit"` | **Discouraged** — quality loss; only if a tool needs it. |
## Merge to 16-bit (recommended keeper)
```python
model.save_pretrained_merged("model_16bit", tokenizer, save_method = "merged_16bit")
# push instead of / in addition to saving:
model.push_to_hub_merged("user/model", tokenizer,
save_method = "merged_16bit", token = "hf_...")
```
`merged_16bit` is the format vLLM and most serving stacks expect. Prefer it over `merged_4bit` — merge
at 16-bit, quantize downstream if you need to.
## LoRA adapters only
```python
model.save_pretrained("finetuned_lora"); tokenizer.save_pretrained("finetuned_lora")
# or save_pretrained_merged(..., save_method="lora")
```
Tiny and hot-swappable (e.g. vLLM LoRA hot-swap). Requires the base at inference time.
## GGUF for Ollama / llama.cpp
```python
# one quant:
model.save_pretrained_gguf("model_gguf", tokenizer, quantization_method = "q4_k_m")
# several at once, pushed to the Hub:
model.push_to_hub_gguf("user/model-gguf", tokenizer,
quantization_method = ["q4_k_m", "q8_0", "f16"], token = "hf_...")
```
Quant guide (size vs quality — this is a real tradeoff, not free):
| Method | ~bits/weight | Use it for |
| -------- | ------------ | ---------- |
| `q4_k_m` | ~4.9 | Everyday default: ~half the size for a few % quality loss. |
| `q5_k_m` | ~5.5 | A notch more quality than Q4 for a bit more size. |
| `q8_0` | ~8.5 | Near-lossless; larger. |
| `f16` | 16 | Unquantized ceiling; biggest. |
Lower than Q4 (Q2/Q3) drops quality sharply — avoid unless desperate for size.
### Manual conversion (when the helper can't, or you want control)
Merge to 16-bit first, then run llama.cpp's converter:
```bash
python llama.cpp/convert_hf_to_gguf.py model_16bit \
--outfile model-F16.gguf --outtype f16 --split-max-size 50G
# then quantize with llama.cpp's quantize tool to q4_k_m / q8_0, etc.
```
## Handoff — this skill stops at the file
- **Run the GGUF locally →** `ollama`. Roughly: `ollama create <name> -f Modelfile` with
`FROM ./model-Q4_K_M.gguf`, then `ollama run <name>`. The Modelfile/quant/VRAM details are its job.
Unsloth's docs also show `OLLAMA_MODELS=unsloth ollama run merged_file.gguf` for a quick check.
- **Serve merged-16bit at throughput →** `vllm` (engine flags, batching, LoRA hot-swap).
- **Host/share the weights →** `huggingface` (the `push_to_hub_*` calls above land there).
Export is the last thing this skill owns; running and serving belong to those siblings.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 471k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |