ericrisco/rsc-harness / skills/unsloth
unsloth
VerifiedUse when fine-tuning an open-weight LLM fast on ONE GPU with low VRAM — Unsloth's fast model loaders with 4-bit QLoRA and the trl trainer, response-only loss masking so the prompt is not trained on, GRPO reasoning fine-tunes, and export to merged 16-bit, GGUF or the Hub. NOT whether, why or which method to fine-tune (that is `finetuning`), NOT running the exported GGUF locally (that is `ollama`), NOT serving-engine flags and throughput (that is `vllm`), NOT building the JSONL dataset (that is `training-data`).
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 14.0 KB |
| evals/README.md | markdown | 756 B |
| evals/cases.yaml | config | 3.8 KB |
| references/export.md | markdown | 3.5 KB |
| references/grpo.md | markdown | 3.7 KB |
| references/masking-and-templates.md | markdown | 4.2 KB |
evals/cases.yaml
View on GitHubskill: unsloth
should_trigger:
- prompt: "I want to QLoRA fine-tune Llama-3.1-8B but it OOMs on my 16GB GPU — how do I make it fit?"
why: Core niche — fast, low-VRAM single-GPU fine-tuning; the fit problem is exactly what load_in_4bit + get_peft_model + adamw_8bit + unsloth gradient checkpointing solve.
- prompt: "How do I fine-tune gpt-oss on a single 24GB GPU roughly 2x faster with less VRAM?"
why: Names the backend's headline (2x/less-VRAM), a single GPU, and a supported MoE model — squarely this skill.
- prompt: "My chat fine-tune's loss looks like it's covering the user turn — how do I mask the prompt so it only trains on the response?"
why: The train_on_responses_only / get_chat_template masking problem, the load-bearing gotcha owned here.
- prompt: "After training my LoRA with Unsloth, how do I export it to GGUF Q4_K_M for Ollama?"
why: Export mechanics (save_pretrained_gguf, quant choice, handoff to ollama) are half of this skill.
- prompt: "Turn a base Qwen into a reasoning model with GRPO on one GPU using reward functions."
why: GRPO/reasoning FT via GRPOTrainer + fast_inference vLLM, the low-VRAM RL path this skill covers.
- prompt: "Vull ajustar un model a una sola GPU sense quedar-me sense VRAM amb Unsloth."
why: Catalan; single-GPU low-VRAM fine-tuning with Unsloth by name — the exact trigger.
should_not_trigger:
- prompt: "Should I fine-tune, use RAG, or just prompt-engineer for my support bot? And SFT vs DPO?"
route_to: finetuning
why: The whether/why/which-method decision is the method layer, not the backend that makes it run.
- prompt: "How do I format my conversations into a JSONL messages dataset for fine-tuning?"
route_to: training-data
why: Dataset construction/quality; this skill assumes the dataset already exists.
- prompt: "I exported a GGUF — how do I run it locally, write a Modelfile, and size VRAM in Ollama?"
route_to: ollama
why: Running the exported artifact locally is Ollama's job; Unsloth stops at producing the file.
- prompt: "Serve my merged 16-bit model at high throughput with tensor parallelism and continuous batching."
route_to: vllm
why: Serving-engine flags/throughput are the serving skill, not the training backend.
- prompt: "Which open model and license should I pick as my base — Llama vs Qwen vs Gemma?"
route_to: open-weights
why: Base-model selection + license tradeoffs are the choosing layer, not the fine-tuning backend.
capability:
- scenario: "I have one RTX 4090 (24GB). Fine-tune Llama-3.1-8B-Instruct on my chat JSONL with Unsloth so loss is only on the assistant turns, then export a Q4_K_M GGUF to run in Ollama. It keeps OOMing."
must_include:
- "FastLanguageModel.from_pretrained with load_in_4bit=True and max_seq_length, using a pre-quantized unsloth/*-bnb-4bit base id (verify the current id)."
- "get_peft_model with r/lora_alpha/target_modules and use_gradient_checkpointing='unsloth'; plus VRAM levers for OOM (batch size + gradient_accumulation_steps, optim='adamw_8bit')."
- "get_chat_template for the correct Llama-3.1 template AND train_on_responses_only with matching instruction_part/response_part so loss is response-only — with a decode/label sanity check."
- "Export via save_pretrained_gguf(quantization_method='q4_k_m') and the note that running it is an ollama job (create from the GGUF, run); merge to 16bit for vLLM instead."
- "The honesty hedge: the ~2x faster / ~70% less VRAM headline is model/GPU/config-specific vs a HF+FA2 QLoRA baseline — verify current numbers at docs.unsloth.ai and don't quote one number as gospel."
- "The single-GPU guardrail (OSS core is single-GPU; multi-GPU/multi-node is the paid/verify-live tier) and the boundary that method choice is finetuning, not this."
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |