Skip to content

ericrisco/rsc-harness / skills/finetuning

finetuning

Verified

Use when adapting an open-weight model to a target form or behavior — tone, output format, reasoning pattern — via LoRA/QLoRA or full fine-tuning with TRL SFTTrainer, then preference optimization (DPO/ORPO/KTO/GRPO), and for fine-tune vs prompt vs RAG. NOT adding facts to a model (that is rag); NOT the single-GPU Unsloth backend or GGUF export (that is unsloth).

FileKindSize
SKILL.mdskill md14.8 KB
evals/README.mdmarkdown1.0 KB
evals/cases.yamlconfig4.6 KB
references/hyperparameters-and-eval.mdmarkdown4.6 KB
references/methods.mdmarkdown8.4 KB

evals/README.md

View on GitHub
# finetuning evals — how to run

Routing and capability checks, not unit tests. Run the `should_trigger` prompts against your skill
router and confirm `finetuning` fires (form/behavior adaptation, method choice, pipeline, and the
overfit/forgetting/target_modules diagnostics). Run `should_not_trigger` and confirm the router
defers to the named sibling instead (`rag` for facts, `prompt-engineering` for the cheaper lever,
`open-weights` for base-model choice, `training-data` for corpus building, `vllm` for serving,
`unsloth` for the fast single-GPU backend) — a miss there means the boundary in the description
needs sharpening. For the `capability` case, hand the scenario to an agent with this skill loaded
and grade the answer against each `must_include` bullet pass/fail: the decision gate (form vs
facts), QLoRA + TRL/PEFT current API, the alpha≈2r and LR/epoch hyperparameters, the chat-template
guardrail, LIMA quality-over-quantity, held-out eval watching eval loss, KTO for unpaired feedback,
and the version/license hedge.
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall