ericrisco/rsc-harness / skills/open-weights
open-weights
VerifiedUse when choosing an open-weight LLM and clearing it for use — which family and size fit the task, the hardware and the budget, and above all whether the license permits shipping. Owns the license-class map (OSI-open versus custom-community versus non-commercial), the always-verify-the-model-card rule, size-to-VRAM budgeting and the quant formats. This layer decides WHICH model and whether the license allows it, then hands off. NOT downloading or hosting on the Hub (that is `huggingface`), NOT running it locally (that is `ollama`), NOT serving at throughput (that is `vllm`), NOT fine-tuning it (that is `finetuning`).
Open weights — pick the model, then clear the license
This is the CHOICE layer. Given a task, a box, and a shipping constraint, it tells you which open model to reach for and whether the license lets you ship it. It does not download, run, serve, or fine-tune anything — it feeds the skills that do (see Related skills).
The one rule (read before you name a model)
Model names and their licenses change monthly. Never state a license from memory — open the model card and confirm it, every time. This file was authored against cards live in mid-2026; by the time you read it, versions have shipped and terms have moved. So:
- Treat every specific version/license line below as [verify the card], not as settled fact.
- State the license class (OSI-open vs custom-community vs restricted), then send the reader to
the actual
LICENSE/ terms page for the exact model + size they intend to ship. - The decision framework and the class taxonomy are the durable parts. The version list is the perishable part. Weight your trust accordingly.
Real, verified-mid-2026 examples of why this rule exists — all confirmed against the source, and all things that were different at my Jan-2026 cutoff:
- Gemma 4 moved to Apache-2.0, while Gemma 1–3 stay on the custom Gemma Terms of Use (confirmed on ai.google.dev/gemma/terms, accessed 2026-07-18). Same brand, opposite license class depending on version.
- Codestral 2 was relicensed Apache-2.0 (per Mistral, Apr 2026), while the original Codestral stays non-production (MNPL). Same name, opposite shippability depending on release.
- DeepSeek-R1 is MIT, but the original DeepSeek-V3 weights ship under a custom DeepSeek License Agreement with OpenRAIL-style use restrictions (confirmed on the V3 LICENSE-MODEL), not MIT. Same family, different license per model.
If three of the load-bearing facts moved in one release cycle, yours have too. Check the card.
1. How to choose an open model
Five gates, in this order. The license gate can veto everything above it, so never fall in love with a model before you clear it.
- Task fit — match the model's design to the job:
- General chat / instruct → Llama, Qwen, Mistral, Gemma instruct variants.
- Coding → Qwen-Coder, Codestral (license!), DeepSeek-Coder, gpt-oss.
- Reasoning / math → DeepSeek-R1-style reasoning models, gpt-oss, Qwen reasoning variants.
- Multilingual → Qwen and Gemma tend to lead; check the card's language list.
- Multimodal (vision) → Llama vision, Qwen-VL, Gemma vision, PaliGemma, Phi multimodal.
- Embeddings / retrieval → a dedicated embedding model, not a chat LLM →
embeddings-search. - Small / edge → 0.5B–4B (Qwen small, Gemma small, Phi-mini, gpt-oss-20b class).
- Size vs your hardware class — can it even fit? See §4. A 70B at fp16 is ~140 GB; if you have one 24 GB GPU, that model is off the table unless you quantize and accept the quality/throughput cost. Pick a size your box can hold before you compare quality.
- License / shippability — the gate that matters most (see §3). Decide up front: is this a hobby/internal use, or a commercial product you distribute? "Open weights" does not mean "open source" and does not guarantee commercial rights. Verify the card.
- Hardware target — consumer GPU (8–24 GB), data-center GPU (A100/H100 80 GB), Apple unified memory, or CPU/edge. This narrows both size and quant format (GGUF for CPU/Mac/llama.cpp; AWQ/GPTQ for GPU serving).
- Community support — a proxy for "will this actually work Monday morning": are there ready-made GGUF/AWQ quants on the Hub, is it supported by your runtime (llama.cpp/Ollama/vLLM), are there fine-tunes and recent downloads? A model with no quants and no runtime support is a research artifact, not a shippable choice.
Rule of thumb: smallest model that passes your eval wins. Don't reach for 70B when a well-chosen
8B clears the bar — it's cheaper to run, faster, and fits more hardware. Prove it with agent-eval.
2. Model families
Families and typical size ladders below. Specific version claims are marked — [verify] at author time; the family and its rough sizing are the durable part. Full per-family notes, strengths, and license pointers in references/model-families.md.
| Family | Maker | Typical open sizes | Notes / current line [verify] |
|---|---|---|---|
| Llama | Meta | ~1B–~400B (dense + MoE) | Llama 4 (Scout/Maverick, MoE) current line; custom Community License, gated |
| Qwen | Alibaba | 0.5B–235B+ (dense + MoE), strong Coder/VL | mostly Apache-2.0 but per-size variance — check each card |
| Mistral / Mixtral | Mistral AI | 7B dense, 8x7B/8x22B MoE, "Small/Large" | open ones Apache-2.0; Codestral original = MNPL non-production |
| Gemma | ~1B–~27B, plus PaliGemma/CodeGemma/ShieldGemma | Gemma 1–3 = custom Gemma Terms; Gemma 4 = Apache-2.0 [verify] | |
| DeepSeek | DeepSeek | V3-class MoE (~600B+), R1 reasoning, distills | per-model license split — R1 MIT, original V3 custom; verify |
| Phi | Microsoft | ~3B–15B ("mini", reasoning, multimodal) | MIT across the Phi-4 family [verify] |
| gpt-oss | OpenAI | 20B and 120B (MoE, open weights) | Apache-2.0 [verify]; OpenAI's first open-weight LLMs since GPT-2 |
| Others | various | — | SmolLM, OLMo (fully-open incl. data), Falcon, Yi, Command — check card |
3. Licenses
This is the load-bearing section. Get it wrong and you ship something you have no right to ship. Every model falls into one of three classes. Identify the class, then open the card.
The three classes
| Class | Examples (class, not a promise) | What it means for shipping |
|---|---|---|
| OSI-open (Apache-2.0, MIT) | Qwen (most), Mistral open, Phi, gpt-oss, DeepSeek-R1, Gemma 4 | Commercial use, modify, redistribute — permissive. Still read the card for attribution/notice. |
| Custom / community | Llama (Meta Community), Gemma 1–3 (Gemma Terms) | Broad free use but with conditions: acceptable-use policy, attribution, sometimes a scale cap. NOT OSI-open. |
| Non-commercial / restricted | original Codestral (MNPL), any "research-only", some RAIL, some "≥$X revenue → buy a license" | You cannot ship it in a commercial product without a separate license. Fatal if missed. |
"Open weights" describes availability of the weights file — it says nothing about your legal rights. A model can be a free download and still be non-commercial. The download button is not a license.
Per-family license notes (state the class, then verify the exact card)
- Llama = Meta Llama Community License — NOT OSI-open. Carries an Acceptable Use Policy, a "Built with Llama" attribution requirement on derivatives/products, naming rules, and a >700M-monthly-active-users clause (above that you must request a separate license from Meta, granted at Meta's discretion). Gated on the Hub (accept terms first). Verify the current Llama version's terms — Meta has revised them across releases.
- Gemma = custom Gemma Terms of Use + Prohibited Use Policy for Gemma 1–3 (not Apache), with a duty to pass the restrictions downstream to every user and to ship the terms/notice file. Gemma 4 reportedly moved to Apache-2.0 — a textbook reason to check the version's card.
- Qwen = mostly Apache-2.0, but with per-size variance. Historically some sizes (often the very largest or a special tier) carried a separate Qwen license instead of Apache. Check each size's card — do not assume "Qwen = Apache" for the whole family.
- Mistral / Mixtral open models = Apache-2.0, but the original Codestral is MNPL (non-production) — free for research/eval, not for commercial deployment. Codestral 2 was relicensed Apache-2.0; verify which Codestral release you actually have. Note some newer Mistral models use a "Modified MIT" with a revenue-threshold commercial clause — verify.
- DeepSeek = per-model split. Code repos are MIT; DeepSeek-R1 weights are MIT; but the original DeepSeek-V3 weights ship under a custom DeepSeek License Agreement with OpenRAIL- style use-based restrictions. "DeepSeek = MIT" is too simple — verify the specific model.
- Phi = MIT across the Phi-4 family [verify] — genuinely permissive, commercial-friendly.
- gpt-oss (OpenAI open-weight models, 20B/120B) = Apache-2.0 [verify] — permissive.
Commercial-use checklist (run before you ship)
- Opened the exact model + size card and read its
LICENSE/ terms — not a blog, not this file. - Confirmed the class: OSI-open, custom-community, or restricted/non-commercial.
- If custom/community: identified the conditions (attribution string, acceptable-use policy, MAU/revenue caps, downstream pass-through duty) and can meet them.
- If restricted: confirmed you are not deploying commercially, or obtained a separate license.
- Checked whether the model is gated (accept terms on the Hub before download).
- Checked the base model license — a fine-tune inherits the base's obligations (a Llama fine-tune still owes "Built with Llama"; a distill can inherit the teacher's terms).
Deeper class breakdown, gated-model mechanics, and the fine-tune-inheritance trap: references/licenses.md.
4. Size ↔ VRAM
A selection-time budget: will this size class even fit your hardware? (Operational per-request
KV-cache math lives in ollama/vllm — this is the "before I pick it" estimate.)
weights_GB ≈ params(B) × bytes_per_param # then add KV cache + runtime overhead on top
fp16/bf16 → 2.0 (≈ 2 GB per 1B params)
8-bit → ~1.0 (≈ 1 GB per 1B params)
4-bit → ~0.6 (≈ 0.6 GB per 1B + cache + overhead; conservative ~0.5–0.6/param)
What that means per hardware class (weights only — leave headroom for context/KV cache):
| Hardware | fp16 ceiling | Practical pick (4-bit) |
|---|---|---|
| 8 GB consumer GPU | ~3B | 7–8B at 4-bit (~5 GB) |
| 16 GB | ~7B | 13–14B at 4-bit |
| 24 GB (e.g. 3090/4090) | ~10B | ~32B at 4-bit (~20 GB) |
| 80 GB (A100/H100) | ~34B | 70B at 4-bit (~40 GB), or a big-MoE like gpt-oss-120b |
| Apple unified (e.g. 64 GB) | shares with the OS | budget against total unified memory |
| multi-GPU / 2×80 GB+ | 70B fp16 (~140 GB) via tensor-parallel | full-precision large models |
A 70B at fp16 is ~140 GB — it does not fit one GPU. You either quantize (4-bit ≈ ~40 GB, fits one
80 GB card) or split across GPUs with tensor parallelism (that's a vllm job). Longer context
grows the KV cache on top of weights, so keep a margin. Full derivation + KV math:
references/sizing-and-quant.md.
5. Quant / formats
Quantization shrinks weights (fewer bits/param) to fit smaller hardware, trading a little quality. Which format you pick is driven by your runtime:
| Format | Where it runs | Use it for |
|---|---|---|
| GGUF | llama.cpp, Ollama, LM Studio | CPU, Apple Silicon, single-box local; the everyday local format |
| AWQ | vLLM, TGI, transformers | GPU serving — activation-aware 4-bit, strong quality/latency |
| GPTQ | vLLM, TGI, transformers | GPU serving — older, widely available 4-bit |
| bitsandbytes (NF4) | transformers | quick load-time 4/8-bit for experiments/fine-tuning |
| EXL2 | ExLlamaV2 | flexible bit-rates on consumer GPUs |
Decoding a GGUF quant tag like Q4_K_M: Q4 = ~4-bit weights; _K = a k-quant (mixed
precision — keeps the more sensitive tensors at higher bit-depth); _M = the medium size/quality
tier (_S smaller/lower, _L larger/higher). Q4_K_M is the standard default — roughly half
the memory of fp16 for a few percent quality loss. Q8_0 is near-lossless (~1 byte/param); below
Q4, quality drops off fast. Don't go sub-Q4 to force a too-big model onto a too-small box — pick a
smaller model instead. More in references/sizing-and-quant.md.
6. Where to get it + how to run it (routing)
You've chosen a model and cleared its license. Now hand off — this skill stops here.
- Get the weights / host on HF →
huggingface(Hub download/upload, Inference Providers, endpoints). Also where you convert to GGUF. - Run it locally on one box (laptop/desktop, GGUF via Ollama) →
ollama. - Serve it at throughput / with tensor-parallel (production, batching, big models) →
vllm. - Fine-tune it (LoRA/QLoRA, SFT, preference tuning) →
finetuning. - Prove the small model is enough before committing →
agent-eval.
The Hub filters (task + license + size + recent downloads) and the Ollama library are where you
actually find candidates — that mechanics lives in huggingface / ollama, not here.
Guardrails / gotchas
- Never assert a license from memory. State the class, open the card. Licenses change per version, per size, and per release month.
- "Open weights" ≠ "open source" ≠ "commercial-use OK". Three different things.
- A fine-tune inherits its base license. Fine-tuning Llama does not launder away "Built with Llama" or the acceptable-use policy; a distill can inherit the teacher model's terms.
- Gated ≠ restricted. Gemma/Llama are gated (accept terms to download) yet usable commercially under their conditions; a "research-only" card is restricted regardless of gating.
- Don't sub-Q4 a too-big model onto a too-small GPU. Quality collapses and it still may OOM at real context. Choose a size that fits.
- Newest ≠ best for you. The smallest model that passes your eval wins on cost, speed, and fit.
- MoE total vs active params. A "120B" MoE may activate only ~5B/token (fast) but you still need VRAM for all experts — size against total params, not active.
Related skills
huggingface— get/host: Hub download/upload, Inference Providers, endpoints, GGUF conversion. This skill tells you which repo to pull;huggingfacepulls it.ollama— run one model locally (GGUF, single box, VRAM sizing at request time). This skill says which model + quant class fits;ollamaruns it and does the KV-cache math.vllm— serve at throughput, batching, tensor-parallel for models too big for one GPU.finetuning— adapt a chosen base with LoRA/QLoRA/SFT. Choose+clear the base here first.- Boundary: this skill is selection + licensing knowledge only. It never re-teaches calling, running, serving, or training — it feeds those four.
Checklist
- Named the task and matched a family designed for it (chat/code/reasoning/vision/embeddings/edge).
- Picked the smallest size likely to pass the eval, and confirmed it fits the hardware class.
- Identified the license class (OSI-open / custom-community / restricted).
- Opened the exact model + size card and confirmed the license — not from memory, not from a blog.
- For commercial use: met the conditions (attribution, acceptable-use, MAU/revenue caps) or confirmed permissive.
- Checked the base model license if it's a fine-tune/distill.
- Picked a quant format matching the runtime (GGUF local / AWQ-GPTQ serving) and a sane tier (
Q4_K_Mdefault). - Handed off:
huggingface(get),ollama(run local),vllm(serve),finetuning(adapt).
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 193k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 193k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 193k | Updated | Claude CodeCodex | Verified | Install |