Skip to content

ericrisco/rsc-harness / skills/training-data

training-data

Verified

Use when assembling or curating the corpus a fine-tune trains on — turning raw examples into the JSONL shape a trainer expects (instruction, conversational, preference-pair or binary-label), matching the data to the model's chat template, generating synthetic or distilled examples, deduplicating and decontaminating against eval sets, and quality-filtering. NOT cleaning tabular rows, nulls and dtypes (that is `data-cleaning`), NOT building a retrieval corpus of chunks and embeddings (that is `embeddings-search`), NOT running the trainer or picking hyperparameters (that is `finetuning`).

FileKindSize
SKILL.mdskill md14.4 KB
evals/README.mdmarkdown1.0 KB
evals/cases.yamlconfig3.5 KB
references/formats.mdmarkdown5.1 KB
references/synthesis-dedup-quality.mdmarkdown6.4 KB

references/formats.md

View on GitHub
# Dataset formats — full field matrix & conversions

Grounded in the TRL "Dataset formats and types" doc (tagged `v1.8.0` at author time). TRL and
`datasets` move monthly — verify field names against the current TRL docs before you rely on
them. Field names below are verbatim from that doc.

## The two axes

- **Format** — `standard` (plain-text string values) vs `conversational` (values are lists of
  `{"role", "content"}` messages). Conversational is the default for chat models.
- **Type** — the task, which sets the columns.

| Type | Standard | Conversational |
| --- | --- | --- |
| Language modeling | `{"text": "..."}` | `{"messages": [{"role","content"}, ...]}` |
| Prompt-only | `{"prompt": "..."}` | `{"prompt": [ ... ]}` |
| Prompt-completion | `{"prompt", "completion"}` | `{"prompt": [...], "completion": [...]}` |
| Preference (explicit) | `{"prompt", "chosen", "rejected"}` | same keys, each a message list |
| Preference (implicit) | `{"chosen", "rejected"}` (no `prompt`) | same, each a full conversation |
| Unpaired preference (KTO) | `{"prompt", "completion", "label"}` | same, `label` a bool |
| Stepwise supervision | `{"prompt", "completions": [...], "labels": [bool,...]}` | — |

`label` / `labels` are JSON booleans (`true`/`false`).

## Which trainer wants which type

| Trainer | Type |
| --- | --- |
| `SFTTrainer` | language modeling **or** prompt-completion |
| `DPOTrainer` | preference (explicit prompt recommended) |
| `KTOTrainer` | unpaired preference, or preference |
| `RewardTrainer` | preference (implicit prompt recommended) |
| `GRPOTrainer`, `RLOOTrainer`, `OnlineDPO/Nash/XPO` | prompt-only |
| `ORPOTrainer`, `CPOTrainer`, `BCOTrainer` | preference (explicit) / unpaired |
| `PRMTrainer` | stepwise supervision |
| `PPOTrainer` | tokenized language modeling |

Experimental trainers live under `trl.experimental.*` — confirm the import path in the current
release.

## Alpaca instruction format (classic, not native TRL)

```json
{"instruction": "Summarize the paragraph.", "input": "<long text>", "output": "<summary>"}
```

`input` is optional (empty string when the instruction is self-contained). TRL does not train on
these keys directly. Convert:

**To `messages` (preferred for chat models):**

```python
def alpaca_to_messages(ex):
    user = ex["instruction"] + (("\n\n" + ex["input"]) if ex.get("input") else "")
    return {"messages": [{"role": "user", "content": user},
                         {"role": "assistant", "content": ex["output"]}]}
```

**To a single `text` string via a prompt template (Unsloth-style) — append EOS:**

```python
TEMPLATE = ("### Instruction:\n{instruction}\n\n### Input:\n{input}\n\n### Response:\n{output}")
def alpaca_to_text(ex, eos):
    return {"text": TEMPLATE.format(**ex) + eos}   # eos = tokenizer.eos_token — REQUIRED
```

Without the EOS token the model never learns to stop generating. When formatting to `text` you
are also bypassing the chat template, so this path is for base/completion-style training; for
chat models prefer `messages` + `apply_chat_template`.

## Tool-calling data

Add a `tools` column (list of JSON-schema function specs); assistant turns carry `tool_calls`
instead of `content`, and tool results come back as a `{"role": "tool", ...}` message.

```python
from transformers.utils import get_json_schema   # build the schema from a Python signature
schema = get_json_schema(my_function)
row = {"messages": [...], "tools": [schema]}
```

On `datasets` ≥4.7 use the `Json()` feature type for arbitrary tool-arg objects
(`Dataset.from_list(data, on_mixed_types="use_json")`); on older `datasets` store `tools` as a
`json.dumps(...)` string.

## Vision (VLM) data

Add an `images` key (list of PIL images per row) or `image` (single), and make each message
`content` a list of typed parts:

```python
"content": [{"type": "image"}, {"type": "text", "text": "What is in the image?"}]
```

Mixing text-only and image rows in one dataset needs `transformers` ≥ 4.57.

## Type→type conversions (with `datasets`)

- **prompt-completion → language-modeling**: concatenate `prompt` + `completion` into `text`.
- **preference → prompt-completion**: drop `rejected`, rename `chosen` → `completion`.
- **preference → prompt-only**: drop `chosen` and `rejected`.
- **implicit → explicit preference**: `trl.extract_prompt` pulls the shared prefix into `prompt`.
- **preference → unpaired (KTO)**: `trl.unpair_preference_dataset` — **only valid if every
  `chosen` is genuinely good and every `rejected` genuinely bad** (KTO's label is absolute
  good/bad, not relative). If your pairs are "better vs worse" but both acceptable, unpairing
  mislabels them.
- **stepwise → unpaired**: join `completions`, merge `labels` (e.g. logical AND).

Do conversions **before** `apply_chat_template`, on the structured `messages`, so they behave
consistently across formats.

## Sanity checks worth automating

- Every SFT conversational row ends on an `assistant` turn.
- Roles alternate sensibly; no empty `content`.
- Preference `chosen != rejected`.
- KTO `label` present and boolean; class balance is not 100/0.
- The whole file round-trips through `apply_chat_template` for the *target* model with no raise.
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall