Skip to content

ericrisco/rsc-harness / skills/structured-extraction

structured-extraction

Unverified

Use when text must become a typed, schema-conformant object you can trust — pulling fields into a fixed JSON shape, extracting line items as typed records, classifying into enums, and building the Pydantic or Zod model plus the validate-and-retry loop. Covers extractors that throw parse errors, leak markdown fences, or fabricate a value where the field is absent instead of returning null. NOT getting the text out of a PDF, scan or DOCX first (that is `document-processing`), NOT general prompt craft untied to a schema (that is `prompt-engineering`).

Structured extraction — text in, a typed object you can trust out

The deliverable is a typed object that conforms to a schema you defined — not prose, not "roughly JSON." The whole skill rests on one distinction the rest of the file keeps returning to:

Native structured outputs make the JSON valid and typed. They never make the values correct.

Constrained decoding guarantees the model cannot emit a token that breaks your schema, so JSON.parse errors, missing keys, wrong types, and stray markdown fences disappear at the source. It does nothing to stop the model from putting a plausible-but-wrong email in a string field, snapping a fuzzy category to the wrong enum, or coercing "$1,200" into 1200.0 when the currency mattered. Owning both halves — the shape (decoding) and the values (validation) — is this skill. If you only do the first half you ship a database full of well-typed lies.

Boundary test (bytes vs. schema). If the input is a PDF, scan, DOCX, or HTML and the deliverable is the raw text/Markdown/cells of that document, that is upstream: document-processing produces the text, this skill turns that text into typed fields. If you're holding text and want it shaped, you're in the right place.

Current as of 2026-06-02: OpenAI Structured Outputs (strict: true json_schema), Anthropic Structured Outputs (GA since the 2025-11-14 public beta; output_config.format), and Instructor (built on Pydantic, ~3M downloads/month). Exact request/response shapes and the per-provider limit tables live in references/providers.md so this file stays lean.

Default: native constrained decoding when the provider has it

If the model and provider support native structured outputs, use them. This is not a tuning knob — it is the difference between ~100% schema conformance and hoping a regex catches the fence.

Bad — prompt-and-pray, then parse raw text:

resp = client.chat.completions.create(
    model="gpt-5.1",
    messages=[{"role": "user", "content": f"Return JSON with name and email:\n{text}"}],
)
data = json.loads(resp.choices[0].message.content)  # markdown fence / preamble / missing key -> crash

Good — OpenAI strict json_schema (Chat Completions):

resp = client.chat.completions.create(
    model="gpt-5.1",
    messages=[{"role": "user", "content": text}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "contact",
            "strict": True,
            "schema": {
                "type": "object",
                "additionalProperties": False,
                "required": ["name", "email"],
                "properties": {
                    "name": {"type": "string", "description": "Full name as written."},
                    "email": {"type": ["string", "null"],
                              "description": "Email exactly as written, or null if none is stated."},
                },
            },
        },
    },
)
data = json.loads(resp.choices[0].message.content)  # now guaranteed valid + typed

On the OpenAI Responses API the same block moves under text.format instead of response_format. On Anthropic, the equivalent is output_config={"format": {"type": "json_schema", "schema": {...}}} on Claude Opus 4.5–4.8 / Sonnet 4.5+ / Haiku 4.5; Anthropic compiles your schema into a grammar and caches it for 24h, and the SDKs ship helpers (client.messages.parse(...) in Python, zodOutputFormat(schema) in TS). The older deprecated output_format param and the deprecated structured-outputs-2025-11-13 beta header still work in a transition window — do not write new code against them. Full shapes in references/providers.md.

The non-negotiable strict-schema rule (OpenAI and Anthropic both):

  • every object sets "additionalProperties": false;
  • every property is listed in required;
  • "optional" is expressed as a union with null ("type": ["string", "null"]), never by leaving the field out of required. Omitting it is the single most common strict-mode error.

Schema design rules (each prevents a specific failure)

You wantExpress it asBecause
A field that may be absentnullable union ["string","null"] + description: "...or null if not stated"A non-null type forces a value, so the model fabricates one. Nullable + instruction yields null instead.
A closed set of categoriesenum: ["open","pending","closed"]Free-text string drifts ("Open", "in progress", "closd"); an enum makes drift impossible to emit.
Many items of one kindone object schema + a top-level {"items": {"type":"array","items": <object>}} wrapperOne object per extraction unit keeps each record independently validatable; arrays of scalars lose field structure.
The model to read your intenta description on every propertyThe model reads field descriptions at decode time; "amount in cents, no currency symbol" beats a bare integer.
A number in a range / a regex / a length capleave it out of the schema; enforce in a post-decode validatorStrict modes reject or silently ignore minimum/maximum/minLength/maxLength/complex regex — see the unsupported-features table in references.
A deeply nested or recursive shapeflatten it, or split into two extractionsNative modes reject recursion and cap nesting/complexity; flat schemas decode reliably.

Keep schemas flat and shallow. If you find yourself nesting four levels deep or describing a tree, that is two extractions, not one heroic schema.

The reliability ladder — escalate only on failure, and cap it

Climb from the cheapest mechanism upward. Each rung catches what the rung below cannot; you stop at the first rung that holds for your data.

RungMechanismCatchesDoes NOT catchWhen you stop here
1Native constrained decodingparse errors, wrong types, missing keys, fenceswrong values, bad units, wrong enumshape+types only, latest single provider
2Pydantic / Zod validation after decodeout-of-range, bad format, cross-field contradictions, null-vs-absentnothing the model genuinely got wrongvalue rules you can express as code
3Bounded reask (Instructor or hand-rolled)semantic errors the model can fix when shown the validation messagesystematic model blind spotsresidual errors; cap retries (e.g. 2) and log every reask
4Human / log revieweverything still wrong after 3—high-stakes fields or low-confidence rows

Rung 1 is mandatory when available. Rung 2 is mandatory the moment any field has a value rule (a range, a format, a "must match the order date") — because rung 1 structurally cannot enforce values. Rungs 3 and 4 are opt-in. Never make rung 3 unbounded: a retry loop with no cap turns one bad document into an unbounded bill.

Value validation the schema can't enforce

This is the half native decoding leaves on the table. Validate values after you have a typed object.

Pydantic — value rules + normalization the schema can't carry:

from pydantic import BaseModel, field_validator

class Order(BaseModel):
    amount_cents: int
    discount_pct: float | None  # nullable: may be absent
    order_date: str             # we'll normalize to ISO

    @field_validator("discount_pct")
    @classmethod
    def pct_in_range(cls, v):
        if v is not None and not (0 <= v <= 100):
            raise ValueError("discount_pct must be between 0 and 100")
        return v

    @field_validator("amount_cents", mode="before")
    @classmethod
    def strip_currency(cls, v):
        if isinstance(v, str):  # "$1,200.00" -> 120000
            return int(round(float(v.replace("$", "").replace(",", "")) * 100))
        return v

The Zod equivalent uses .refine() for cross-field and range checks and .transform() for normalization. Three normalizations bite constantly: currency ("$1,200" vs 1200 vs 120000 cents — pick one and enforce it), dates (free text → ISO 8601, and decide what a missing year means), and enum snapping (the model rounds "kinda urgent" to urgent; validate that the snap was legitimate, or widen the enum).

Null vs. absent. A nullable field with a clear instruction is the entire fix for "the model invents an email." "email": {"type": ["string","null"], "description": "...or null if the text states no email"} plus a one-line system instruction ("use null for any field not present in the source; never guess"). If you make the field non-nullable, you have told the model to produce a value — it will.

Bounded reask with Instructor — failed validation is fed back to the model as an error message:

import instructor

client = instructor.from_provider("openai/gpt-5.1")
order = client.chat.completions.create(
    response_model=Order,            # your Pydantic model, validators and all
    max_retries=2,                   # BOUND it; each retry is another paid call
    messages=[{"role": "user", "content": text}],
)

On a validation failure Instructor reasks with the ValueError text, so @field_validator rules the model never saw in the schema still get enforced through the loop. Log every reask (count + reason): a quietly climbing reask rate is your early signal that a field's instruction or schema is wrong.

Multi-provider: Instructor from_provider

When you want one Pydantic model to run across OpenAI, Anthropic, and local backends without rewriting per SDK, use Instructor's unified entrypoint:

client = instructor.from_provider("anthropic/claude-opus-4-8")  # or "openai/gpt-5.1", "ollama/llama3.3"

Reach for Instructor when you need provider portability or value-level validation with reask. Reach for the native SDK helper (client.messages.parse, zodOutputFormat) when you're on one provider and want the simplest path with the fewest dependencies. Both sit on the same native decoding underneath.

Scale and accuracy live next door

This skill is the single extraction node and its per-call validation loop. Two concerns are explicitly not here:

  • Running 10k documents — batching, idempotency, retries-across-calls, cost/latency budgeting, multi-step chains → llm-pipeline.
  • Measuring extraction quality offline — a golden set, precision/recall, a regression gate that fails CI when accuracy drops → agent-eval. This skill builds the extractor; that one scores it.
  • Answering questions over a corpus by retrieving chunks → rag.
  • Post-extraction tabular cleanup (dedupe rows, coerce columns, normalize categories across a whole dataset) → data-cleaning.
  • Improving a prompt that isn't bound to a schema → prompt-engineering.

(Some routed siblings may not be built in this collection yet; the routing decision still holds.)

Anti-patterns

BadWhy it bitesGood
json.loads(resp.text) on raw model outputmarkdown fence, chatty preamble, or a missing key crashes at runtimenative structured outputs; parse only a decoder-guaranteed string
Stripping ```json fences with a regextreats the symptom; the model can still drop a key or change a typeturn on native decoding — the fence never appears
"type": "string" on a field that's often absentforces a value, so the model fabricates a plausible wrong onenullable union ["string","null"] + "null if not stated"
Omitting an optional field from required (strict mode)OpenAI/Anthropic strict reject it — every property must be in requiredkeep it in required, make its type a union with null
minimum/maxLength/lookahead-regex inside a strict schemarejected or silently ignored — the constraint does nothingleave value rules out of the schema; enforce in a Pydantic/Zod validator
max_retries unbounded (or a while reask loop)one bad doc becomes an unbounded bill and a hung jobcap at 2–3, log each reask, route the rest to review
Deep/recursive schema in one callnative modes reject recursion and cap complexity → compile failureflatten, or split into multiple extractions
Trusting decoding to make values correctvalid+typed ≠ true; you ship well-formed wrong dataadd the rung-2 validation step for every value rule
Building on Anthropic output_format / structured-outputs-2025-11-13 headerdeprecated transition-window APIuse output_config={"format": {...}}
One giant array of scalars for "many things"loses per-item field structure and per-item validationone object schema per unit, wrapped in a top-level items array

Checklist before you ship an extractor

  1. Native structured outputs ON (or a documented reason the provider has no native mode).
  2. Every object has additionalProperties: false; every property is in required.
  3. Every maybe-absent field is nullable with a "use null if not stated" instruction.
  4. Closed sets are enums; value rules (ranges/format/cross-field) live in validators, not the schema.
  5. A reask loop, if any, is bounded and logged.
  6. You can state, for each field, whether a wrong output would be caught by decoding (shape) or only by validation (value) — and you have the validation for the value cases.
canvas-designanthropics/skillsCreate beautiful visual art in .png and .pdf documents using design philosophy. You should use this skill when the user asks to create a poster, piece of art, design, or other static piece. Create original visual designs, never copying existing artists' work to avoid copyright violations.Stars: 196kUpdated Claude CodeCodexVerifiedInstall
docxanthropics/skillsUse this skill whenever the user wants to create, read, edit, or manipulate Word documents (.docx files) or Word templates (.dotx files). Triggers include: any mention of 'Word doc', 'word document', '.docx', '.dotx', or requests to produce professional documents with formatting like tables of contents, headings, page numbers, or letterheads. Also use when extracting or reorganizing content from .docx or .dotx files, inserting or replacing images in documents, performing find-and-replace in Word files, working with tracked changes or comments, or converting content into a polished Word document. If the user asks for a 'report', 'memo', 'letter', 'template', or similar deliverable as a Word or .docx file, use this skill. Do NOT use for PDFs, spreadsheets, Google Docs, or general coding tasks unrelated to document generation.Stars: 196kUpdated Claude CodeCodexVerifiedInstall
pdfanthropics/skillsUse this skill whenever the user wants to do anything with PDF files. This includes reading or extracting text/tables from PDFs, combining or merging multiple PDFs into one, splitting PDFs apart, rotating pages, adding watermarks, creating new PDFs, filling PDF forms, encrypting/decrypting PDFs, extracting images, and OCR on scanned PDFs to make them searchable. If the user mentions a .pdf file or asks to produce one, use this skill.Stars: 196kUpdated Claude CodeCodexVerifiedInstall
pptxanthropics/skillsUse this skill any time a .pptx or .potx file is involved in any way — as input, output, or both. This includes: creating slide decks, pitch decks, or presentations; reading, parsing, or extracting text from any .pptx or .potx file (even if the extracted content will be used elsewhere, like in an email or summary); editing, modifying, or updating existing presentations; combining or splitting slide files; working with templates (.potx), layouts, speaker notes, or comments. Trigger whenever the user mentions "deck," "slides," "presentation," or references a .pptx or .potx filename, regardless of what they plan to do with the content afterward. If a .pptx or .potx file needs to be opened, created, or touched, use this skill.Stars: 196kUpdated Claude CodeCodexVerifiedInstall