Skip to content

ericrisco/rsc-harness / skills/prompt-engineering

prompt-engineering

Verified

Use when one prompt must give the same right answer across reruns, models, and pasted-in hostile input: forcing a fixed schema, picking the few-shot set, ordering the prompt blocks, or the inline cases you run while tuning. NOT the agent loop, tools, or retrieval (that is `building-agents`), NOT a standing CI eval harness (that is `agent-eval`).

FileKindSize
SKILL.mdskill md9.3 KB
evals/README.mdmarkdown806 B
evals/cases.yamlconfig4.1 KB
references/eval-templates.mdmarkdown3.6 KB
references/output-contracts.mdmarkdown4.4 KB
scripts/verify.shexecutablescript3.5 KB

references/output-contracts.md

View on GitHub
# Output contracts: per-provider strict recipes

Pick the strongest mechanism the provider gives you. Provider APIs and version notes
below verified 2026-06-02 (access date); each external figure carries its own source
date inline — do not read the access banner as the date a cited number was published.

## OpenAI — strict `json_schema` via `response_format`

Highest reliability: the provider compiles the schema into a token-masking FSM, so the
output is guaranteed to parse against the schema (**<0.1% schema-failure rate** — figure
published by OpenAI at the Aug 2024 Structured Outputs launch, not a 2026 number). Set
`strict: true` and mark every field required (use unions/nullable for optional).

```python
from openai import OpenAI
from pydantic import BaseModel

class Triage(BaseModel):
    category: str  # "bug" | "billing" | "other"

client = OpenAI()
resp = client.chat.completions.parse(
    model="gpt-…",                       # resolve the model id from config, not here
    messages=[{"role": "user", "content": prompt}],
    response_format=Triage,              # SDK emits strict json_schema under the hood
)
result = resp.choices[0].message.parsed  # already a Triage, or None on refusal
```

Raw form (no SDK helper):

```json
{
  "type": "json_schema",
  "json_schema": {
    "name": "triage",
    "strict": true,
    "schema": {
      "type": "object",
      "properties": { "category": { "type": "string", "enum": ["bug", "billing", "other"] } },
      "required": ["category"],
      "additionalProperties": false
    }
  }
}
```

## Anthropic — strict tool use

Reliable schema adherence, close behind OpenAI strict. Define a single tool whose
`input_schema` is your contract and force it. **Assistant-prefill structured output is
dead in the latest models** — Anthropic's migration guide states prefill returns a `400`
on Sonnet 4.6 / Opus 4.6 / Opus 4.7 and routes you to structured outputs / `output_config.format`
(platform.claude.com/docs/en/about-claude/models/migration-guide). Do not use prefill to
coerce a shape.

```python
import anthropic
client = anthropic.Anthropic()
msg = client.messages.create(
    model="claude-…",                    # from config
    max_tokens=256,
    tools=[{
        "name": "emit_triage",
        "description": "Return the ticket category.",
        "input_schema": {
            "type": "object",
            "properties": {"category": {"type": "string", "enum": ["bug", "billing", "other"]}},
            "required": ["category"],
        },
    }],
    tool_choice={"type": "tool", "name": "emit_triage"},
    messages=[{"role": "user", "content": prompt}],
)
block = next(b for b in msg.content if b.type == "tool_use")
result = block.input  # {"category": "..."}
```

**Streaming caveat:** Anthropic tool-use output arrives as ONE block at end of stream —
you cannot read fields progressively. OpenAI/Gemini stream field-by-field. If your UI
parses-as-it-streams, that path does not exist on Claude tool use.

## Google Gemini — `responseSchema`

Set `response_mime_type: "application/json"` and a `response_schema`. Validate after,
same as any JSON path.

```json
{ "generationConfig": { "response_mime_type": "application/json", "response_schema": { "type": "OBJECT", "properties": { "category": { "type": "STRING", "enum": ["bug","billing","other"] } } } } }
```

## TypeScript schema surface (zod)

```typescript
import { z } from "zod";
const Triage = z.object({ category: z.enum(["bug", "billing", "other"]) });
// OpenAI: zodResponseFormat(Triage, "triage"); Anthropic: zodToJsonSchema(Triage) as input_schema.
const parsed = Triage.parse(JSON.parse(raw));   // throws -> trigger retry
```

## Retry-on-parse-fail (for JSON mode / freeform only)

Strict mechanisms above rarely need this. When you only have JSON mode or freeform,
bound the retries and feed the parser error back:

```python
def call_with_retry(prompt, schema, parse, max_tries=2):
    last_err = None
    for _ in range(max_tries + 1):
        raw = call_model(prompt if last_err is None
                         else f"{prompt}\n\nYour previous output failed: {last_err}. Return only valid output.")
        try:
            return parse(raw, schema)          # raises on invalid
        except Exception as e:
            last_err = str(e)
    raise ValueError(f"contract not met after retries: {last_err}")
```

Keep `max_tries` small (1-2). If a strict mechanism is available, use it instead of
spending tokens on retries.
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall