Skip to content

hashgraph-online/awesome-codex-plugins / plugins/alpoxdev/hypercore/skills/hyper-documentation

hyper-documentation

Flagged

[Hyper] Use this skill when the user asks to create, refactor, or update a documentation artifact: a project or package README.md, a repository AGENTS.md with its AGENTS.ko.md mirror, an AI-readable instruction base, runbook, specification, or rule pack, a reusable prompt, prompt template, prompt pack, or prompt eval fixture, or an agent's persistent memory files (MEMORY.md index, memory bank, lessons file) with their write, pruning, and deletion rules. It picks one mode from the requested output and grounds every claim in the project and its sources. Do not use for reusable skill folders, CHANGELOG, CONTRIBUTING, LICENSE, release notes, answer-only research, or implementation work; it never writes CLAUDE.md.

FileKindSize
SKILL.ko.mdmarkdown21.2 KB
SKILL.mdskill md19.3 KB
assets/evals/hyper-documentation-cases.jsonlother90.3 KB
assets/prompt/eval-harness.template.jsonconfig3.0 KB
assets/prompt/prompt-pack.template.ko.mdmarkdown2.1 KB
assets/prompt/prompt-pack.template.mdmarkdown1.9 KB
assets/prompt/source-ledger.template.ko.mdmarkdown2.1 KB
assets/prompt/source-ledger.template.mdmarkdown2.0 KB
references/memory/evidence-ledger.ko.mdmarkdown55.2 KB
references/memory/evidence-ledger.mdmarkdown51.1 KB
references/official/anthropic.ko.mdmarkdown12.7 KB
references/official/anthropic.mdmarkdown11.9 KB
references/official/openai.ko.mdmarkdown13.5 KB
references/official/openai.mdmarkdown12.8 KB
references/prompt/eval-harness-guide.ko.mdmarkdown5.8 KB
references/prompt/eval-harness-guide.mdmarkdown5.3 KB
references/prompt/prompt-pack-schema.ko.mdmarkdown3.9 KB
references/prompt/prompt-pack-schema.mdmarkdown3.6 KB
references/readme/readme-templates.ko.mdmarkdown4.4 KB
references/readme/readme-templates.mdmarkdown4.1 KB
rules/agents-md/instruction-design.ko.mdmarkdown13.8 KB
rules/agents-md/instruction-design.mdmarkdown12.6 KB
rules/agents-md/mode-contract.ko.mdmarkdown18.4 KB
rules/agents-md/mode-contract.mdmarkdown17.0 KB
rules/agents-md/nested-placement.ko.mdmarkdown3.5 KB
rules/agents-md/nested-placement.mdmarkdown3.2 KB
rules/agents-md/project-discovery.ko.mdmarkdown8.2 KB
rules/agents-md/project-discovery.mdmarkdown7.7 KB
rules/agents-md/rule-file-splitting.ko.mdmarkdown5.9 KB
rules/agents-md/rule-file-splitting.mdmarkdown5.2 KB
rules/agents-md/validation.ko.mdmarkdown10.2 KB
rules/agents-md/validation.mdmarkdown9.4 KB
rules/docs/context-engineering.ko.mdmarkdown13.1 KB
rules/docs/context-engineering.mdmarkdown12.0 KB
rules/docs/forbidden-patterns.ko.mdmarkdown6.0 KB
rules/docs/forbidden-patterns.mdmarkdown5.8 KB
rules/docs/harness-engineering.ko.mdmarkdown15.1 KB
rules/docs/harness-engineering.mdmarkdown13.7 KB
rules/docs/layer-model.ko.mdmarkdown1.3 KB
rules/docs/layer-model.mdmarkdown1.3 KB
rules/docs/mode-contract.ko.mdmarkdown13.1 KB
rules/docs/mode-contract.mdmarkdown12.7 KB
rules/docs/required-behaviors.ko.mdmarkdown9.4 KB
rules/docs/required-behaviors.mdmarkdown8.7 KB
rules/docs/sourcing.ko.mdmarkdown5.7 KB
rules/docs/sourcing.mdmarkdown5.3 KB
rules/docs/structured-reasoning.ko.mdmarkdown2.9 KB
rules/docs/structured-reasoning.mdmarkdown2.8 KB
rules/docs/validation.ko.mdmarkdown10.0 KB
rules/docs/validation.mdmarkdown9.3 KB
rules/memory/lifecycle.ko.mdmarkdown5.6 KB
rules/memory/lifecycle.mdmarkdown4.8 KB
rules/memory/mode-contract.ko.mdmarkdown8.3 KB
rules/memory/mode-contract.mdmarkdown6.9 KB
rules/memory/safety.ko.mdmarkdown5.9 KB
rules/memory/safety.mdmarkdown5.3 KB
rules/memory/structure.ko.mdmarkdown6.1 KB
rules/memory/structure.mdmarkdown5.3 KB
rules/memory/validation.ko.mdmarkdown3.8 KB
rules/memory/validation.mdmarkdown3.4 KB
rules/mode-routing.ko.mdmarkdown6.1 KB
rules/mode-routing.mdmarkdown5.7 KB
rules/prompt/anti-patterns.ko.mdmarkdown2.4 KB
rules/prompt/anti-patterns.mdmarkdown2.3 KB
rules/prompt/artifact-levels.ko.mdmarkdown1.1 KB
rules/prompt/artifact-levels.mdmarkdown1.1 KB
rules/prompt/context-source-safety.ko.mdmarkdown5.2 KB
rules/prompt/context-source-safety.mdmarkdown4.6 KB
rules/prompt/evaluation-and-iteration.ko.mdmarkdown6.9 KB
rules/prompt/evaluation-and-iteration.mdmarkdown6.3 KB
rules/prompt/mode-contract.ko.mdmarkdown14.7 KB
rules/prompt/mode-contract.mdmarkdown14.0 KB
rules/prompt/prompt-contract.ko.mdmarkdown6.8 KB
rules/prompt/prompt-contract.mdmarkdown6.1 KB
rules/prompt/prompt-pack-workflow.ko.mdmarkdown3.0 KB
rules/prompt/prompt-pack-workflow.mdmarkdown2.7 KB
rules/readme/mode-contract.ko.mdmarkdown8.1 KB
rules/readme/mode-contract.mdmarkdown7.1 KB
rules/readme/project-discovery.ko.mdmarkdown6.1 KB
rules/readme/project-discovery.mdmarkdown5.5 KB
rules/readme/section-design.ko.mdmarkdown5.5 KB
rules/readme/section-design.mdmarkdown4.9 KB
rules/readme/validation.ko.mdmarkdown2.9 KB
rules/readme/validation.mdmarkdown2.6 KB

rules/prompt/evaluation-and-iteration.md

View on GitHub
# Evaluation And Iteration

**Purpose**: Define eval cases, judges, and bounded optimization rounds. **Read when**: building, scoring, or optimizing a reusable prompt.

## Minimum Categories

Include positive, negative, boundary, workflow, source, safety, regression, and adversarial cases when creating a JSONL prompt eval fixture. The set is composed, not counted: no number of cases is a validated threshold.

## Anti-Tautology Rule

An eval must be able to fail under a plausible bad prompt. Do not count checks that only assert file existence, keyword presence unrelated to behavior, or a restatement of the implementation text. A case whose expected output was written from the current implementation measures the implementation, not the behavior.

## Case Shape

Each medium-or-higher case should define:

- non-empty `id`, `category`, `language`, `invocation`, `risk`, and `prompt`
- observable `expected.must` and `expected.mustNot`
- runner and judge identity
- required trajectory evidence in `trace`
- a binary acceptance rule in `gate`

Cover the same intent across English, Korean, and mixed-language prompts. Include explicit, implicit, and contextual invocation so trigger tests do not pass from the skill name alone.

## Measurement

A non-deterministic behavior is not judged from one run.

- Every case declares its own `runs` and its aggregation rule. A case carrying neither is not a gate.
- A gate declares two numbers: the equivalence margin it will accept and the interval width its run count buys. A change inside the margin does not block, and a narrow interval alone never licenses ignoring a regression.
- Report `pass^k` (all k runs succeed) alongside `pass@k` (at least one run succeeds). Only the first is a reliability claim. At a 0.75 per-trial success rate, three consecutive successes occur about 42 percent of the time, so `pass@3` looks strong while `pass^3` is already weak.
- Record the model and runtime version, the case-set hash, the grader version, the benchmark version, and the run count with every result. A result without them cannot be compared later.
- A run that could not execute is `SKIP`, not a pass. A trial aborted by a harness fault is excluded from the denominator and reported as an error trial, never averaged in as a failure and never counted as a pass.
- 3 to 5 cases is a starting point for an instruction-only change and 8 to 15 for a standard prompt change; neither is a validated floor. State the target interval width and the run count together.
- Cost is a peer of quality. Two prompts at similar accuracy can differ by orders of magnitude in tokens and money, so a change that buys a point for ten times the tokens reports both numbers.
- A single-digit difference stays unresolved until the grader and the resource configuration are shown to be equivalent. Re-grading unchanged output has moved a published score by about nine points.
- A saturated case stops producing signal. When a case passes at the ceiling, move it to the regression suite and add a harder case in its place.

## Guard Integrity

- The candidate may not modify the metric procedure, the case set, the judge, the mandatory guards, or the evidence log.
- A rising metric with shrinking guard coverage is a tampering signal, not progress. Fewer executed checks, a changed evaluator, or a reworded metric all count as narrowing.
- When the artifact is allowed to modify its own harness, the evaluator, the guards, and the log stay outside the mutable set, and that change is reviewed as a separate change.

## Judge Calibration

- State the rubric with explicit prohibitions and a counterexample per dimension.
- Give the judge an `unknown` verdict so it does not have to guess.
- Label a small human sample, have the judge label the same set, compute the alignment rate, inspect the disagreements, and revise the judge prompt until the rate clears a stated threshold. Record the judge model, its version, and the achieved rate.
- A judge that looks good on style is not a judge for correctness. Score dimensions separately instead of averaging several judges.
- A judge that was not calibrated is reported as uncalibrated, not as a pass.

## Iteration Loop

Use no loop when direct deterministic checks prove the requested artifact. For optimization, run at most three candidate iterations:

1. Capture baseline output and score, with the run count and the guard set.
2. Declare the target metric or rubric and the improvement direction.
3. Freeze cases, runner, judge, and non-regression guards, and record their versions.
4. Diagnose failure as instruction gap, context gap, schema mismatch, source-boundary issue, safety issue, or model/runtime mismatch.
5. Patch the smallest prompt surface and rerun the same cases.
6. Keep the candidate only when the target improves and every guard still passes at its declared coverage; otherwise discard it.
7. Add a regression case for every new failure pattern, record the cost delta, and record the remaining risk.

## Stop

Stop optimizing when the target is met, three candidates have been evaluated, a guard failure would require broader scope, or remaining failures need missing context, capability, model/runtime, or user authority. Never change the baseline, cases, runner, judge, or guards to claim improvement; a widened guard or a narrowed case set resets the comparison to baseline instead of counting as progress.

## Sources

> Checked 2026-11-03 against the repository instruction base.

| Claim | Source | Checked |
|---|---|---|
| Judgement rules, the eval case shape, `pass^k` alongside `pass@k`, and the run-count record | `instructions/harness-engineering/HARNESS_ENGINEERING.md` | 2026-11-03 |
| The noise floor, re-grading movement, cost as an axis, and eval awareness | `instructions/harness-engineering/references/measurement.md` | 2026-11-03 |
| Grader hierarchy, judge calibration, partial credit, and the reporting rules | `instructions/validation/references/evaluation-design.md` | 2026-11-03 |
| A candidate may not edit the metric, guards, or log, and a narrower guard set is a stop rule | `instructions/autoresearch/AUTORESEARCH.md`, `instructions/autoresearch/references/core-loop.md` | 2026-11-03 |
| The with/without baseline and removing assertions that pass in both conditions | `instructions/skill/references/validation.md` | 2026-11-03 |

No external URL is cited in this file; every rule is a restatement of the repository base above.
rea-changelog-updatemorluto/reaPrepare or rewrite REA release changelogs and GitHub release notes from pinned Git history, with verified contributor thanks and Release Please synchronization.Stars: 470kUpdated Claude CodeCodexVerifiedInstall
academy-guideanthropics/skillsStop and check this skill before finishing any reply to a question about how to use Claude or a Claude product — it recommends matching courses, tutorials, and use cases from Claude Academy (academy.claude.com), Anthropic's learning hub. Trigger on: "how do I", "how can I", "getting started with", "what can Claude do", "teach me", "learn to use"; questions about artifacts, projects, skills, plugins, connectors, MCP; requests about rolling Claude out to a team, class, or organization; and any ask for training materials, onboarding content, or learning resources. Use it when the user is learning how to use a feature or product — not when they are mid-task and just want the task done. This skill composes with other skills: after consulting product documentation to answer how a Claude feature works, also check here for a matching course or tutorial — a docs-grounded answer and an Academy recommendation belong together. Only recommend on a strong match; never invent Academy content.Stars: 197kUpdated Claude CodeCodexVerifiedInstall
claude-apianthropics/skillsReference for the Claude API / Anthropic SDK — model ids, pricing, params, streaming, tool use, MCP, agents, caching, token counting, model migration. TRIGGER — read BEFORE opening the target file; don't skip because it "looks like a one-liner" — whenever: the prompt names Claude/Anthropic in any form (Claude, Anthropic, Fable, Opus, Sonnet, Haiku, `anthropic`, `@anthropic-ai`, `claude-*`, `us.anthropic.*`, `[1m]`); the user asks about an LLM (pricing/model choice/limits/caching) — never answer from memory; OR the task is LLM-shaped with provider unstated (agent/MCP/tool-definition/multi-agent/RAG/LLM-judge/computer-use; generate/summarize/extract/classify/rewrite/converse over NL; debugging refusals/cutoffs/streaming/tool-calls/tokens). SKIP only when another provider is being worked on (overrides all triggers): OpenAI/GPT/Gemini/Llama/Mistral/Cohere/Ollama named in the query; OR `grep -rE 'openai|langchain_openai|google.generativeai|genai|mistralai|cohere|ollama'` over the project hits (run this grep FIRST if no provider named — don't Read the file).Stars: 197kUpdated Claude CodeCodexUnverifiedInstall
doc-coauthoringanthropics/skillsGuide users through a structured workflow for co-authoring documentation. Use when user wants to write documentation, proposals, technical specs, decision docs, or similar structured content. This workflow helps users efficiently transfer context, refine content through iteration, and verify the doc works for readers. Trigger when user mentions writing docs, creating proposals, drafting specs, or similar documentation tasks.Stars: 197kUpdated Claude CodeCodexVerifiedInstall