rrrrrredy/research-toolkit / plugins/research-toolkit
research-toolkit
FlaggedUse for source-backed long-form research and writing in English or Chinese across fields, including revisions and continuation. Understand requirements, develop analysis through substantive research collaboration, and deliver a complete report after independent review, audit and acceptance.
docs/delivery-verification.md
View on GitHub# Delivery checker: contract and observation limits
[English](delivery-verification.md) | [简体中文](delivery-verification.zh-CN.md)
Use this contract when CLI/MCP manages delivery or the task explicitly requires its receipt. Skill-only reports complete their agreed research and checks without creating these state files or receipts.
The checker is an optional offline record-consistency check, not a runtime permission hook. Use actual execution records to verify an invocation. Research and writing requirements remain in [SKILL.md](https://github.com/rrrrrredy/research-toolkit/blob/main/SKILL.md); this page documents the checker interface.
The default is **delivery contract 3**; receipt `schema_version: 1` is unchanged. `delivery_contract_version` and `current_contract_checked` identify the checks performed. With `semantic_verification: false`, verify consent against user decisions, reading against source notes and originals, and editorial quality through independent review.
The shared CLI needs Python 3.10+ and only the standard library. Prefer `python3` on Linux; any installed Python 3.10+ interpreter is enough.
```bash
python3 <skill-directory>/scripts/check_delivery.py <task-directory>
```
The default checks record consistency and can pass for a valid in-progress task. To ask whether the **current delivery is complete**, use:
```bash
python3 <skill-directory>/scripts/check_delivery.py <task-directory> --require-complete --json
```
This read-only mode exits 0 only for `stage: final` and `status: complete` with the existing delivery, requirement, independent review, audit, acceptance and version checks passing. Otherwise it exits 1 with the specific `flags` and `findings`, including `incomplete_task` for nonterminal work. It does not call `finish`, launch reviews, change task files or create a receipt. It uses the current delivery contract; `--contract-version 1/2` remains available without this flag for historical diagnostics.
The receipt binds the current artifact, brief, progress, source and claim records, review log, and intended delivery message. Applicable requirement and uncertainty files are also bound. Every path recorded in the receipt remains subject to existence, task-directory boundary and hash checks, even if it was optional when the receipt was created. An optional file that was never bound may remain absent. Text hashes normalize CRLF and CR to LF. Finish the relevant edits before sealing; a later edit makes the old receipt stale. Do not rerun unrelated research just to create a new receipt.
Use `--delivery-message note.md` for a non-default intended message. The receipt must bind that selected filename. Artifact, receipt and intended-message paths must resolve inside the task directory.
New and continued deliveries follow the applicable [review contract](review-completion.md). New full reports use three independent review groups, specialist review where needed, a separate audit and an independent final check. See [compatibility](compatibility.md) for saved plans and historical receipts.
<a id="full-delivery-completion"></a>
## Research delivery completion
Reconcile the actual answer with effective requirements and evidence. Complete current-version independent content reviews, audit and final acceptance, resolving necessary corrections in the report. A valid negative review completes that review but leaves delivery pending.
Once the current report passes independent checks, call `research_finish` with the artifact and delivery message, then deliver the report. A changed report needs current-version review and independent final checking. The operation binds inputs and review results, writes `state/final_delivery.json`, and enters `stage: final` with `status: complete` only when the required work is complete. Do not hand-write a duplicate receipt or author a global PASS to replace independent acceptance. Use the standalone checker when inspecting retained records; it is not another content-review invocation.
For both CLI `finish` and MCP `research_finish`, `completed: false` means work remains, including when `action_required` is present. Address the returned gaps; CLI `finish` exits 1 in this case and exits 0 only for `completed: true`. MCP keeps the same business result in its JSON: successful transport alone does not mean completion. CLI `review` still exits 0 when it successfully prepares independent assignments requiring execution. On first delivery, reuse the successful `finish.delivery_check`; an immediate duplicate check is unnecessary. Use `--require-complete` when later checking retained completion.
## Requirement closure and reading records
In `state/requirements.jsonl`, `satisfied` needs a non-empty evidence string or list of strings. A requirement closed as `waived`, `out_of_scope`, or `accepted_limitation` needs a `user_decision` object with non-empty `source_turn` and `quote`. The quoted decision must concern that requirement; an original user exclusion can establish scope. A status label, the author's decision, or disclosure of a gap does not replace the user's decision.
For example, these are **synthetic record examples**, not evidence of a real approval:
```json
{"requirement_id":"R-read-1","status":"waived","user_decision":{"source_turn":"user-message-7","quote":"You may skip the inaccessible article."}}
```
For reading that the task actually requires:
```json
{"requirement_id":"R-read-2","status":"satisfied","evidence":"notes/source-2.md","reading_requirement":"full_text","required_source_ids":["S02"]}
```
The linked `data/source_registry.csv` row must occur exactly once and have `read_scope` and a non-empty `read_evidence` reference. Use `full_text`, `relevant_sections`, `abstract`, `partial`, or `not_read` to describe the actual reading. A `full_text` requirement accepts only `full_text`; `relevant_sections` accepts completed relevant-section reading or full text. The evidence reference should identify the read sections and notes or observations that support the record. HTTP 200 and a URL in the registry establish access, not reading. Optional background sources do not become mandatory full reads.
Clear local file references in `read_evidence` must resolve to an existing, nonempty file inside the task directory. Missing files, directories, empty/whitespace-only files and paths escaping that directory fail. A file may include an anchor or page/section suffix. URLs and section/page locators remain accepted without fetching or verifying their content.
The checker compares declared fields. Reconcile the ledger with the full request, verify quoted decisions against their source turns, and check reading notes against original sections. The standalone checker and eval runner both check requirement closure, including when the runner's optional receipt check is disabled.
## Explicit length requirements
Tasks can measure a declared range using an optional `length_constraint` in the existing requirement ledger. Record the task's agreed range and counting convention before drafting/review; do not invent a default minimum or infer one from prose. This **synthetic format example** is not a universal length target:
```json
{"requirement_id":"R-length","summary":"The agreed whole-file limit is 800–1200 nonwhitespace characters.","status":"pending","evidence":"","length_constraint":{"unit":"nonwhitespace_characters","scope":"whole_artifact","min":800,"max":1200}}
```
The agent passes this row in `research_save(requirements=[...])` / CLI `save`; the tool maintains `state/requirements.jsonl` and retains other requirements. Users state the length in ordinary language. See [record saving](usage-modes.md#save-research-records-and-drafts). Start as `pending`; after measuring and checking the agreement, change it to `satisfied` and fill `evidence` (for example `final.md, whole file`).
| Field | Correct entry |
| --- | --- |
| Bounds | `min` / `max`, at least one; not `minimum` or `max_chars`. |
| Values | Nonnegative integers such as `3000`, without quotes; min must not exceed max. |
| Counting unit | `whitespace_words` for whitespace-delimited words; `nonwhitespace_characters` is often suitable for Chinese. |
| Counting scope | `scope: "whole_artifact"` includes headings, Markdown, citations and references. |
The research example provides editable [English](https://github.com/rrrrrredy/research-toolkit/blob/main/examples/full/requirements.en.jsonl) and [Chinese](https://github.com/rrrrrredy/research-toolkit/blob/main/examples/full/requirements.zh.jsonl) length records. Use the task's agreed numbers, not an automatic example target.
Save `{"task":"your-task-id"}` as `check-draft.json`, then run from the repository root:
```bash
python3 scripts/research_workflow.py check-draft --workspace ../research-work --request check-draft.json
```
On Windows, replace `python3` with `py -3` or an installed interpreter path; with MCP, call `research_check_draft(task="your-task-id")`. Read `counts` and `length_checks` for actual counts and bounds; incorrect fields appear in `errors` with the expected form. `bounds_met: true` means the measured file is within the recorded range; `status: pending` means the requirement record still needs checking. In this case `next_action` identifies the requirement and measured count: verify the agreed bounds/scope, then mark only that row `satisfied` with the artifact, count and scope as evidence. Address other listed errors separately. The check remains read-only.
Use `whitespace_words` for Python whitespace-delimited segments, or `nonwhitespace_characters` for Unicode code points except whitespace. Both count the entire UTF-8 artifact, including Markdown syntax, headings, citations and references. They are not model tokens or editor-rendered word counts. Only `whole_artifact` is supported; a body-only or page limit needs its own appropriate measurement, not a relabeled whole-file count. At least one nonnegative integer `min`/`max` is required; bounds are inclusive. Keep a distinct requirement ID and a summary of the actual agreement.
`research_finish` recomputes counts and refuses out-of-range `satisfied` claims before writing a receipt or replacing the delivery message. The receipt stores `length_checks`; the offline checker recomputes them. The historical checklist preview returns the counts and any errors. Existing requirement-closure rules apply to these rows for current and historical tasks: an unresolved status still needs resolution, and an existing user decision can waive/change scope or accept a limitation. A waived range retains its actual count and `bounds_met: false` when outside the range; it is not rewritten as a satisfied measurement. Current reviews already freeze the ledger, so changing the agreed range requires the normal current-version review. Previous receipts without structured length requirements keep their original checks.
The eval runner also checks declared length requirements at terminal delivery when its optional receipt check is disabled. Compare the ledger's bounds with the actual request, and review whether the report develops the required reasoning; word/character counts assess the declared range only.
## Comparing a captured reply
```bash
python3 <skill-directory>/scripts/check_delivery.py <task-directory> --actual-message captured-reply.md
```
For current and historical tasks, the supplied capture is compared with the bound intended message, normalizing only line endings and outer whitespace. A runtime caller can retain its final-message output separately and call the checker after the run. Retain the original runtime output and document any extraction of client metadata; do not silently remove substantive differences.
The `delivery_observation` result is `not_provided`, `matched`, `mismatch`, or `unreadable`. Missing or different supplied captures fail. `not_provided` means only the intended message was checked, not that the actual user received it.
Saved checklist-only task captures remain historical evidence under their original contract. A new continuation preserves those originals and completes the current independent requirements; it does not obtain a current acceptance by repeating the old checklist.
Use the runtime's retained output as capture evidence, rather than an agent-authored copy of the intended message. The checker compares it with the bound intended message; it does not authenticate the capture or control the live final reply.
## Required model-review completion
The delivery checker inspects the [review-completion records](review-completion.md). New full-report tasks record `progress.review_contract: 3`. Their plans bind three independent groups covering seven perspectives, specialist reviews where needed, a separate audit and an independent final check. Saved v2 plans follow their own recorded assignments. Each role retains actual replies, execution evidence and input versions. Author checks do not satisfy those roles.
A global PASS cannot fill a missing assignment. Failed invocations remain separate from editorial findings. Review completion and report readiness are distinct: an actual negative review can be complete, while required corrections still block delivery. The independent auditor checks findings and sampled content; final acceptance covers the complete resulting report. Historical record inspection does not rewrite the earlier contract.
## Task-declared review requirements
Tasks may declare additional required scopes in `progress.required_review_scopes`. Their current results must be complete without open necessary issues, in addition to the current independent content, audit and acceptance duties. Preserve the actual scope and artifact binding; a partial-unit result cannot stand in for a whole-report decision.
The latest global review and each declared required scope also need `artifact_sha256`, the SHA-256 of the primary report they reviewed after LF newline normalization. For example, add that hash to the actual review row alongside its scope and result. A missing or malformed hash fails with `missing_review_artifact_binding`; a different current report fails with `stale_review_artifact`. The receipt separately binds the current review log. Resealing that receipt cannot make an earlier review cover edited prose. A local recheck cannot refresh the global binding, and a global PASS cannot refresh a task-required specialist binding.
Recognized progress statuses are `in_progress`, `paused`, `blocked`, and `complete`. Terminal state requires both `stage: final` and `status: complete`; neither alone is sufficient.
The standalone checker and eval runner share review and open-issue semantics. A clean latest global review covers earlier ordinary unit reviews; later blocking findings still need a clean review of the same scope or a new global review. A local PASS cannot clear another scope or a failed global review. Explicitly required scopes remain independent: a global PASS does not remove their requirements. Malformed history cannot be repaired by appending a PASS; preserve and correct the invalid record explicitly. A `routed_action` alone is a plan, not a resolution.
## Limitation disclosure is not keyword certification
`limitation_disclosure.status` distinguishes `not_applicable`, `absent`, `contradiction`, `text_covered`, and `needs_review`. An absent disclosure signal or a recognized blanket denial of existing limitations fails the mechanical check. The denial detector is bounded and heuristic; it does not recognize every possible wording.
`text_covered` means the recorded limitation text was found after punctuation/whitespace normalization. `needs_review` reports `unmatched_limitations` and a visible warning for partial matches, paraphrases or generic wording. A valid paraphrase does not fail solely because its wording differs. `semantic_verification` is always false; content reviewers check factual accuracy, meaning, coverage and material contradictions. No fixed wording, extra declaration file or model/API dependency is required.
## Draft checks before delivery
`research_check_draft` (CLI `check-draft`) measures the current artifact and checks draft record prerequisites without changing progress or writing a receipt. It returns explicit length checks even before a delivery message exists; an absent bound is a warning, not a range pass. `ok` covers these mechanical checks only; semantic quality and delivery remain `not_assessed`. The shared task lock may create its normal empty file for a legacy task. `finish` always checks the current version again.
The existing `warnings` list also reports ordinary prose paragraphs repeated verbatim at least twice, with at least 24 nonwhitespace characters (the existing evaluator's long-text cutoff). Each warning preserves the paragraph text and every occurrence's section heading and inclusive line range. Matching is case-sensitive and does not normalize wording, citations or internal whitespace. Code blocks, headings, tables, lists and block quotes are excluded. The warning neither deletes text nor changes `ok` or delivery eligibility; the researcher decides whether the repetition is intentional.
## Historical records
Use `--contract-version 1` or `--contract-version 2` only to inspect unchanged historical records that predate these fields. The result and CLI output explicitly label legacy checks and set `current_contract_checked: false`. The eval runner has matching `--delivery-contract-version 1` and `2` options. Version 2 includes requirement decisions, reading scope and report hashes, but omits model-review completion and sampling. Legacy success is not acceptance under the current contract. Do not add invented approval quotes or review hashes to old runs to make them pass; perform and record the missing work for a new delivery instead.
Retain original historical outcomes without backfilling reviews. On continuation, reuse original reports and sources while completing the current independent work.
## Regression coverage
`python3 scripts/check_delivery_contract.py` reseals unrelated hashes so a stale receipt cannot mask the target defect. It includes known-good controls, custom message binding, captured-message differences, declared review scopes, bounded review-event permutations and the terminal-state truth table.
`python3 scripts/check_evaluator_contract.py` adds cross-entry review recovery, later local blockers, malformed/invalid-UTF-8 history, routed-only issues, limitation-denial and partial-coverage controls, and distinct versus repeated English/Chinese text. All diagnostic fixture changes are isolated and resealed; frozen research inputs and historical reports are not rewritten.
Version 2 controls cover unauthorized/authorized requirement closure, declared full/partial reading, unchanged/currently reviewed report controls, and stale global/specialist reviews. The old conformance and regression fixture runners explicitly use version 1 to preserve historical comparisons; their original inputs and hashes stay unchanged. New positive controls are constructed only in isolated test copies and are not backfilled research records.
Contract 3 controls in `scripts/check_review_completion_contract.py` cover missing and failed slots, incomplete or stale evidence, reused responses, independent adjudication, negative evaluation outcomes, sampling and legacy compatibility. All record fixtures are explicitly synthetic; they are not model-review evidence.
In an offline installation, open the following files from the plugin root: `SKILL.md`.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |