yzsp-studio/mediakit-shared-suite / verifier-hub
verifier-hub
VerifiedDeterministic artifact verifier CLI (file/xlsx/docx/pdf/pptx/text/archive/rubric) for pre-delivery artifact checks and agentic judges. 58 subcommands callable as `verifier {family} {subcmd} [args]`. Use before delivering or uploading a file, or whenever you need to check file format, sheet/section presence, cell values, formulas, keywords, character counts, page counts, placeholders, signatures or zip contents, instead of guessing from an LLM reading. Every call prints one JSON object with a quotable `evidence` field.
| File | Kind | Size |
|---|---|---|
| README.md | markdown | 3.9 KB |
| SKILL.md | skill md | 8.5 KB |
| bin/verifier | other | 8.2 KB |
| bin/verifier.cmd | script | 482 B |
| lib/__init__.py | script | 299 B |
| lib/_common.py | script | 14.3 KB |
| lib/_pdf_backend.py | script | 8.1 KB |
| lib/archive.py | script | 6.2 KB |
| lib/docx.py | script | 32.2 KB |
| lib/file_io.py | script | 10.3 KB |
| lib/pdf.py | script | 8.9 KB |
| lib/pptx.py | script | 11.1 KB |
| lib/rubric.py | script | 17.9 KB |
| lib/text.py | script | 21.9 KB |
| lib/xlsx.py | script | 25.7 KB |
| references/questionnaire.md | markdown | 4.1 KB |
| references/subcommands.md | markdown | 11.0 KB |
| requirements.txt | manifest | 2.3 KB |
references/questionnaire.md
View on GitHub# questionnaire 评分员模式
只有当前工作区里存在 `questionnaire.md`、`rubric_index.yaml`、`run_verifiers.py`
这些文件时才适用。这些脚本由评分 harness 现场铺进工作区,**不在本 skill 包里**。
工作区里没有它们,就按 SKILL.md 的交付前自检流程用 verifier,不要套用这里的工作流。
## 工作区文件
此时当前 session 工作区(`judge_<id>/`)已经由 harness 准备好:
| 文件 | 用途 |
| ---- | ---- |
| `task.md` | 被评估 Agent 当时拿到的任务描述(只读) |
| `trajectory.jsonl` | 被评估 Agent 的完整工具调用轨迹(只读) |
| `artifacts/*` | 被评估 Agent 实际交付的文件(xlsx/pptx/docx/pdf/html…) |
| `ARTIFACT_INVENTORY.txt` | bootstrap 已经预跑了 `verifier file artifact-list artifacts/`,**先 cat 这个**,不要重复跑 |
| `PREFLIGHT.txt` | bootstrap dry-run `verify_questionnaire.py` 的结果,用来确认环境健康 |
| `questionnaire.md` | 你要填的问卷(唯一可写文件) |
| `rubric_index.yaml` | rubric_id → score_labels 的索引(只读) |
| `verify_questionnaire.py` | 自检脚本:填完后必须跑到看见 `OK`(已 `chmod 555`) |
| `fill_rubric.py` | 写盘助手:单条或批量(`--batch-file`)写 questionnaire.md(已 `chmod 555`) |
| `run_verifiers.py` | 批量 verifier 调度:1 次 Bash 调用并行跑 N 个 verifier(已 `chmod 555`) |
## 推荐工作流(4 步)
优先走批量,减少串行等待和重复调用。
1. `cat ARTIFACT_INVENTORY.txt PREFLIGHT.txt task.md questionnaire.md`
—— 先读预计算清单和静态信息。
2. `python run_verifiers.py --spec /tmp/spec.yaml --output /tmp/evidence.json`
—— spec 里每条 rubric 一个 entry(`{id, cmd: [<family>, <subcmd>, <args...>]}`),
harness 并行调度(默认并发 8)。同一子命令要跑多个文件就开多个 entry。
`evidence.json.results[*].stdout` 就是 verifier 的原始 JSON 信封。
3. `python fill_rubric.py --batch-file /tmp/decisions.yaml`
—— decisions.yaml 是一个 list,元素为 `{rubric_id, score, rationale}`,一次写完所有 rubric。
任一条失败只影响它自己,已写盘的保留。
4. `python verify_questionnaire.py`
—— 看到 `OK: N/N rubrics valid` 即结束。
## `run_verifiers.py` 的 spec 格式
```yaml
- id: hard_0
cmd: [rubric, check-file-format, artifacts/report.pptx, --expected-ext, .pptx]
- id: hard_1
cmd: [pptx, list-slides, artifacts/report.pptx]
# 同一个子命令对多个文件 → 多个 entry(并行跑)
- id: soft_3_a
cmd: [xlsx, assert-value, artifacts/data.xlsx, --sheet, Sum, --cell, B2, --expected, '1234']
- id: soft_3_b
cmd: [xlsx, assert-value, artifacts/data2.xlsx, --sheet, Sum, --cell, B2, --expected, '5678']
```
输出信封(写到 `--output` 或 stdout):
```json
{
"ok": true,
"concurrency": 8,
"elapsed_sec": 3.21,
"results": [
{"id": "hard_0", "ok": true, "exit_code": 0,
"stdout": "{\"ok\": true, \"tool\": \"rubric.check-file-format\", \"evidence\": {...}}",
"stderr": "", "elapsed_sec": 0.42},
{"id": "soft_3_b", "ok": false, "exit_code": 1,
"stdout": "...{\"ok\": false, \"error\": ...}", "stderr": "", "elapsed_sec": 0.38}
]
}
```
按 `id` 把 `stdout` 里 verifier 信封的 `evidence.quote`(rubric 家族是
`result.evidence_quote`)拷进 decisions.yaml 的 `rationale` 字段。
注意 `exit_code` 为 1 只说明该条 verifier 的 `ok` 是 false,也就是工具本身没跑成;
校验结论不通过属于 `ok: true` + `result.passed: false`,`exit_code` 仍是 0。
## `fill_rubric.py` 批量与单条
```bash
# 推荐:批量
python fill_rubric.py --batch-file /tmp/decisions.yaml # 文件
python fill_rubric.py --batch-stdin < /tmp/decisions.yaml # stdin
# 单条(仅在补打补丁时用)
python fill_rubric.py <rubric_id> <score> "<rationale,不少于 30 字>"
python fill_rubric.py <rubric_id> <score> --rationale-file /tmp/r.txt
```
每条 rubric 一次 Bash 的单条链式写法,只在批量确实不合适时用,
例如某条 rubric 必须先读大量 trajectory 上下文、无法提前批量取证。
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| canvas-designanthropics/skillsCreate beautiful visual art in .png and .pdf documents using design philosophy. You should use this skill when the user asks to create a poster, piece of art, design, or other static piece. Create original visual designs, never copying existing artists' work to avoid copyright violations. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| docxanthropics/skillsUse this skill whenever the user wants to create, read, edit, or manipulate Word documents (.docx files) or Word templates (.dotx files). Triggers include: any mention of 'Word doc', 'word document', '.docx', '.dotx', or requests to produce professional documents with formatting like tables of contents, headings, page numbers, or letterheads. Also use when extracting or reorganizing content from .docx or .dotx files, inserting or replacing images in documents, performing find-and-replace in Word files, working with tracked changes or comments, or converting content into a polished Word document. If the user asks for a 'report', 'memo', 'letter', 'template', or similar deliverable as a Word or .docx file, use this skill. Do NOT use for PDFs, spreadsheets, Google Docs, or general coding tasks unrelated to document generation. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| pdfanthropics/skillsUse this skill whenever the user wants to do anything with PDF files. This includes reading or extracting text/tables from PDFs, combining or merging multiple PDFs into one, splitting PDFs apart, rotating pages, adding watermarks, creating new PDFs, filling PDF forms, encrypting/decrypting PDFs, extracting images, and OCR on scanned PDFs to make them searchable. If the user mentions a .pdf file or asks to produce one, use this skill. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| pptxanthropics/skillsUse this skill any time a .pptx or .potx file is involved in any way — as input, output, or both. This includes: creating slide decks, pitch decks, or presentations; reading, parsing, or extracting text from any .pptx or .potx file (even if the extracted content will be used elsewhere, like in an email or summary); editing, modifying, or updating existing presentations; combining or splitting slide files; working with templates (.potx), layouts, speaker notes, or comments. Trigger whenever the user mentions "deck," "slides," "presentation," or references a .pptx or .potx filename, regardless of what they plan to do with the content afterward. If a .pptx or .potx file needs to be opened, created, or touched, use this skill. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |