kklimuk/docx-cli / .claude/skills/weak-agent-test
weak-agent-test
VerifiedRun the weak-agent adversarial test harness against docx-cli. Spawns weak exercise agents (Haiku by default, Sonnet to probe, or a local agent harness's pre-produced runs) to perform real document tasks over six scenarios — five editing (MNDA form-fill + font fidelity, invoice table-edit/restructure + logo replace, résumé styling, contract redlining + commenting, contract finalize via accept/reject + comment reply/resolve) and one authoring (T. S. Eliot poetry journal: multi-column, verse, footnotes, links, figure) — renders every result with Word, has opus judge them against ground-truth rubrics, measures each exercise's tool economy, token cost, wall-clock, and correctness (from transcripts for Claude, the exercise.json ledger for the local harness), and synthesizes a prioritized ergonomics report. Use when the user says 'adversarial review', 'test docx-cli with weak agents', 'run the haiku harness', 'weak agent test', or wants to re-run yesterday's adversarial process.
scenarios/contract-markup/criteria.md
View on GitHub# Grading rubric — contract-markup (JUDGE ONLY)
## Pass conditions
1. **Changes are tracked** — the clause edits exist as real tracked revisions
(`docx track-changes list` is non-empty). Either route counts: the document-level
toggle (`track-changes on`) OR per-call `--track` on each edit. The document's
toggle being OFF at the end is NOT a failure by itself — what matters is that the
wording changes are visible redlines.
2. **≥ 3 tracked text edits or deletions** — at least three of the following egregious
terms are revised as tracked changes (visible redlines, not silent edits):
- §6 Non-Competition: five-year / worldwide scope reduced or struck
- §9 Limitation of Liability: the asymmetric $100 cap made mutual or raised
- §3 Compensation: Net-90 changed to Net-30 (or similar)
- §8 Indemnification: Company's-own-negligence carve-out added
Verified by: `docx track-changes list` shows ≥ 3 entries of type `insertion` or
`deletion` touching the relevant clause paragraphs.
3. **§9 line spacing increased** — the §9 Limitation of Liability paragraph's line
spacing is opened up (to roughly 1.5, and/or extra space added after). Tracking
this formatting fix is OPTIONAL: a human reviewer applies house-style and
legibility fixes directly and redlines only the wording, so an untracked spacing
change (`docx read --ast` shows `spacing.line` ≈ 360 with `lineRule` auto, or a
`docx:p … line-spacing="1.5"` hint) passes exactly like a tracked one (a
`track-changes list` entry whose `current` spacing exceeds its `prior`).
Restructuring §9's wording or leaving it cramped does not satisfy this — the
spacing on that paragraph must actually increase.
4. **≥ 4 anchored comments** — comments attached to text in at least four of:
- §4 IP over-assignment
- §11 Personal guarantee
- §12 Company-selected arbitration / one-way fee-shifting
- §7 Waiver of independent counsel
- §13 Unilateral amendment by URL posting
Verified by: `docx comments list` returns ≥ 4 entries, each with a non-empty
`anchor` field pointing to text in the relevant clause.
5. **IP sub-points reformatted to lowercase Roman numerals** — the lettered
sub-points under the Intellectual Property clause (the level-1 list items: "all
work product…", "all intellectual property…", "all of the Contractor's
pre-existing…", "all feedback…") are switched from lower-alpha (a/b/c/d) to
lower-roman (i/ii/iii/iv). The top-level clause numbering is left untouched.
Verified by: `docx read contract.docx --ast` shows those level-1 list
paragraphs with `list.format` = `"lower-roman"` (the natural path is
`docx lists set --at <sub-point pN> --format lower-roman`). Note the markdown
body renders ordered items as decimal regardless, so the `--ast` `list.format`
(or a `<!-- docx:list … format="lower-roman" -->` hint) is the signal, not the
rendered glyph.
## How to verify
```
# 1. Check tracked changes (the clause redlines; the §9 spacing may or may not be here)
docx track-changes list contract.docx
# 2. Check anchored comments
docx comments list contract.docx
# 3. Spot-check §9 for the line-spacing increase — tracked (a spacing entry whose
# values go up, e.g. "spacing.line ·→360") OR applied directly (the docx:p hint)
docx track-changes list contract.docx | grep -iE 'format|spacing'
docx read contract.docx | grep -E 'line-spacing='
# 4. Confirm the IP sub-points are lower-roman (level-1 list items)
docx read contract.docx --ast | grep -o '"format":"lower-roman"'
```
All five conditions must pass for the scenario to be scored green.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |