Skip to content

HK-hub/AgentSkills / ax-audit

ax-audit

Verified

Audits agentic applications for architecture and trust: tool parity, tool granularity, context injection, completion signals, approval gates, confidence cues, escape hatches, intent handshakes, memory visibility, and adaptive canvases. Produces a ship-readiness verdict plus an AX Relationship Summary. Use when reviewing agentic feature PRs or asking "is this agent-native", "AX review", "critique this AI feature", "does this earn user trust", or "audit this for AX". For traditional frontend UX use ui-audit.

AX Audit

Feature-level reviewer for apps where an agent acts for the user. One question: does it earn trust, and where does it break?

  • IS: rules-based audit of agentic surfaces (agent chat, tool execution panels, agent config, dashboards) across two layers (architecture in rules-arch/, trust/relationship design in rules-ax/), ending in a ship-readiness verdict plus an AX Relationship Summary.
  • IS NOT: traditional frontend UX (forms, states, focus, async, microcopy, accessibility, layout, typography, performance, use ui-audit); agent instruction-file quality (use agents-md).

No agentic features in scope (only forms, lists, modals)? Route to ui-audit; AX rules against traditional UI produce only noise.

Contents

Audit workflow

Track this checklist:

AX Audit progress:
- [ ] Step 1: Scope, via `git diff --name-only main` (PR mode) or explicit path (full sweep)
- [ ] Step 2: Detect agentic features per references/feature-playbooks.md
- [ ] Step 3: Run each detected feature's playbook in order, plus the diff-wide checks
- [ ] Step 4: For each check, load the rule file and follow its detection recipe
- [ ] Step 5: Tier each finding per references/ship-readiness.md (rule override table wins)
- [ ] Step 6: Render verdict + findings + AX Relationship Summary per references/output-format.md
- [ ] Step 7: Run the audit self-check and report its evidence counts

Step notes:

  1. Scope. Default: PR diff plus the tool definitions and orchestrator code it touches. Findings in untouched files belong in a full sweep, not a PR verdict.
  2. Detect. Heuristics (component names, hooks, routes) for the four feature types live in references/feature-playbooks.md.
  3. Playbooks. Each feature has 5-9 ordered checks; run all, even expected passes (a pass with evidence belongs in the report). The diff-wide parity-orphan-ui-action runs on every PR-mode audit regardless of detected features.
  4. Rules. Each rule file carries its own detection commands, false-positive guards, tier override table, and suppression syntax, and is authoritative; playbook annotations are a convenience copy.
  5. Tier. Three tiers; precedence below.
  6. Render. Group findings by surface; verdict block first, AX Relationship Summary last.
  7. Self-check. Evidence or it didn't happen (see below).

Two rule layers

LayerFolderRulesQuestion it answersCategory index
1: Agent-native architecturerules-arch/11Can the agent do what the user can do? Are tools atomic? Does the agent know what exists? Is completion explicit?rules-arch/_sections.md
2: Agentic experiencerules-ax/12Does the agent earn trust? Can the user interrupt, undo, push back? Is memory visible?rules-ax/_sections.md

Load rules-arch/<category>-<slug>.md or rules-ax/<category>-<slug>.md when a playbook check names it. Categories: arch = parity, granularity, context, comm; ax = trust, control, context, comm. Both layers share the comm and context prefixes, but the rules differ: rules-arch/comm-no-approval-gate.md (orchestrator code has no gate logic) is not rules-ax/control-no-approval-gate.md (approval UI doesn't match the stakes).

Tiers and verdict

Every finding gets exactly one tier (full trigger lists in references/ship-readiness.md):

  • release-blocker, fix before merge: no escape hatch, silent execution, heuristic completion, broken parity, ungated high-stakes actions
  • fix-this-sprint, merge with a tracked issue: no confidence cues, no intent handshake, opaque memory, bundled config tools
  • backlog, ship and track: static canvas, no generative momentum, static API mapping, no checkpoint/resume

Tier precedence: a rule's own surface-override table > the generic surface bump in references/ship-readiness.md > the rule's defaultTier. Apply at most one adjustment; never stack the generic bump on a rule's explicit override.

Verdict: ✅ READY (0 blockers, ≤3 sprint) · ⚠️ READY WITH FOLLOW-UP (0 blockers, ≥4 sprint) · ❌ NOT READY (≥1 blocker) · 🚫 INCOMPLETE (self-check failed).

AX Relationship Summary

Rendered after findings when any agentic feature was detected. Findings serve engineers; this serves designers and PMs, so never skip it. Four fields:

  • Evolution stage: behavior description, not a label (see references/ax-evolution-curve.md)
  • Trust signal: high/moderate/low, one-line reasoning from trust-critical rules
  • Key gap: the single most important gap, one actionable sentence
  • Trust question: one question only prototyping or research can answer

Reference files

FileRead when
references/feature-playbooks.mdSteps 2-3: detection heuristics, per-feature ordered checks, diff-wide checks
references/ship-readiness.mdStep 5: tier triggers, precedence, verdict logic
references/output-format.mdStep 6: findings JSON schema, summary schema, terminal rendering
references/agent-native-principles.mdA Layer 1 finding needs grounding: parity, granularity, CRUD completeness, context patterns, approval matrices, checkpoint/resume
references/ax-evolution-curve.mdWriting the evolution-stage field of the AX summary
rules-arch/_sections.mdLayer 1 categories and default tiers
rules-ax/_sections.mdLayer 2 categories, default tiers, co-firing rule pairs

Gotchas

  • Scope before rules. Running all 23 rules repo-wide on a 3-file PR buries a new release-blocker under pre-existing backlog noise; the verdict stops meaning "can this PR merge."
  • The rule's override table is authoritative. comm-no-intent-handshake defaults to fix-this-sprint but its table says release-blocker on tool execution. Stacking the generic "+1 tier on tool execution" bump on an explicit override double-upgrades backlog findings into blockers.
  • A stop button not wired to AbortController.abort() is a false affordance. control-no-escape-hatch still fails: verify the abort() call, not the button label, or the audit passes a UI that lies to users.
  • Absence checks need a recorded file list. "Find components lacking X" greps return nothing both when everything passes and when nothing was scanned. List candidate files first (rg -l <feature-pattern>), check each for the counter-pattern, and cite the file list as evidence.
  • detection: observational rules cannot fail on grep evidence alone. granularity-static-api-mapping, trust-no-uncertainty-markers, control-over-conversational, and comm-no-generative-momentum need interaction-flow judgment; on static evidence alone, return unknown with a reason, not fail.
  • ax-audit-ignore:<slug> comments count as suppressed, not pass. Report the count in the verdict block; a suppression with no reason is itself worth a warn.
  • Don't duplicate ui-audit findings. "Missing loading state" and "form clears on error" are ui-audit territory; duplicating them trains engineers to dismiss the whole AX report.
  • Don't inflate tiers. comm-no-generative-momentum and granularity-static-api-mapping default to backlog. Promoting cosmetic findings to blocker trains the team to ignore ❌ verdicts.

Audit self-check

Flag the audit INCOMPLETE if any of these hold, and include the counts as evidence (planned vs. run rules per playbook, unknown rate, suppressed count):

  • Fewer rules ran than the playbooks planned
  • More than 30% of rules returned unknown
  • Any fail/warn finding lacks file:line evidence or a fix snippet
  • Every finding landed in the same tier (suspect blanket assignment)
  • AX Relationship Summary is missing despite detected agentic features

Related skills

  • ui-audit: traditional frontend UX quality around agentic surfaces; run both on agentic feature PRs, with ax-audit covering the agent layer
  • agents-md: audit CLAUDE.md / AGENTS.md agent instruction files
  • define-architecture: repo structure and module boundaries
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 470kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 470kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 470kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall