heymegabyte/agent-skills / 07-quality-and-verification
quality-and-verification
Unverified5-level verification pyramid: static→unit→Playwright E2E (homepage-first, 6bp)→AI visual→post-deploy. 8-check quality gate. Multi-agent testing (functional/security/a11y/performance). Playwright v1.56+ AI agents (Planner/Generator/Healer, v1.63 current). WCAG 2.2 AA via axe-core v4.13. Percy+Chromatic visual regression. ADA Title II 2027/2028 deadlines.
visual-inspection-loop.md
View on GitHub---
name: "Visual Inspection Loop"
version: "3.0.0"
updated: "2026-05-12"
description: "Dual-vision protocol: Claude Vision (Sonnet 4.6 via Max 20x OAuth, FREE primary, fires everywhere) + GPT Image 2 vision (METERED judge/arbiter, $0.50/build cap, hero/ATF + brand-fidelity + final pre-publish gate + disagreement arbitration). Screenshot → Tier 1 a11y tree+axe-core FREE → Tier 2 Claude Vision FREE → Tier 3 GPT Image 2 vision METERED. Three-round max."
related: completeness-verification.md (5-pass full-project), ui-completeness-sweep.md (pre-done placeholder detection), ~/.claude/rules/visual-inspection.md (canonical rule), ~/.claude/rules/auth-spawned-claude.md (Max 20x OAuth)
---
# Visual Inspection Loop (Dual-Vision)
> **Model migration note (pass-74, 2026-06-09)**: References to `GPT-4o Vision` migrated to **GPT Image 2 vision** (current OpenAI multimodal flagship). Per `platform.openai.com/docs/deprecations`: GPT-4o retired 2026-02-13. Dual-vision protocol (Claude primary + OpenAI judge) unchanged — only API endpoint names updated. Cost cap ($0.50/build) was computed against legacy GPT-4o pricing; re-verify against current `gpt-image-2` rates.
## Canonical Rule
See `~/.claude/rules/visual-inspection.md` — supreme dual-vision doctrine. This skill implements that rule.
## Dual-Vision Stack
### Claude Vision (PRIMARY, FREE on Max 20x)
- Sonnet 4.6 via `~/.claude/.credentials.json` OAuth
- Fires per-slice + per-section + per-route + per-iteration + 6bp; no marginal cost
- Subject only to Max 20x 5hr / weekly rate windows
### GPT Image 2 vision (JUDGE, METERED, $0.50/build cap)
Reserved for: hero/ATF, brand-fidelity vs source, final pre-publish gate, arbitration when Claude Vision <8 or Claude + a11y-tree disagree.
### Consensus
- Both ≥8/10 → ship
- Either <8 → fix → re-run
- Persistent disagreement → third pass (Computer-Use screenshot + checklist)
- Never override <8 with self-judgment
## Cost-Risk Matrix (***BEFORE ANY GPT Image 2 vision CALL***)
### Claude Vision
ZERO marginal cost. 6bp × N routes × every iteration — uncapped within Max 20x rate.
### GPT Image 2 vision per-build cap
**$0.50 hard**. Allocation:
- ~$0.10 hero / ATF final
- ~$0.15 brand-fidelity vs source
- ~$0.10 final pre-publish 6bp homepage
- ~$0.15 reserve for arbitration
Beyond cap → escalate to memory pin + ship with carryover; never silently overspend.
### Build tiers
- **Cost-sensitive** (free-tier, paid-floor) — Claude Vision ONLY; GPT Image 2 vision for single final pre-publish call on homepage
- **Brand-critical** (paying client, founder portfolio, flagship SaaS) — full $0.50 GPT Image 2 vision budget + Claude Vision at every checkpoint
`external_llm.ts` checks `site.pricing_tier` + `vision_budget_remaining_cents` before each GPT Image 2 vision call; auto-degrades to Claude Vision when budget exhausted.
## Where Dual-Vision Fires
- **Per-slice (skill 06)** — Claude Vision YES (2bp: 375+1280) | GPT Image 2 vision NO — after slice deploys to preview
- **Per-section (skill 15)** — Claude Vision YES (1bp: 1280) | GPT Image 2 vision NO — after section renders
- **Per-route 6bp (skill 15)** — Claude Vision YES (6bp full) | GPT Image 2 vision NO (unless route=homepage) — after route assembles
- **Iteration diff (skill 15)** — Claude Vision YES (compare prev + current) | GPT Image 2 vision NO — progressive rebuild loop
- **Brand-fidelity vs source** — Claude Vision YES (auxiliary) | GPT Image 2 vision YES (primary judge) — once per build, hero/ATF
- **Hero / ATF final** — Claude Vision YES | GPT Image 2 vision YES (judge) — pre-publish gate
- **Final pre-publish homepage 6bp** — Claude Vision YES (every route) | GPT Image 2 vision YES (homepage only)
- **Disagreement arbitration** — GPT Image 2 vision YES (tiebreaker) — when Claude <8 or Claude + a11y disagree
## Tier-1 → Tier-2 → Tier-3 Order (***ALWAYS THIS ORDER***)
1. **Tier 1 (FREE, FAST)** — Playwright a11y tree 6bp + axe-core scan + DOM-walker contrast + computed-style overflow. Catches ~80% of issues with zero token spend.
2. **Tier 2 (FREE on Max 20x)** — Claude Vision on screenshots Tier 1 cannot resolve (aesthetics, brand, hierarchy, motion, cinematic feel).
3. **Tier 3 (METERED, CAPPED)** — GPT Image 2 vision on critical checkpoints + arbitration.
## Loop (3-round max, $0.50 GPT Image 2 vision cap)
1. Deploy to production / staging
2. Tier 1 — Playwright a11y tree + axe-core + DOM-walker (FREE) ALL pages 6bp
3. Fix functional / a11y / contrast issues
4. Tier 2 — Claude Vision ALL pages 6bp (FREE) — score + evidence per breakpoint
5. Fix issues scoring <8
6. Re-deploy + purge
7. Round 2: repeat Tier 1 → Tier 2 to verify fixes
8. Round 3 (if still <8): Tier 3 — GPT Image 2 vision arbiter on remaining surfaces + final pre-publish call
9. After round 3: report carryover to `_iteration_log.json`, ship
**Per round:** track GPT Image 2 vision spend in cents; when `vision_budget_remaining_cents < 10` → skip GPT Image 2 vision, ship with Claude Vision verdict.
## Screenshot Capture Template (Dual-Vision)
```typescript
const ALL_BREAKPOINTS = [
{ name: 'iPhone-SE', width: 375, height: 667 },
{ name: 'iPhone-14', width: 390, height: 844 },
{ name: 'iPad', width: 768, height: 1024 },
{ name: 'iPad-Landscape', width: 1024, height: 768 },
{ name: 'Laptop', width: 1280, height: 720 },
{ name: 'Desktop', width: 1920, height: 1080 },
];
// Claude Vision (FREE) — every breakpoint
const CLAUDE_VISION_BREAKPOINTS = ALL_BREAKPOINTS;
// GPT Image 2 vision (METERED) — homepage only at final gate
const GPT4O_BREAKPOINTS = ALL_BREAKPOINTS;
// GPT Image 2 vision brand-fidelity — single ATF screenshot
const BRAND_FIDELITY_BP = [{ name: 'ATF', width: 1280, height: 720 }];
async function captureAllScreenshots(page: Page, pages: string[], outputDir: string) {
for (const route of pages) {
await page.goto(`${PROD_URL}${route}`);
await page.waitForLoadState('networkidle');
for (const bp of ALL_BREAKPOINTS) {
await page.setViewportSize({ width: bp.width, height: bp.height });
await page.waitForTimeout(300);
await page.screenshot({
path: `${outputDir}/${route.replace(/\//g, '_')}_${bp.name}.png`,
fullPage: true,
});
}
}
}
```
## Claude Vision Call (PRIMARY — FREE on Max 20x)
Auth via OAuth token (`~/.claude/.credentials.json.claudeAiOauth.accessToken`), NOT API key.
```typescript
async function claudeVisionCritique(screenshotPath: string, context: { route: string; breakpoint: string; brand?: BrandJson }): Promise<DualVisionResult> {
const imageData = fs.readFileSync(screenshotPath, { encoding: 'base64' });
const oauthToken = JSON.parse(fs.readFileSync(`${os.homedir()}/.claude/.credentials.json`, 'utf8')).claudeAiOauth.accessToken;
const response = await fetch('https://api.anthropic.com/v1/messages', {
method: 'POST',
headers: {
'Authorization': `Bearer ${oauthToken}`,
'anthropic-version': '2023-06-01',
'anthropic-beta': 'oauth-2025-04-20',
'Content-Type': 'application/json',
},
body: JSON.stringify({
model: 'claude-sonnet-4-6',
max_tokens: 2000,
temperature: 0,
messages: [{
role: 'user',
content: [
{ type: 'image', source: { type: 'base64', media_type: 'image/png', data: imageData } },
{ type: 'text', text: DUAL_VISION_CRITIQUE_PROMPT(context) },
],
}],
}),
});
return JSON.parse((await response.json()).content[0].text);
}
```
**Fallback:** OAuth refresh fails OR outside Max 20x → `claude-sonnet-4-6` via API key; log `auth_mode=api-key` to D1 audit, alert.
## GPT Image 2 vision Call (JUDGE — METERED, $0.50 cap)
```typescript
async function gpt4oVisionJudge(screenshotPath: string, context: { route: string; breakpoint: string; brand?: BrandJson; sourceScreenshot?: string }, budgetCents: number): Promise<DualVisionResult | null> {
if (budgetCents < 10) return null; // skip, budget exhausted
const imageData = fs.readFileSync(screenshotPath, { encoding: 'base64' });
const sourceData = context.sourceScreenshot ? fs.readFileSync(context.sourceScreenshot, { encoding: 'base64' }) : null;
const content: any[] = [{ type: 'text', text: DUAL_VISION_JUDGE_PROMPT(context) }];
content.push({ type: 'image_url', image_url: { url: `data:image/png;base64,${imageData}`, detail: 'high' } });
if (sourceData) content.push({ type: 'image_url', image_url: { url: `data:image/png;base64,${sourceData}`, detail: 'high' } });
const response = await fetch('https://api.openai.com/v1/chat/completions', {
method: 'POST',
headers: {
'Authorization': `Bearer ${await getSecret('OPENAI_API_KEY')}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({
model: 'gpt-4o',
messages: [{ role: 'user', content }],
response_format: { type: 'json_object' },
temperature: 0,
}),
});
const r = await response.json();
const costCents = Math.ceil(((r.usage?.prompt_tokens || 0) * 0.00025 + (r.usage?.completion_tokens || 0) * 0.001) / 10);
await logVisionCall({ provider: 'gpt-4o', auth_mode: 'api-key', cost_cents: costCents, route: context.route, breakpoint: context.breakpoint });
return JSON.parse(r.choices[0].message.content);
}
```
## DualVisionResult Schema
```typescript
interface DualVisionResult {
score: number; // 0-10 overall
cinematic_floor: number; // 0-10 — see visual-qa agent doctrine
latest_tech_flex: number; // 0-10 — advanced browser APIs visible
brand_fidelity: number; // 0-10 — colors/type/logo match source
hero_impact: number; // 0-10 — ATF wow factor
issues: Array<{
severity: 'critical' | 'major' | 'minor';
category: 'layout' | 'brand' | 'hierarchy' | 'typography' | 'contrast' | 'motion' | 'a11y' | 'overflow' | 'image';
breakpoint: '375' | '390' | '768' | '1024' | '1280' | '1920';
selector: string | null;
evidence: string; // MANDATORY — anti-hallucination
fix_suggestion: string;
}>;
strengths: string[];
provider: 'claude-vision' | 'gpt-4o';
cost_cents: number;
}
```
## Consensus & Arbitration
```typescript
async function dualVisionGate(screenshot: string, ctx: VisionCtx, budget: BudgetState): Promise<{ pass: boolean; remediation: Issue[] }> {
const claudeResult = await claudeVisionCritique(screenshot, ctx);
// Default: Claude Vision verdict is enough
if (claudeResult.score >= 8 && ctx.tier !== 'final-gate' && ctx.tier !== 'brand-fidelity' && ctx.tier !== 'hero-atf') {
return { pass: true, remediation: [] };
}
// Critical checkpoint OR Claude flagged issues → invoke GPT Image 2 vision judge
const gpt4oResult = budget.remainingCents >= 10 ? await gpt4oVisionJudge(screenshot, ctx, budget.remainingCents) : null;
if (!gpt4oResult) {
return { pass: claudeResult.score >= 8, remediation: claudeResult.issues.filter(i => i.severity !== 'minor') };
}
budget.remainingCents -= gpt4oResult.cost_cents;
// Both ≥ 8 → ship
if (claudeResult.score >= 8 && gpt4oResult.score >= 8) return { pass: true, remediation: [] };
// Either < 8 → fail+remediate (merge issues)
const merged = [...claudeResult.issues, ...gpt4oResult.issues];
return { pass: false, remediation: dedupeIssues(merged) };
}
```
## Vision Prompt Template (Dual-Vision)
```
You are a senior visual QA engineer auditing this screenshot of route {{route}} at breakpoint {{breakpoint}} for site {{brand.name}}.
Brand: primary={{brand.colors.primary}}, accent={{brand.colors.accent}}, font_pair={{brand.typography.headlinePair}}.
{{#if sourceScreenshot}}Source site reference attached — score brand_fidelity by comparing rendered vs source: colors, type pairing, logo placement, image treatment.{{/if}}
Score 0-10 on each:
- score (overall)
- cinematic_floor (HBO opening / Vimeo Staff Picks energy — motion above the fold, narrative beats)
- latest_tech_flex (advanced browser APIs visible — View Transitions, scroll-driven, OKLCH, :has(), container queries, anchor positioning, popover)
- brand_fidelity (colors/typography/logo match source)
- hero_impact (ATF wow factor — would a CMO say "ship it"?)
For each issue you find, include MANDATORY evidence field (selector + breakpoint + visible defect description — no claim without evidence).
Return JSON: { score, cinematic_floor, latest_tech_flex, brand_fidelity, hero_impact, issues: [...], strengths: [...] }
```
## Dual-Strategy Coverage
- **AI vision** — layout, brand, hierarchy, aesthetics, motion, cinematic feel (~57% a11y signal)
- **axe-core + Playwright a11y tree** — ARIA, screen reader, focus, keyboard (43% a11y signal)
- **Never vision alone for WCAG 2.2 AA** — always pair
## When Vision vs Accessibility Tree
- **Does it look good / layout broken / color on-brand / brand-fidelity / hero ATF / hierarchy / cinematic / latest tech?** — Claude Vision YES (free) | GPT Image 2 vision YES (judge/final) | a11y Tree NO
- **Button works / screen reader usable / form submits?** — Claude Vision NO | GPT Image 2 vision NO | a11y Tree YES (axe-core / Stagehand)
- **Apple-approved aesthetics?** — Claude Vision YES (free, gate) | GPT Image 2 vision YES (final judge) | a11y Tree NO
**Use ALL.** Tier 1 a11y tree (free, fast) → Tier 2 Claude Vision (free on Max 20x) → Tier 3 GPT Image 2 vision (metered, capped, judge).
## Auth Path
- **Container / spawned-CLI Claude Vision** — Max 20x OAuth (NEVER API key — see `auth-spawned-claude.md`). Log `auth_mode` to D1 every call; API-key fallback alert.
- **GPT Image 2 vision** — always API key (`OPENAI_API_KEY`). Log `cost_cents` per call.
## Reference Incidents
- **2026-05-09 LMG brand-fidelity regression** — code review said OK; iteration diff vision caught regression. Lesson: iteration diff vision is mandatory.
- **2026-05-10 LMG white-on-white footer** — token check passed, DOM-walker failed. Lesson: computed-DOM-walker + vision catches what static can't.
- **2026-05-12 cost discovery** — container uses Max 20x OAuth, Claude Vision FREE. Use it everywhere.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |