Skip to content

heymegabyte/agent-skills / 07-quality-and-verification

quality-and-verification

Unverified

5-level verification pyramid: static→unit→Playwright E2E (homepage-first, 6bp)→AI visual→post-deploy. 8-check quality gate. Multi-agent testing (functional/security/a11y/performance). Playwright v1.56+ AI agents (Planner/Generator/Healer, v1.63 current). WCAG 2.2 AA via axe-core v4.13. Percy+Chromatic visual regression. ADA Title II 2027/2028 deadlines.

FileKindSize
SKILL.mdskill md5.5 KB
accessibility-gate.mdmarkdown5.8 KB
adversarial-testing.mdmarkdown5.5 KB
agentic-security.mdmarkdown3.4 KB
audio-video-sync.mdmarkdown1.7 KB
build-breaking-rules.mdmarkdown17.0 KB
chrome-and-browser-workflows.mdmarkdown5.5 KB
completeness-verification.mdmarkdown5.3 KB
computer-use-automation.mdmarkdown6.0 KB
contract-testing.mdmarkdown4.3 KB
e2e-accumulation.mdmarkdown1.6 KB
eval-driven-development.mdmarkdown6.3 KB
evidence-collection.mdmarkdown3.7 KB
llm-evals.mdmarkdown15.7 KB
performance-optimization.mdmarkdown4.8 KB
picovoice-eagle-biometric.mdmarkdown1.6 KB
security-hardening.mdmarkdown8.7 KB
semgrep-codebase-rules.mdmarkdown4.2 KB
slop-detection.mdmarkdown4.3 KB
spec-driven-development.mdmarkdown5.0 KB
stagehand-ai-fallback.mdmarkdown1.6 KB
stagehand-ai-testing.mdmarkdown7.7 KB
tdd-verification.mdmarkdown6.7 KB
testing-matrices.mdmarkdown5.3 KB
ui-completeness-sweep.mdmarkdown6.3 KB
visual-inspection-loop.mdmarkdown14.0 KB
visual-regression.mdmarkdown7.0 KB
wcag-2-2-2026.mdmarkdown3.2 KB

evidence-collection.md

View on GitHub
---
name: "Evidence Collection"
version: "2.0.0"
updated: "2026-04-23"
description: "Every automated form submission creates a complete evidence package: Playwright video, annotated screenshots, action log, manifest.json. R2 storage patterns with presigned URLs. Accessible from user dashboard."
---

# Evidence Collection Pattern

## Per-Submission Package

Every `fillForm()` call creates:

- EvidenceCollector instance
- Screenshots at each step
- Red circle overlays on button clicks (Node canvas, 30px radius `rgba(255,0,0,0.9)`)
- Action log
- `manifest.json`
- Video via Playwright `recordVideo` API

Package uploaded to R2 at `evidence/{echoId}/{appId}/{packageId}/`.

## R2 Storage Patterns

```typescript
// Upload evidence package to R2
async function uploadEvidencePackage(env: Env, pkg: EvidencePackage): Promise<string> {
  const prefix = `evidence/${pkg.echoId}/${pkg.appId}/${pkg.packageId}`;

  // Upload screenshots in parallel — never sequential
  await Promise.all(pkg.screenshots.map(async (ss, i) => {
    await env.R2.put(`${prefix}/screenshots/${i}-${ss.label}.png`, ss.buffer, {
      httpMetadata: { contentType: 'image/png' },
      customMetadata: { label: ss.label, timestamp: ss.timestamp, buttonClicked: ss.buttonClicked ?? '' },
    });
  }));

  // Upload video (WebM from Playwright recordVideo)
  if (pkg.videoPath) {
    const videoBuffer = await readFile(pkg.videoPath);
    await env.R2.put(`${prefix}/session.webm`, videoBuffer, {
      httpMetadata: { contentType: 'video/webm' },
    });
  }

  // Upload manifest last (signals package is complete)
  await env.R2.put(`${prefix}/manifest.json`, JSON.stringify(pkg.manifest), {
    httpMetadata: { contentType: 'application/json' },
  });

  return `${prefix}/manifest.json`;
}

// Generate presigned URL for user access (1hr expiry)
async function getEvidenceUrl(env: Env, key: string): Promise<string> {
  const obj = await env.R2.get(key);
  if (!obj) throw new Error(`Evidence not found: ${key}`);
  // R2 presigned URLs via Workers — return public URL pattern
  return `${env.R2_PUBLIC_URL}/${key}`;
}

// List all packages for an application
async function listEvidencePackages(env: Env, echoId: string, appId: string) {
  const prefix = `evidence/${echoId}/${appId}/`;
  const listed = await env.R2.list({ prefix, delimiter: '/' });
  return listed.delimitedPrefixes.map(p => p.replace(prefix, '').replace('/', ''));
}
```

## Video Recording

- `context.newContext({ recordVideo: { dir, size: { width:1280, height:720 } } })` captures full session
- Video finalized on `page.close()`
- Uploaded to R2 as WebM
- Public URL returned in API response + stored in application record

## Manifest Schema

```typescript
interface EvidenceManifest {
  packageId: string; echoId: string; appId: string; program: string;
  startTime: string; endTime: string; status: 'success'|'partial'|'failed';
  screenshots: { index: number; label: string; timestamp: string; size: number; buttonClicked?: string }[];
  actions: { type: string; label: string; timestamp: string; selector?: string }[];
  videoKey?: string; // R2 key for WebM
}
```

## Access Patterns

- **Dashboard** — status cards show video play button when `videoUrl` exists
- **Your Data panel** — "Submission Videos" section lists all recordings with R2 presigned URLs
- **Chat** — AI surfaces video via `application-video` widget
- **API** — `GET /api/evidence/:appId` returns manifest + URLs

## R2 Binding (wrangler.toml)

```toml
[[r2_buckets]]
binding = "R2"
bucket_name = "evidence-bucket"
```

- **Public access:** enable R2 public development URL or custom domain for `R2_PUBLIC_URL`
- **Retention:** 90-day lifecycle rule on bucket (legal compliance without unbounded storage)
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall