Skip to content

heymegabyte/agent-skills / 07-quality-and-verification

quality-and-verification

Unverified

5-level verification pyramid: static→unit→Playwright E2E (homepage-first, 6bp)→AI visual→post-deploy. 8-check quality gate. Multi-agent testing (functional/security/a11y/performance). Playwright v1.56+ AI agents (Planner/Generator/Healer, v1.63 current). WCAG 2.2 AA via axe-core v4.13. Percy+Chromatic visual regression. ADA Title II 2027/2028 deadlines.

FileKindSize
SKILL.mdskill md5.5 KB
accessibility-gate.mdmarkdown5.8 KB
adversarial-testing.mdmarkdown5.5 KB
agentic-security.mdmarkdown3.4 KB
audio-video-sync.mdmarkdown1.7 KB
build-breaking-rules.mdmarkdown17.0 KB
chrome-and-browser-workflows.mdmarkdown5.5 KB
completeness-verification.mdmarkdown5.3 KB
computer-use-automation.mdmarkdown6.0 KB
contract-testing.mdmarkdown4.3 KB
e2e-accumulation.mdmarkdown1.6 KB
eval-driven-development.mdmarkdown6.3 KB
evidence-collection.mdmarkdown3.7 KB
llm-evals.mdmarkdown15.7 KB
performance-optimization.mdmarkdown4.8 KB
picovoice-eagle-biometric.mdmarkdown1.6 KB
security-hardening.mdmarkdown8.7 KB
semgrep-codebase-rules.mdmarkdown4.2 KB
slop-detection.mdmarkdown4.3 KB
spec-driven-development.mdmarkdown5.0 KB
stagehand-ai-fallback.mdmarkdown1.6 KB
stagehand-ai-testing.mdmarkdown7.7 KB
tdd-verification.mdmarkdown6.7 KB
testing-matrices.mdmarkdown5.3 KB
ui-completeness-sweep.mdmarkdown6.3 KB
visual-inspection-loop.mdmarkdown14.0 KB
visual-regression.mdmarkdown7.0 KB
wcag-2-2-2026.mdmarkdown3.2 KB

contract-testing.md

View on GitHub
---
name: "Contract Testing"
version: "2.0.0"
updated: "2026-04-23"
description: "Zod schema validation against live Hono API endpoints. Runtime type checking for API responses. No mocks — hit real API, validate real response. Drizzle v1 schema-to-Zod pattern. Vitest integration."
---

# Contract Testing

## Core Pattern

- Zod = source of truth (see 05/api-design)
- Contract tests verify live endpoints conform at runtime
- No mocks — hit the real API, validate the real response

```
Define Zod schema → fetch live endpoint → z.parse(response) → fail on mismatch
```

## Drizzle v1 Schema-to-Zod (Auto-Derive Contracts)

Drizzle v1 (`drizzle-zod`) generates Zod schemas directly from table definitions — contracts stay in sync with DB schema automatically:

```typescript
import { pgTable, text, timestamp } from 'drizzle-orm/pg-core';
import { createSelectSchema, createInsertSchema } from 'drizzle-zod';

export const users = pgTable('users', {
  id: text('id').primaryKey(),
  email: text('email').notNull(),
  createdAt: timestamp('created_at').defaultNow(),
  role: text('role', { enum: ['user', 'admin'] }).notNull().default('user'),
});

// Auto-derived from table — updates when schema changes
export const UserSelectSchema = createSelectSchema(users);
export const UserInsertSchema = createInsertSchema(users);
export type User = typeof UserSelectSchema._type;

// API response wraps table row — extend rather than duplicate
export const UserResponseSchema = UserSelectSchema.omit({ createdAt: true }).extend({
  createdAt: z.string().datetime(), // JSON serialized timestamp
});
export const UserListSchema = paginatedContract(UserResponseSchema);
```

## Shared Schema Pattern

```
src/shared/schemas.ts     ← Drizzle tables + drizzle-zod derived schemas
  ├── used by Hono route  ← @hono/zod-validator (insert schema)
  ├── used by Angular     ← z.infer<typeof Schema> for types
  └── used by contracts   ← select schema for runtime validation
```

## Contract Runner

```typescript
// tests/contracts/contract-runner.ts
const BASE_URL = process.env.PROD_URL ?? 'http://localhost:8787';

export async function runContractTest(tc: {
  name: string; method: 'GET'|'POST'|'PUT'|'DELETE'|'PATCH';
  path: string; body?: unknown; headers?: Record<string, string>;
  expectedStatus: number; schema: z.ZodType;
}): Promise<{ passed: boolean; errors?: z.ZodError }> {
  const res = await fetch(`${BASE_URL}${tc.path}`, {
    method: tc.method,
    headers: { 'Content-Type': 'application/json', ...tc.headers },
    body: tc.body ? JSON.stringify(tc.body) : undefined,
  });
  if (res.status !== tc.expectedStatus) return { passed: false };
  const result = tc.schema.safeParse(await res.json());
  return result.success ? { passed: true } : { passed: false, errors: result.error };
}
```

## Standard Contracts (every project)

```typescript
export const HealthContract = z.object({ status: z.enum(['ok','degraded','down']), version: z.string(), timestamp: z.string().datetime() });
export const ErrorContract = z.object({ error: z.string(), code: z.string().optional(), details: z.unknown().optional() });
export function paginatedContract<T extends z.ZodType>(item: T) {
  return z.object({ data: z.array(item), total: z.number().int().nonneg(), page: z.number().int().positive(), pageSize: z.number().int().positive(), hasMore: z.boolean() });
}
```

## Vitest Suite

```typescript
describe('API Contracts', () => {
  it('GET /health', async () => expect((await runContractTest({ name:'health', method:'GET', path:'/health', expectedStatus:200, schema:HealthContract })).passed).toBe(true));
  it('GET /api/nonexistent returns 404 envelope', async () => expect((await runContractTest({ name:'404', method:'GET', path:'/api/nonexistent-xyz', expectedStatus:404, schema:ErrorContract })).passed).toBe(true));
});
```

## When to Run

- **Post-deploy** — always (verify live API)
- **PR checks** — against staging URL
- **Nightly** — production
- **Never pre-deploy** (schemas may not match yet)
- **Never mock API responses** (defeats the purpose)

## Anti-Patterns

### Never

- Mock responses
- Skip error envelope validation
- Test against localhost in CI
- Hardcode test data that changes

### Always

- Test success + error paths
- Validate pagination contracts
- Run against real deployed URL
- Derive Drizzle contracts from table schemas
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall