jaktestowac/awesome-copilot-for-testers / plugins/testing-api-contracts/skills/testing-api-contracts
testing-api-contracts
VerifiedValidates API responses against OpenAPI or JSON Schema, detects breaking changes between spec versions, and builds consumer-driven contract checks. Use when an API has a published spec, when a backend change might break a client, when API tests assert only status codes, or when mocked fixtures need a guard against drifting from the real service.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 9.1 KB |
| resources/breaking-change-checklist.md | markdown | 4.8 KB |
| resources/contract-coverage-checklist.md | markdown | 3.4 KB |
| resources/contract-report-template.md | markdown | 4.0 KB |
| resources/schema-validation-recipes.md | markdown | 6.6 KB |
SKILL.md
View on GitHub---
name: testing-api-contracts
description: 'Validates API responses against OpenAPI or JSON Schema, detects breaking changes between spec versions, and builds consumer-driven contract checks. Use when an API has a published spec, when a backend change might break a client, when API tests assert only status codes, or when mocked fixtures need a guard against drifting from the real service.'
argument-hint: 'OpenAPI or JSON Schema path or URL, the endpoints in scope, consumer or provider side, and the previous spec version if a diff is wanted'
user-invocable: true
---
# Testing API Contracts
Use this skill when the question is not "does the endpoint work" but "does the endpoint still promise what its consumers were built against".
A contract test checks **shape and compatibility**, not business behaviour. It answers: did a field disappear, did a type change, did an enum gain a value the client cannot handle, did a required field become optional. Functional API tests answer whether the value is correct. Both are needed and they fail for different reasons, which is the point of keeping them separate.
## When to Use
- an API publishes an OpenAPI or JSON Schema document and nothing verifies the implementation matches it
- API tests assert `status === 200` and nothing about the body
- a backend change is about to ship and no one can say which clients it breaks
- mocked fixtures need a drift guard against the real service
- two teams are arguing about whether a change is breaking
- a spec exists but has visibly diverged from what the service returns
## Operating Principles
- **The spec is the contract, the implementation is the claim.** Test the claim against the contract, in both directions.
- **Shape checks and value checks stay separate.** Mixing them produces a test that fails for two unrelated reasons and reports one message.
- **Additive is safe, subtractive is not.** Adding an optional response field breaks nobody. Removing one, or tightening a type, breaks everybody who read it.
- **Validate every response, not a sampled one.** Schema validation is cheap enough to run on every API call the suite already makes.
- **A breaking change is defined by consumers, not by the provider.** A field no client reads can be removed freely; the argument is about evidence, not opinion.
- **Undocumented behaviour is a finding.** A response field absent from the spec is either a spec gap or an accidental leak. Both get reported.
## Workflow
### Phase 0: Locate the contract
Find the authoritative artifact and record which it is:
- an OpenAPI document in the repository, generated from code or hand-written
- an OpenAPI document served at runtime (`/openapi.json`, `/swagger.json`, `/v3/api-docs`)
- standalone JSON Schema files
- a GraphQL SDL
- **none of the above**, in which case the first deliverable is a spec derived from observed traffic, marked as derived rather than authoritative
If the spec is hand-written and the implementation is not generated from it, say so. Hand-written specs drift, and the drift is exactly what this skill exists to surface.
### Phase 1: Decide which side you are testing
| Side | Question | Typical artifact |
| --- | --- | --- |
| **Provider** | Does our service return what our spec promises? | Schema validation over live responses |
| **Consumer** | Do we still work against what the provider promises? | Fixture validation plus a live sample check |
| **Both** | Are these two versions of the spec compatible? | Spec diff and breaking-change report |
Choose one per run. A single output that mixes provider defects with consumer risks is unreadable to either team.
### Phase 2: Wire validation into requests already being made
The cheapest contract coverage is a validator attached to the existing API test suite. Every request the suite already sends gets its response validated for free.
Recipes for `ajv`, `zod`, `openapi-response-validator`, and a Playwright `expect` extension are in `./resources/schema-validation-recipes.md`.
Rules for the validator:
- **Strict by default.** `additionalProperties: false` when the spec allows it, so undocumented fields surface instead of passing silently.
- **Validate the error responses too.** The 4xx body shape is part of the contract and is where drift is most common.
- **Report the path, not just "invalid".** `/items/0/price: expected number, got string` is actionable; `schema validation failed` is not.
- **Fail the test.** A validator that logs a warning gets ignored within a week.
### Phase 3: Cover the shape, not the sample
For each endpoint in scope, check the cases in `./resources/contract-coverage-checklist.md`. The ones that catch real defects:
- **empty collection** - `items: []` with the pagination envelope intact
- **nullable fields actually null** - the spec says nullable, so make one null
- **optional fields absent** - not present rather than present-and-null; these are different contracts
- **maximum-length and boundary values** - the field the spec caps at 255 characters
- **every enum member** - a client switch statement that has no default breaks on the value you never sent
- **error responses** - one per documented status code
- **pagination edges** - first page, last page, past the end
### Phase 4: Diff the versions
When a spec has changed, produce a compatibility verdict rather than a list of edits. Classify every change using `./resources/breaking-change-checklist.md`:
| Class | Examples | Verdict |
| --- | --- | --- |
| **Safe** | New optional response field, new endpoint, new optional request parameter, widened response enum a client treats as opaque | Ship |
| **Risky** | New enum value in a field clients branch on, tightened validation, changed default, changed ordering, changed pagination size | Ship with a consumer check |
| **Breaking** | Removed or renamed field, narrowed type, required field added to a request, optional response field made absent, status code change, auth requirement change | Version it or coordinate |
For every Risky and Breaking item, name the affected consumers if they are knowable, and say what evidence supports the claim. "Unknown consumers" is a valid finding and raises the severity.
### Phase 5: Guard the mocks
If the codebase mocks this API anywhere, the spec is also the guard for those fixtures. Validate the fixtures against the same schema:
- fixtures that fail validation are stale and are reported as defects in the test suite, not in the API
- run this check on the same cadence as the spec changes, not on every commit
This is the handoff point with `mocking-network-and-time`.
### Phase 6: Report
Use `./resources/contract-report-template.md`. A useful report contains:
- which spec, which version, which side was tested
- endpoints covered and endpoints skipped, with the reason for skipping
- validation failures grouped by endpoint, with the JSON path and both values
- the compatibility verdict per change, with severity
- undocumented fields found in responses
- what remains unverified
## Common Failure Modes
- asserting only the status code and calling it an API test
- validating the happy path and never the documented error bodies
- `additionalProperties` left permissive, so a leaked internal field passes forever
- diffing two specs and reporting every line changed rather than classifying compatibility
- calling a change non-breaking because no test failed, when no test covered it
- treating a generated spec as proof of correctness; a spec generated from the code agrees with the code by construction
- one report that mixes provider bugs, consumer risks, and spec gaps
## Resource Map
- `./resources/schema-validation-recipes.md` - `ajv`, `zod`, `openapi-response-validator`, and a reusable Playwright matcher
- `./resources/breaking-change-checklist.md` - full classification of spec changes into safe, risky, and breaking, for requests and responses
- `./resources/contract-coverage-checklist.md` - the response shapes to force per endpoint
- `./resources/contract-report-template.md` - report structure with a worked example
## Related Skills
- `api-playwright-test-developer` - when the functional API tests these validators attach to are being written
- `mocking-network-and-time` - when the spec is being used to guard mocked fixtures against drift
- `analyzing-regression-scope` - when a breaking change needs its retest blast radius mapped
- `verifying-acceptance-criteria` - when the question is whether behaviour meets its stated contract rather than its shape
- `assessing-release-readiness` - when a breaking change feeds a go/no-go decision
- `documenting-test-suites` - when the spec gaps found here need writing up
## Definition of Done
This skill is complete when:
- the authoritative spec is identified and its trustworthiness stated
- the side under test (provider, consumer, or version diff) is declared
- validation runs on every response the suite produces, including error responses, and fails the test on mismatch
- the shape cases from the coverage checklist are forced rather than hoped for
- every spec change carries a safe, risky, or breaking classification with its rationale
- undocumented response fields are reported
- mocked fixtures for this API are validated against the same schema, or their absence is noted
- the report says what was not covered
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |