sb-dev/openquant / .agents/skills/property-based-testing
property-based-testing
VerifiedWrites, reviews, and debugs property-based tests — Hypothesis, fast-check, proptest, jqwik, rapid, and Echidna or Medusa for Solidity invariants. Use whenever tests should cover a whole input domain instead of a hand-picked list of examples: encode/decode and serialize/deserialize pairs, parsers, canonicalizers and normalizers, validators, numeric and Decimal types, comparators and sort order, data structures, and smart-contract state invariants. Also use when adding cases to an existing @given, fast-check, or proptest suite, when judging whether existing property tests assert anything real, and when a generator has shrunk a counterexample and you need to tell a wrong property from a genuine bug. Not for coverage-guided binary fuzzing (libFuzzer, AFL), mutation-testing campaigns, static analysis, benchmarking, or end-to-end UI tests.
| File | Kind | Size |
|---|---|---|
| README.md | markdown | 15.0 KB |
| SKILL.md | skill md | 4.2 KB |
| agents/openai.yaml | config | 254 B |
| assets/trail-of-bits-mark.svg | other | 3.0 KB |
| references/generating.md | markdown | 2.6 KB |
| references/interpreting-failures.md | markdown | 3.3 KB |
| references/libraries.md | markdown | 2.9 KB |
| references/refactoring.md | markdown | 4.7 KB |
| references/reviewing.md | markdown | 2.8 KB |
references/generating.md
View on GitHub# Generating Property-Based Tests
Writing the `@given` decorator is the easy part. These are the decisions that make
the difference between a suite that finds bugs and one that just runs.
## Put constraints in the strategy, not in `assume()`
This is the single highest-value habit. `assume()` discards inputs after generation,
so a filter that rejects most candidates wastes the budget and eventually trips
Hypothesis's exhausted-filter guard — which surfaces as a warning nobody reads.
```python
# Slow, and mostly discards
@given(st.integers())
def test_positive(x):
assume(x > 0)
...
# Generates only what you want
@given(st.integers(min_value=1))
def test_positive(x):
...
```
Reserve `assume()` for conditions you genuinely cannot express as a generator — a
relationship between two already-generated values, usually.
Build compound inputs with `st.builds`, and derive dependent fields with
`st.composite` or `.flatmap` rather than generating independently and filtering:
```python
@st.composite
def sized_list_and_index(draw):
xs = draw(st.lists(st.integers(), min_size=1))
i = draw(st.integers(min_value=0, max_value=len(xs) - 1))
return xs, i
```
## Pin the edge cases you already know about
Random generation finds boundaries eventually; `@example` finds them on every run and
documents that you thought about them.
```python
@given(st.lists(st.integers()))
@example([])
@example([1])
@example([1, 1, 1])
def test_sort(xs): ...
```
Empty, single-element, all-duplicates, zero, negative, and the maximum representable
value are the ones that recur.
## Settings
Defaults (100 examples, 200ms deadline) are wrong at both ends of the workflow:
```python
@settings(max_examples=10) # local iteration
@settings(max_examples=200) # CI
@settings(max_examples=1000, deadline=None) # nightly
```
Set `deadline=None` for anything doing real work — the default deadline turns a slow
machine into a failing test, and that flake gets the whole suite deleted.
## Determinism as a property
`f(x) == f(x)` is a tautology for a pure function and a real test for anything else.
Serializers over dicts or sets, anything involving hashing, iteration order, or time —
those can and do return different output for the same input. Assert it where a broken
implementation could falsify it, and not otherwise.
## Testing the error path
`st.binary()` against a decoder is worth writing: the contract is usually "raises
`DecodeError` or succeeds, never `IndexError`, never hangs". Catch only the documented
exception and let everything else fail the test.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |