Skip to content

nWave-ai/nWave / nWave/skills/nw-solution-architect-formal-verification

nw-solution-architect-formal-verification

Verified

Load in the model step to decide which design claims need a formal model, pick Quint or TLA+, and write, check and mutate-test it.

Formal model step

1. Decide before writing

  1. List every design claim about state, protocol, ordering, concurrency, retry, crash recovery, progress (liveness), refinement (implementation matches spec) or reachability. Mark each "model" or "no model".
  2. Model when: several actors can change the same state; a protocol has more than one step; a retry, crash or restart must leave state consistent; something must eventually happen; an implementation must refine a spec.
  3. No model when the claim names no shared state, no multi-step protocol, no concurrency or crash window, and examples cover its transitions. A pure function with a stated law (totality, round trip, canonical form, ordering) is still a claim: check it (Quint over the transition function, or Agda).
  4. Write the reason as No model for <claim>: <the specific state, protocol, concurrency or crash window that is absent>. "Simple", "small", "no tool" or "timed out" is never a reason; a missing checker, or a timed-out checker with its repair attempts recorded (item 12), is INDETERMINATE.
  5. Choose the checker per claim: Quint (default: quint run, then quint verify); TLA+ with TLC when the claim needs liveness under fairness, PlusCal processes, or Quint's backend cannot finish; Agda only for proofs over types. Use two checkers when one cannot cover the claim. Reuse an existing model of the same protocol.
  6. The closing statement maps every modelled claim to the checker that ran on it.

2. Write the model

  1. One file per model in the repository at the path the design names (<name>.qnt, or <name>.tla plus <name>.cfg), complete; never paste it into the design text.
  2. Declare state variables, init, one action per atomic step, every invariant, every liveness property. Model only the concurrency, failure and crash steps the claim needs, two processes first; leave out logging, serialization, UI.
  3. Name each atom after one runtime symbol (class, function, field). Closing statement: atom -> symbol, atoms with none as model-only, and what the model does not cover (timing, throughput, unbounded scheduling). A step count is not time; claim no latency or availability from a model.

3. Check it

  1. Check the tool exists (quint --version, java -version). Run from a copy outside the repository so _apalache-out and _build stay out of the tree. quint typecheck, then each invariant. Fix syntax errors and rerun.
  2. Use the real checker even on a tiny state space; a Python enumerator or hand trace is not a check.
  3. Budget each run at 180 seconds. Start with two processes and the fewest steps reaching the seam; grow to three, then failures, then liveness. A timed-out bound is not verified and is not a stopping point: diagnose it before any outcome. Name the observed cause (state or branching growth, redundant histories or variables, solver or translation behaviour, checker or harness configuration, a budget too small for a sound model) apart from any hypothesis, then apply the remedy it supports: raise the budget and run the unchanged check when the budget is what is too small; change the checker strategy or configuration, or fix a checker or harness fault; or reduce the model to a narrower slice recorded as its own claim (never presented as the original claim) with an argument that it keeps the property and the failure cases, interleavings included. Record each attempt with its bound, budget and result. Never rerun a timed-out bound unchanged at the same budget. The repair is bounded: declare the budget and the attempt limit before the first run, and make each attempt a different remedy that the diagnosis supports.
  4. After the recorded repair attempts, a claim the checker still cannot verify within the budget is INDETERMINATE with the tool name and the bounds tried, never PASS. It is never the first answer to a timeout, and never becomes accepted without the verification. A missing or unexecuted check is INDETERMINATE with the tool name; there is no run to diagnose. A resource, tool or authorization limit that stops the repair is reported with its evidence and the decision it needs; a timeout alone is never a request to waive the claim or to publish part of the design. Continue the other claims.
  5. Commit the model only, not output, logs or traces.

4. One witness per invariant

  1. Per invariant, copy the model and remove the exact behaviour it guards (lock check, idempotency guard, rollback). Leave invariant and flags unchanged.
  2. Run quint run MODEL.qnt --invariant=INV on the mutant (a counterexample is a kill); run quint verify only if none is found. The invariant must fail. A tool error or timeout is not a kill.
  3. One mutant per invariant. An invariant no mutant breaks may be vacuous: strengthen it.
  4. Delete the mutant after the run.
  5. Each invariant also gets a default-suite unit or property test of the same rule on the pure core, including the sad path (repeated step, interrupted step, refused stale write). Faults the projection cannot reach stay in the model.

5. Liveness

  1. Per liveness property give an executable witness (a test or model run reaching the good state) or no liveness claim: <concrete reason>.
  2. Declare fairness only where the result depends on it: weakFair(action, vars) or strongFair(action, vars) on the real action, naming whose scheduling you assume.
  3. A fairness-dependent property needs three TLC runs (quint verify MODEL.qnt --init=init --step=step --temporal=PROP --backend tlc): without fairness it fails with a stuttering counterexample; with fairness it passes; with fairness on a mutant removing the progress or release step it fails.
  4. A property that holds without fairness: state which step is always enabled and why.
  5. Slice finitely but keep the state the property needs; name what the slice drops.

6. Self-check

  1. Each claim has a named checker or a No model for reason naming the absent window.
  2. Model files complete in the repository, no run output.
  3. Each invariant failed on its own mutant and has a default-suite test.
  4. Each liveness property has a witness or reason; fairness only where needed.
  5. No timed-out or unexecuted check is called a pass; every timed-out check shows its recorded repair attempts.
  6. Closing statement lists claim -> checker, atom -> symbol, model-only atoms, and what is not covered.
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 321kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 321kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 321kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 197kUpdated Claude CodeCodexVerifiedInstall