Skip to content

deanpeters/ai-product-operating-model-skills / skills/aipom-behavior-contract-builder

aipom-behavior-contract-builder

Verified

Define expected, acceptable, and prohibited AI behavior with representative cases, thresholds, escalation, and consequences. Use before evaluation, launch, or autonomy decisions.

AIPOM Behavior Contract Builder

What Is It

Define how an AI product should behave, the range of acceptable variation, what it must refuse or escalate, and what happens when behavior falls outside the contract. The contract connects product intent to evaluation and operating decisions.

Why Use It

Teams cannot evaluate “helpful,” “safe,” or “accurate” consistently without observable expectations and consequences. A contract makes disagreement inspectable before it becomes a launch dispute or incident. Writing it does not prove conformance; representative evaluations and production evidence do.

When to Use It

Use before building an evaluation set, setting autonomy boundaries, launching a probabilistic feature, or revising behavior after incidents or user evidence. It is not a legal compliance determination or a substitute for specialist review.

What It Produces

  • Purpose and scope
  • Expected, acceptable, escalated, and prohibited behaviors
  • Representative normal, edge, adversarial, and failure cases
  • Thresholds tied to ship, continue, pause, rollback, or retire decisions
  • Owners, evidence gaps, and review triggers

Who Should Participate

Include the Product Manager, technical and evaluation owners, design or research, workflow operators, and the person accountable for affected outcomes. Add legal, privacy, security, safety, accessibility, or domain specialists in proportion to consequences.

Evidence to Bring

Bring user research, intended-use statements, real inputs, known failures, policies, evaluation results, complaints, incidents, and affected-user perspectives. Label hypothetical cases and unsupported thresholds as assumptions.

How to Do It

  1. Set scope. Name the system version, users, affected people, contexts, and decision the contract must support.
  2. Define purpose. State the user outcome and boundaries; avoid capability marketing.
  3. Describe behaviors. Use observable input-condition-output-consequence statements.
  4. Classify behavior. Mark each as expected, acceptable variation, escalate/refuse, or prohibited.
  5. Add cases. Cover frequent use, difficult ambiguity, affected groups, misuse, adversarial inputs, and consequential failure.
  6. Set evidence rules. Name measures, human judgment, sampling, thresholds, and limitations. Avoid fake precision.
  7. Connect decisions. Specify which evidence triggers ship, continuation, correction, pause, rollback, or retirement.
  8. Assign ownership. Name who interprets evidence, decides, remains accountable, and approves changes.

AI may organize cases and draft language. Humans decide acceptable consequences, thresholds, and authority.

Key Concepts

  • Behavior, not intention: describe what users can observe.
  • Acceptable range: probabilistic systems require bounded variation, not one ideal answer.
  • Consequence-aware evaluation: weight failures by who is affected and what follows.
  • Contract evolution: version behavior when evidence or context changes.

Organizational Applications

Use for assistants, recommendations, classifiers, generated content, decision support, and agents. Reuse the contract across design reviews, evaluation datasets, autonomy decisions, production monitoring, and incident learning.

Common Pitfalls

  • Using adjectives without observable cases
  • Testing only happy paths
  • Treating an average score as permission to ignore severe failures
  • Setting thresholds without evidence or a decision
  • Omitting affected people, refusals, escalation, or owners
  • Treating the document as proof of reliable behavior

Combine With

Use aipom-autonomy-boundary-designer to decide authority, aipom-golden-dataset-builder to operationalize representative cases, and aipom-eval-scorecard-builder to define measurement and decision rules.

Assets and Templates

Sources

  • NIST, AI Risk Management Framework 1.0, January 26, 2023. Supports lifecycle risk mapping, measurement, management, and accountable governance. Accessed July 16, 2026. NIST notes that the framework is being revised.
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall