Skip to content

alivirgo/Major-AI-Skills / plugins/mas-ai-workflows/skills/agent-tool-replay-test

agent-tool-replay-test

Verified

Replay recorded agent tool/LLM trajectories against deterministic fixtures to test argument validation, errors, idempotency, and side-effect boundaries—separate from live tool-selection quality.

FileKindSize
SKILL.mdskill md3.9 KB
gemini.mdmarkdown608 B
gpt.mdmarkdown802 B
---
title: "Agent Tool Replay Test (GPT & Codex)"
description: "Wire record/replay cassettes, schema assertions, and idempotent mutating-tool tests for agent CI."
category: "AI Workflows / Testing"
tags: ["record-replay", "cassette", "tools", "gpt-codex"]
---

# Agent Tool Replay Test (GPT & Codex)

Implement `CASSETTE_MODE=replay` in CI (fail on miss). Record sanitized trajectories in PRs when prompts/tools change. Assert tool sequences and argument JSON Schema. For payments/messages: simulate timeout then assert reconcile/idempotent retry—not double execute.

```python
# Pseudocode policy
assert "process_refund" not in calls or approval_hash_ok
assert call.args["idempotency_key"]  # mutating tools
```

> **MANDATORY**: No prod credentials in cassettes. Replay ≠ live selection quality.
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 323kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 323kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 323kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall