paradigmxyz/centaur / .agents/skills/e2e-testing
e2e-testing
VerifiedWrite, run, and debug Centaur's end-to-end tests under e2e/. Use when adding or changing an e2e scenario, running the e2e stack locally, or investigating a failing e2e CI job.
Centaur E2E Testing
How the system works
e2e/stack.sh deploys the real chart's Slack path (Postgres, console, api-rs, slackbotv2, sandboxes, per-sandbox iron-proxies) to a dedicated kind cluster, with two fakes standing in for the outside world:
e2e/fakes/fake-slack.ts: a stateful Slack Web/Events/Interactivity API. slackbotv2 runs unmodified against it.e2e/fakes/model-server.ts: a scripted model provider. CoreDNS sends provider hostnames to it, so real harnesses reach it through iron-proxy. It answers by a token in the user's message and records every request.
Tests in e2e/tests/*.test.ts run under bun test on the host and reach the fakes, Postgres, and api-rs through NodePorts. e2e/lib/index.ts is the test vocabulary: slack, model, api, cluster, workflows, ironControl, and Thread. Shared Slack identities live in e2e/fakes/slack-fixture.ts. Stack configuration lives in e2e/infra/.
CI (.github/workflows/e2e.yml, called by publish-images.yml and runnable manually against any published tag) runs every scenario against the published images with API keys, and runs harnesses.test.ts alone in subscription (access_token) mode.
Running
e2e/stack.sh up # build images from the tree, deploy
e2e/stack.sh test # all scenarios
e2e/stack.sh test e2e/tests/turns.test.ts # one file
e2e/stack.sh logs # pods, events, component logs
e2e/stack.sh down
The script uses a private kubeconfig, never the ambient context. Rerun up after changing service code. CENTAUR_E2E_IMAGE_TAG pulls published images, and CENTAUR_E2E_AUTH_MODE=access_token selects subscription auth. Run down before switching from access_token back to api_key.
Writing a scenario
- Act only through the doors a user or operator has: Slack for input and visible output, durable session tables, recorded model requests, api-rs operator routes, and Kubernetes faults. Do not reach into service internals.
- Script each turn with
model.says(...)ormodel.fails(...), then drive it withslack.mention(...),thread.mention(...), andthread.nextTurn().nextTurnwaits for a terminal execution and exactly one finished Slack reply. - Assert observable outcomes:
turn.reply,turn.execution.status,turn.sandbox, andturn.request(what the harness actually sent the provider). - Use
test.concurrentwith aturnTimeoutMs-based timeout. Each test makes its own thread and script tokens, so tests must not share state. - Wait with
eventually(...), not fixed sleeps. UsedelayMsplusthread.inFlight()to keep a turn running. - Put a short comment at the top of each file stating the behavior it guarantees. Add a scenario to the existing file that covers that area before creating a new file.
- Extend
e2e/libor a fake only when a scenario needs a new capability, and name helpers in user terms. When a test needs new stack configuration, add it toe2e/infra/values.yaml, and keep fixture values aligned withstack.sh. - E2E scenarios are slow. Cover cross-service behavior here and leave logic that a unit test can reach to unit tests.
Debugging failures
Failures name the execution and the replies seen. Next, check e2e/stack.sh logs (CI dumps the same output), then the session_executions and session_events rows for the thread key, then the model server's recorded requests.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |