Skip to content

paradigmxyz/centaur / .agents/skills/e2e-testing

e2e-testing

Verified

Write, run, and debug Centaur's end-to-end tests under e2e/. Use when adding or changing an e2e scenario, running the e2e stack locally, or investigating a failing e2e CI job.

FileKindSize
SKILL.mdskill md3.8 KB
---
name: e2e-testing
description: "Write, run, and debug Centaur's end-to-end tests under e2e/. Use when adding or changing an e2e scenario, running the e2e stack locally, or investigating a failing e2e CI job."
---

# Centaur E2E Testing

## How the system works

`e2e/stack.sh` deploys the real chart's Slack path (Postgres, console, api-rs, slackbotv2, sandboxes, per-sandbox iron-proxies) to a dedicated kind cluster, with two fakes standing in for the outside world:

- `e2e/fakes/fake-slack.ts`: a stateful Slack Web/Events/Interactivity API. slackbotv2 runs unmodified against it.
- `e2e/fakes/model-server.ts`: a scripted model provider. CoreDNS sends provider hostnames to it, so real harnesses reach it through iron-proxy. It answers by a token in the user's message and records every request.

Tests in `e2e/tests/*.test.ts` run under `bun test` on the host and reach the fakes, Postgres, and api-rs through NodePorts. `e2e/lib/index.ts` is the test vocabulary: `slack`, `model`, `api`, `cluster`, `workflows`, `ironControl`, and `Thread`. Shared Slack identities live in `e2e/fakes/slack-fixture.ts`. Stack configuration lives in `e2e/infra/`.

CI (`.github/workflows/e2e.yml`, called by `publish-images.yml` and runnable manually against any published tag) runs every scenario against the published images with API keys, and runs `harnesses.test.ts` alone in subscription (`access_token`) mode.

## Running

```bash
e2e/stack.sh up                                   # build images from the tree, deploy
e2e/stack.sh test                                 # all scenarios
e2e/stack.sh test e2e/tests/turns.test.ts         # one file
e2e/stack.sh logs                                 # pods, events, component logs
e2e/stack.sh down
```

The script uses a private kubeconfig, never the ambient context. Rerun `up` after changing service code. `CENTAUR_E2E_IMAGE_TAG` pulls published images, and `CENTAUR_E2E_AUTH_MODE=access_token` selects subscription auth. Run `down` before switching from `access_token` back to `api_key`.

## Writing a scenario

- Act only through the doors a user or operator has: Slack for input and visible output, durable session tables, recorded model requests, api-rs operator routes, and Kubernetes faults. Do not reach into service internals.
- Script each turn with `model.says(...)` or `model.fails(...)`, then drive it with `slack.mention(...)`, `thread.mention(...)`, and `thread.nextTurn()`. `nextTurn` waits for a terminal execution and exactly one finished Slack reply.
- Assert observable outcomes: `turn.reply`, `turn.execution.status`, `turn.sandbox`, and `turn.request` (what the harness actually sent the provider).
- Use `test.concurrent` with a `turnTimeoutMs`-based timeout. Each test makes its own thread and script tokens, so tests must not share state.
- Wait with `eventually(...)`, not fixed sleeps. Use `delayMs` plus `thread.inFlight()` to keep a turn running.
- Call `releaseSandboxesAfterAll()` at the top level of every test file. A thread's sandbox and iron-proxy pod otherwise outlive the test by hours, and the single kind node runs out of pods.
- Put a short comment at the top of each file stating the behavior it guarantees. Add a scenario to the existing file that covers that area before creating a new file.
- Extend `e2e/lib` or a fake only when a scenario needs a new capability, and name helpers in user terms. When a test needs new stack configuration, add it to `e2e/infra/values.yaml`, and keep fixture values aligned with `stack.sh`.
- E2E scenarios are slow. Cover cross-service behavior here and leave logic that a unit test can reach to unit tests.

## Debugging failures

Failures name the execution and the replies seen. Next, check `e2e/stack.sh logs` (CI dumps the same output), then the `session_executions` and `session_events` rows for the thread key, then the model server's recorded requests.
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall