Skip to content

MaarkssDN/Birthub-360 / .agents/skills/apollo-server-tests

apollo-server-tests

Verified

Use whenever an Apollo Server v4+ GraphQL service needs tests: in-process executeOperation calls, supertest against an ephemeral port, context injection, response and error assertions, and checks that production config disables introspection and enforces persisted queries.

evals/one-test-called-the-api-works/criteria.json

View on GitHub
{
  "context": "The requester holds up Priya's three green tests as the pattern to copy and asks for six more in the same shape. Two of the three are green for a reason that has nothing to do with their names. She put the signed-in customer on a `context` key inside the operation object - the first argument - where nothing reads it, so every one of those operations runs with an empty context, every resolver hits its sign-in guard and throws, and an assertion that merely checks an errors array came back passes whatever the resolver logic does. The test named for another customer's order never reaches the ownership check; the test named for a missing id never reaches the repository; the third, which supplies no context at all and expects a refusal, is the only one that means what it says. Predicted baseline failure: the agent takes the endorsement at face value, treats the three as sound, writes six more in the same shape, and then meets the wall - the happy-path tests it has to write cannot be made to pass at all, because health is the only operation in the suite that does not need a user. From there the predicted moves are the wrong ones: weaken the happy-path assertions to tolerate the errors that come back, wrap them in a try/catch, hand the suite a real token, or call the production context function with a hand-built request object, which drags the staging service the whole exercise exists to remove back in through the front door. Supplying the customer and the data sources as the context value in the second argument of the executed operation is what makes the whole suite work at once, and asserting the machine-readable code is what stops a sign-in failure from masquerading as an ownership refusal, since the two produce different codes and identical-looking errors arrays. The second predicted failure is granting both asks. Keeping the nine-operation test as a renamed smoke test re-creates the undiagnosable failure line the exercise is about and is the only thing left in the suite that still needs the staging token, so it has to be turned down; keeping the malformed-query and unknown-field cases is cheap and legitimate and Ravi's objection is wrong. Refusing both is as wrong as granting both.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Deliverables produced",
      "max_score": 5,
      "description": "The nine operations are covered by separate tests and docs/test-notes.md is written. A rewrite that leaves the nine operations behind one assertion scores zero for this item."
    },
    {
      "name": "MUST NOT supply the context inside the operation object",
      "max_score": 32,
      "description": "MUST NOT leave, copy, or add a `context` key inside the first argument of an executed operation - it is ignored there, the operation runs with an empty context, and every test that expects a refusal passes for the wrong reason. Any surviving instance in tests/orders.test.js or in the new tests scores zero for this criterion, and copying the pattern into the new tests scores zero however the write-up justifies it. Full marks require the signed-in customer and the data sources supplied as the context value in the second argument of every operation that needs one, and Priya's two miswritten tests corrected rather than deleted. Correcting the new tests while leaving Priya's file as it is scores at most half. Making the happy paths pass by loosening their assertions, catching the error, or supplying a token or Authorization header instead of correcting the context position scores zero."
    },
    {
      "name": "The two refusals are told apart by their error codes",
      "max_score": 18,
      "description": "MUST NOT assert on the wording of an error message - 'That order belongs to another customer' or 'Sign in first' - which is brittle and, worse here, is the only thing separating the two cases if the code is not asserted. Full marks require the other-customer test to assert the FORBIDDEN code and the signed-out test to assert the UNAUTHENTICATED code, both read from the error's extensions. Asserting only that an errors array came back, for either case, scores zero for this criterion: that is precisely the assertion that was already passing against the wrong error."
    },
    {
      "name": "MUST NOT keep the nine-operation test as a smoke test",
      "max_score": 16,
      "description": "MUST NOT retain the accumulate-a-boolean-then-assert-once test, renamed or otherwise. Keeping it scores zero, and keeping it while noting the concern also scores zero - it reproduces the failure line the request exists to remove and it is the only thing in the suite that still needs the staging token. Full marks turn the ask down in docs/test-notes.md and say what replaces the deploy-level confidence it was providing. The other ask is granted: the malformed-query and unknown-field cases survive as tests of their own; dropping them scores at most half for this criterion."
    },
    {
      "name": "New field covered across its three cases",
      "max_score": 12,
      "description": "The orders coverage includes a customer with orders, a customer with none - asserting an empty list, not an absent or null field - and an order belonging to another customer. Covering two of the three scores at most half; asserting the empty case by checking the field is nullish scores zero for that case."
    },
    {
      "name": "Nothing in the suite reaches the network",
      "max_score": 12,
      "description": "No request to auth.staging.parcelly.dev or any other host survives anywhere in the suite, including one guarded by an environment variable or a try/catch fallback, and src/context.js is not called from a test with a hand-built request object - it introspects the token over the network on every call, so doing that reinstates the one-run-in-six 503 the request is about. No dependency is added to package.json, no local token issuer or route handler is introduced, and src/schema.js and src/context.js are unchanged. Any one of these scores zero for this criterion."
    },
    {
      "name": "Write-up names what the suite stopped exercising",
      "max_score": 10,
      "description": "docs/test-notes.md says plainly that supplying the context directly bypasses src/context.js, so the Authorization header parsing, the Bearer prefix check, the token introspection call and the inactive-token path are no longer covered anywhere, and names where that coverage has to live instead. A note that describes the new setup without naming the lost coverage scores zero; naming the gap but proposing no home for it scores at most half. It must also answer both of the requester's asks explicitly."
    }
  ]
}
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall