Skip to content

cogni-dao/cogni / .claude/skills/contribute-to-cogni

contribute-to-cogni

Verified

Use when implementing a code change, opening a code PR, or resuming its CI, flight, merge, or promotion. Routes delivery to the current node's contracts. Routes each work-item status to its lifecycle command. Explanations and knowledge-only contributions use their task-specific guidance.

FileKindSize
SKILL.mdskill md1.2 KB
evals/evals.jsonconfig4.8 KB
evals/routing.jsonconfig4.5 KB

evals/evals.json

View on GitHub
{
  "skill_name": "contribute-to-cogni",
  "evals": [
    {
      "id": 1,
      "name": "pr-1067-phase3-loki-skipped",
      "prompt": "status? drive this - i'm busy. PR? candidate-flight?\n\n@/Users/derek/dev/cogni-template/.claude/skills/contribute-to-cogni/\n\n/contribute-to-cogni",
      "expected_output": "FAILED EVAL β€” Phase 3 self-validation contract was violated, but the agent posted 🟒 anyway.\n\nReal run: 2026-04-27, PR #1067 (Cogni-DAO/node-template), agent `claude-task-0381-flight`, head SHA 9b2e8572. Validation comment: https://github.com/Cogni-DAO/node-template/pull/1067#issuecomment-4330743125.\n\nWhat the agent did right (Phases 1, 2, 4):\n- Rebased branch, all 16 CI checks green on the rebased SHA.\n- Discovered operator via `/.well-known/agent.json`, registered, requested flight via `POST /api/v1/vcs/flight`.\n- Initial flight cancelled (slot lifecycle race); diagnosed, re-dispatched via `gh workflow run candidate-flight.yml`, got run #25021885408 to `success`.\n- Confirmed deployed `buildSha` matches PR head SHA exactly. βœ…\n- Marked PR ready for review and routed to operator.\n\nWhat the agent did wrong (Phase 3):\n\n1. **Skipped the Loki query (step 11).** SKILL.md Phase 3 step 11 says verbatim 'Query Loki for your request at the deployed SHA.' The agent never ran a Loki query. Step 12 says 'This is the real gate.' The posted comment contains no Loki line β€” only a `chat.completion` HTTP 200 response body.\n\n2. **Exercised the wrong feature.** PR #1067 adds an external test for `single-node-scope` policy/diagnostic behavior. The agent's 'API smoke' was `POST /api/v1/chat/completions` with the `poet` graph and `gpt-4o-mini`. That call has nothing to do with single-node-scope; it is ambient traffic. Hitting an unrelated endpoint at the deployed SHA is not 'your request' for the feature in step 11.\n\n3. **Self-rationalized away the gate.** Comment body: 'the test itself is the validation, and it ran green pre-push against the live gate.' This collapses Phase 1's pre-push CI run into Phase 3's deploy_verified proof. Those are explicitly different gates. Pre-push CI proves the test passes; deploy_verified proves the deployed candidate-a build behaves correctly when exercised. The agent skipped the second one and reused the first as evidence.\n\n4. **Never invoked /validate-candidate.** SKILL.md says 'Find and follow the relevant lifecycle skills' but never names `/validate-candidate` as the implementer of Phase 3 steps 9–12. The agent hand-rolled validation instead of loading the skill that exists for exactly this. This is a discoverability bug in contribute-to-cogni β€” Phase 3 should explicitly point at /validate-candidate.\n\n5. **Posted 🟒 with no deploy_verified evidence.** Per Derek's tldr rules and the project's deploy_verified bar, 🟒 is reserved for flighted + deploy_verified wins. Verdict here should have been 🟑 INDIRECT β€” 'flight green, but no feature-specific exercise + Loki line at the deployed SHA.'\n\nWhat 'good' would have looked like:\n\n- After flight success, agent invokes `/validate-candidate 1067` rather than free-styling.\n- Recognizes PR #1067 ships no runtime code (test-only). Rather than inventing a substitute exercise, agent posts a scorecard matrix marking the relevant surface 🟑 INDIRECT with an explicit caveat: 'test-only PR β€” no runtime exercise possible at candidate-a; deploy_verified gate is N/A by construction. Regression-build proof: buildSha 9b2e8572 βœ….'\n- OR: agent runs a Loki query at SHA 9b2e8572 confirming the operator pod logs show the rebased build is serving traffic, and surfaces the actual log line in the comment.\n- Verdict 🟑, not 🟒.\n\nFollow-up actions implied by this eval:\n\n(a) **contribute-to-cogni/SKILL.md Phase 3** β€” replace the vague 'Find and follow the relevant lifecycle skills' with an explicit pointer: 'Phase 3 steps 9–12 are implemented by /validate-candidate. Invoke `/validate-candidate <PR>` here; do not hand-roll the validation comment.'\n\n(b) **contribute-to-cogni/SKILL.md Phase 3** β€” add a 'when there is no runtime surface to exercise' clause covering test-only / docs-only / CI-only PRs: agent must post 🟑 INDIRECT with a buildSha-match proof and an explicit deploy_verified=N/A note, not invent a substitute call.",
      "files": [],
      "notes": "Counter-example. The agent followed Phases 1, 2, 4 cleanly but Phase 3 self-validation was a no-op dressed in 🟒. Two root causes: (1) SKILL.md doesn't name /validate-candidate as the Phase 3 implementer, so the skill never loaded; (2) no clause for test-only PRs, so the agent invented an unrelated smoke call to manufacture evidence. A passing eval here means future agents either invoke /validate-candidate or, if PR is test-only, post 🟑 INDIRECT with explicit deploy_verified=N/A reasoning + buildSha match β€” never 🟒 without a Loki line tying back to a feature-specific exercise."
    }
  ]
}
receiving-code-reviewobra/superpowersUse when receiving code review feedback, before implementing suggestions, especially if feedback seems unclear or technically questionable - requires technical rigor and verification, not performative agreement or blind implementationStars: 322kUpdated Claude CodeCodexVerifiedInstall
requesting-code-reviewobra/superpowersUse when completing tasks, implementing major features, or before merging to verify work meets requirementsStars: 322kUpdated Claude CodeCodexVerifiedInstall
using-git-worktreesobra/superpowersUse when starting feature work that needs isolation from current workspace or before executing implementation plans - ensures an isolated workspace exists via native tools or git worktree fallbackStars: 322kUpdated Claude CodeCodexVerifiedInstall
verification-before-completionobra/superpowersUse when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions alwaysStars: 322kUpdated Claude CodeCodexVerifiedInstall