Skip to content

alexandervashurin/semaphore / .claude/skills/gstack

gstack

Flagged

Fast headless browser for QA testing and site dogfooding. Navigate any URL, interact with elements, verify page state, diff before/after actions, take annotated screenshots, check responsive layouts, test forms and uploads, handle dialogs, and assert element states. ~100ms per command. Use when you need to test a feature, verify a deployment, dogfood a user flow, or file a bug with evidence. gstack also includes development workflow skills. When you notice the user is at these stages, suggest the appropriate skill: - Brainstorming a new idea → suggest /office-hours - Reviewing a plan (strategy) → suggest /plan-ceo-review - Reviewing a plan (architecture) → suggest /plan-eng-review - Reviewing a plan (design) → suggest /plan-design-review - Creating a design system → suggest /design-consultation - Debugging errors → suggest /investigate - Testing the app → suggest /qa - Code review before merge → suggest /review - Visual design audit → suggest /design-review - Ready to deploy / create PR → suggest /ship - Post-ship doc updates → suggest /document-release - Weekly retrospective → suggest /retro - Wanting a second opinion or adversarial code review → suggest /codex - Working with production or live systems → suggest /careful - Want to scope edits to one module/directory → suggest /freeze - Maximum safety mode (destructive warnings + edit restrictions) → suggest /guard - Removing edit restrictions → suggest /unfreeze - Upgrading gstack to latest version → suggest /gstack-upgrade If the user pushes back on skill suggestions ("stop suggesting things", "I don't need suggestions", "too aggressive"): 1. Stop suggesting for the rest of this session 2. Run: gstack-config set proactive false 3. Say: "Got it — I'll stop suggesting skills. Just tell me to be proactive again if you change your mind." If the user says "be proactive again" or "turn on suggestions": 1. Run: gstack-config set proactive true 2. Say: "Proactive suggestions are back on."

FileKindSize
.env.exampleother171 B
.github/workflows/skill-docs.ymlconfig648 B
.gitignoreother137 B
AGENTS.mdmarkdown2.5 KB
ARCHITECTURE.mdmarkdown20.7 KB
BROWSER.mdmarkdown16.5 KB
CHANGELOG.mdmarkdown73.3 KB
CLAUDE.mdmarkdown12.8 KB
CONTRIBUTING.mdmarkdown15.1 KB
ETHOS.mdmarkdown5.5 KB
LICENSEother1.0 KB
README.mdmarkdown21.5 KB
SKILL.mdskill md27.2 KB
SKILL.md.tmplother8.4 KB
TODOS.mdmarkdown21.9 KB
VERSIONother8 B
bin/dev-setupother2.4 KB
bin/dev-teardownother1.6 KB
bin/gstack-analyticsother6.3 KB
bin/gstack-community-dashboardother4.7 KB
bin/gstack-configother1.1 KB
bin/gstack-diff-scopeother2.4 KB
bin/gstack-review-logother409 B
bin/gstack-review-readother526 B
bin/gstack-slugother478 B
bin/gstack-telemetry-logother6.7 KB
bin/gstack-telemetry-syncother4.4 KB
bin/gstack-update-checkother6.9 KB
conductor.jsonconfig87 B
docs/images/github-2013.pngbinary61.9 KB
docs/images/github-2026.pngbinary59.1 KB
docs/skills.mdmarkdown44.2 KB
package.jsonmanifest3.1 KB
scripts/analytics.tsscript5.3 KB
scripts/dev-skill.tsscript2.4 KB
scripts/eval-compare.tsscript3.0 KB
scripts/eval-list.tsscript3.5 KB
scripts/eval-select.tsscript3.1 KB
scripts/eval-summary.tsscript6.4 KB
scripts/eval-watch.tsscript5.4 KB
scripts/gen-skill-docs.tsscript89.5 KB
scripts/skill-check.tsscript5.8 KB
setupother8.4 KB
supabase/config.shscript454 B
supabase/functions/community-pulse/index.tsscript1.7 KB
supabase/functions/telemetry-ingest/index.tsscript4.1 KB
supabase/functions/update-check/index.tsscript1.1 KB
supabase/migrations/001_telemetry.sqlother3.2 KB
test/analytics.test.tsscript10.2 KB
test/codex-e2e.test.tsscript7.5 KB
test/fixtures/eval-baselines.jsonconfig388 B
test/fixtures/qa-eval-checkout-ground-truth.jsonconfig1.6 KB
test/fixtures/qa-eval-ground-truth.jsonconfig1.5 KB
test/fixtures/qa-eval-spa-ground-truth.jsonconfig1.6 KB
test/fixtures/review-eval-design-slop.cssother1.5 KB
test/fixtures/review-eval-design-slop.htmlother1.3 KB
test/fixtures/review-eval-enum-diff.rbscript986 B
test/fixtures/review-eval-enum.rbscript759 B
test/fixtures/review-eval-vuln.rbscript406 B
test/gemini-e2e.test.tsscript6.3 KB
test/gen-skill-docs.test.tsscript37.3 KB
test/helpers/codex-session-runner.tsscript8.9 KB
test/helpers/e2e-helpers.tsscript9.5 KB
test/helpers/eval-store.test.tsscript19.8 KB
test/helpers/eval-store.tsscript23.6 KB
test/helpers/gemini-session-runner.test.tsscript4.4 KB
test/helpers/gemini-session-runner.tsscript6.0 KB
test/helpers/llm-judge.tsscript4.2 KB
test/helpers/observability.test.tsscript9.9 KB
test/helpers/session-runner.test.tsscript3.7 KB
test/helpers/session-runner.tsscript11.6 KB
test/helpers/skill-parser.tsscript5.6 KB
test/helpers/touchfiles.tsscript11.0 KB
test/hook-scripts.test.tsscript13.6 KB
test/skill-e2e-browse.test.tsscript11.8 KB
test/skill-e2e-deploy.test.tsscript11.2 KB
test/skill-e2e-design.test.tsscript24.4 KB
test/skill-e2e-plan.test.tsscript19.7 KB
test/skill-e2e-qa-bugs.test.tsscript8.1 KB
test/skill-e2e-qa-workflow.test.tsscript15.4 KB
test/skill-e2e-review.test.tsscript23.2 KB
test/skill-e2e-workflow.test.tsscript23.2 KB
test/skill-llm-eval.test.tsscript33.6 KB
test/skill-parser.test.tsscript5.3 KB
test/skill-routing-e2e.test.tsscript26.3 KB
test/skill-validation.test.tsscript54.5 KB
test/telemetry.test.tsscript10.3 KB
test/touchfiles.test.tsscript10.1 KB

test/skill-routing-e2e.test.ts

View on GitHub
import { describe, test, expect, afterAll } from 'bun:test';
import { runSkillTest } from './helpers/session-runner';
import type { SkillTestResult } from './helpers/session-runner';
import { EvalCollector } from './helpers/eval-store';
import type { EvalTestEntry } from './helpers/eval-store';
import { selectTests, detectBaseBranch, getChangedFiles, E2E_TOUCHFILES, GLOBAL_TOUCHFILES } from './helpers/touchfiles';
import { spawnSync } from 'child_process';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';

const ROOT = path.resolve(import.meta.dir, '..');

// Skip unless EVALS=1.
const evalsEnabled = !!process.env.EVALS;
const describeE2E = evalsEnabled ? describe : describe.skip;

// Eval result collector
const evalCollector = evalsEnabled ? new EvalCollector('e2e-routing') : null;

// Unique run ID for this session
const runId = new Date().toISOString().replace(/[:.]/g, '').replace('T', '-').slice(0, 15);

// --- Diff-based test selection ---
// Journey routing tests use E2E_TOUCHFILES (entries prefixed 'journey-' in touchfiles.ts).
let selectedTests: string[] | null = null;

if (evalsEnabled && !process.env.EVALS_ALL) {
  const baseBranch = process.env.EVALS_BASE
    || detectBaseBranch(ROOT)
    || 'main';
  const changedFiles = getChangedFiles(baseBranch, ROOT);

  if (changedFiles.length > 0) {
    const selection = selectTests(changedFiles, E2E_TOUCHFILES, GLOBAL_TOUCHFILES);
    selectedTests = selection.selected;
    process.stderr.write(`\nRouting E2E selection (${selection.reason}): ${selection.selected.length}/${Object.keys(E2E_TOUCHFILES).length} tests\n`);
    if (selection.skipped.length > 0) {
      process.stderr.write(`  Skipped: ${selection.skipped.join(', ')}\n`);
    }
    process.stderr.write('\n');
  }
}

// --- Helper functions ---

/** Copy all SKILL.md files into tmpDir/.claude/skills/gstack/ for auto-discovery */
function installSkills(tmpDir: string) {
  const skillDirs = [
    '', // root gstack SKILL.md
    'qa', 'qa-only', 'ship', 'review', 'plan-ceo-review', 'plan-eng-review',
    'plan-design-review', 'design-review', 'design-consultation', 'retro',
    'document-release', 'investigate', 'office-hours', 'browse', 'setup-browser-cookies',
    'gstack-upgrade', 'humanizer',
  ];

  for (const skill of skillDirs) {
    const srcPath = path.join(ROOT, skill, 'SKILL.md');
    if (!fs.existsSync(srcPath)) continue;

    const destDir = skill
      ? path.join(tmpDir, '.claude', 'skills', 'gstack', skill)
      : path.join(tmpDir, '.claude', 'skills', 'gstack');
    fs.mkdirSync(destDir, { recursive: true });
    fs.copyFileSync(srcPath, path.join(destDir, 'SKILL.md'));
  }
}

/** Init a git repo with config */
function initGitRepo(dir: string) {
  const run = (cmd: string, args: string[]) =>
    spawnSync(cmd, args, { cwd: dir, stdio: 'pipe', timeout: 5000 });
  run('git', ['init']);
  run('git', ['config', 'user.email', 'test@test.com']);
  run('git', ['config', 'user.name', 'Test']);
}

function logCost(label: string, result: { costEstimate: { turnsUsed: number; estimatedTokens: number; estimatedCost: number }; duration: number }) {
  const { turnsUsed, estimatedTokens, estimatedCost } = result.costEstimate;
  const durationSec = Math.round(result.duration / 1000);
  console.log(`${label}: $${estimatedCost.toFixed(2)} (${turnsUsed} turns, ${(estimatedTokens / 1000).toFixed(1)}k tokens, ${durationSec}s)`);
}

function recordRouting(name: string, result: SkillTestResult, expectedSkill: string, actualSkill: string | undefined) {
  evalCollector?.addTest({
    name,
    suite: 'Skill Routing E2E',
    tier: 'e2e',
    passed: actualSkill === expectedSkill,
    duration_ms: result.duration,
    cost_usd: result.costEstimate.estimatedCost,
    transcript: result.transcript,
    output: result.output?.slice(0, 2000),
    turns_used: result.costEstimate.turnsUsed,
    exit_reason: result.exitReason,
  });
}

// --- Tests ---

describeE2E('Skill Routing E2E — Developer Journey', () => {
  afterAll(() => {
    evalCollector?.finalize();
  });

  test.concurrent('journey-ideation', async () => {
    const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'routing-ideation-'));
    try {
      initGitRepo(tmpDir);
      installSkills(tmpDir);
      fs.writeFileSync(path.join(tmpDir, 'README.md'), '# New Project\n');
      spawnSync('git', ['add', '.'], { cwd: tmpDir, stdio: 'pipe', timeout: 5000 });
      spawnSync('git', ['commit', '-m', 'initial'], { cwd: tmpDir, stdio: 'pipe', timeout: 5000 });

      const testName = 'journey-ideation';
      const expectedSkill = 'office-hours';
      const result = await runSkillTest({
        prompt: "I've been thinking about building a waitlist management tool for restaurants. The existing solutions are expensive and overcomplicated. I want something simple — a tablet app where hosts can add parties, see wait times, and text customers when their table is ready. Help me think through whether this is worth building and what the key design decisions are.",
        workingDirectory: tmpDir,
        maxTurns: 5,
        allowedTools: ['Skill', 'Read', 'Bash', 'Glob', 'Grep'],
        timeout: 60_000,
        testName,
        runId,
      });

      const skillCalls = result.toolCalls.filter(tc => tc.tool === 'Skill');
      const actualSkill = skillCalls.length > 0 ? skillCalls[0]?.input?.skill : undefined;

      logCost(`journey: ${testName}`, result);
      recordRouting(testName, result, expectedSkill, actualSkill);

      expect(skillCalls.length, `Expected Skill tool to be called but got 0 calls. Claude may have answered directly without invoking a skill. Tool calls: ${result.toolCalls.map(tc => tc.tool).join(', ')}`).toBeGreaterThan(0);
      expect([expectedSkill], `Expected skill ${expectedSkill} but got ${actualSkill}`).toContain(actualSkill);
    } finally {
      fs.rmSync(tmpDir, { recursive: true, force: true });
    }
  }, 150_000);

  test.concurrent('journey-plan-eng', async () => {
    const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'routing-plan-eng-'));
    try {
      initGitRepo(tmpDir);
      installSkills(tmpDir);
      fs.writeFileSync(path.join(tmpDir, 'plan.md'), `# Waitlist App Architecture

## Components
- REST API (Express.js)
- PostgreSQL database
- React frontend
- SMS integration (Twilio)

## Data Model
- restaurants (id, name, settings)
- parties (id, restaurant_id, name, size, phone, status, created_at)
- wait_estimates (id, restaurant_id, avg_wait_minutes)

## API Endpoints
- POST /api/parties - add party to waitlist
- GET /api/parties - list current waitlist
- PATCH /api/parties/:id/status - update party status
- GET /api/estimate - get current wait estimate
`);
      spawnSync('git', ['add', '.'], { cwd: tmpDir, stdio: 'pipe', timeout: 5000 });
      spawnSync('git', ['commit', '-m', 'initial'], { cwd: tmpDir, stdio: 'pipe', timeout: 5000 });

      const testName = 'journey-plan-eng';
      const expectedSkill = 'plan-eng-review';
      const result = await runSkillTest({
        prompt: "I wrote up a plan for the waitlist app in plan.md. Can you take a look at the architecture and make sure I'm not missing any edge cases or failure modes before I start coding?",
        workingDirectory: tmpDir,
        maxTurns: 5,
        allowedTools: ['Skill', 'Read', 'Bash', 'Glob', 'Grep'],
        timeout: 60_000,
        testName,
        runId,
      });

      const skillCalls = result.toolCalls.filter(tc => tc.tool === 'Skill');
      const actualSkill = skillCalls.length > 0 ? skillCalls[0]?.input?.skill : undefined;

      logCost(`journey: ${testName}`, result);
      recordRouting(testName, result, expectedSkill, actualSkill);

      expect(skillCalls.length, `Expected Skill tool to be called but got 0 calls. Claude may have answered directly without invoking a skill. Tool calls: ${result.toolCalls.map(tc => tc.tool).join(', ')}`).toBeGreaterThan(0);
      expect([expectedSkill], `Expected skill ${expectedSkill} but got ${actualSkill}`).toContain(actualSkill);
    } finally {
      fs.rmSync(tmpDir, { recursive: true, force: true });
    }
  }, 150_000);

  test.concurrent('journey-think-bigger', async () => {
    const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'routing-think-bigger-'));
    try {
      initGitRepo(tmpDir);
      installSkills(tmpDir);
      fs.writeFileSync(path.join(tmpDir, 'plan.md'), `# Waitlist App Architecture

## Components
- REST API (Express.js)
- PostgreSQL database
- React frontend
- SMS integration (Twilio)

## Data Model
- restaurants (id, name, settings)
- parties (id, restaurant_id, name, size, phone, status, created_at)
- wait_estimates (id, restaurant_id, avg_wait_minutes)

## API Endpoints
- POST /api/parties - add party to waitlist
- GET /api/parties - list current waitlist
- PATCH /api/parties/:id/status - update party status
- GET /api/estimate - get current wait estimate
`);
      spawnSync('git', ['add', '.'], { cwd: tmpDir, stdio: 'pipe', timeout: 5000 });
      spawnSync('git', ['commit', '-m', 'initial'], { cwd: tmpDir, stdio: 'pipe', timeout: 5000 });

      const testName = 'journey-think-bigger';
      const expectedSkill = 'plan-ceo-review';
      const result = await runSkillTest({
        prompt: "Actually, looking at this plan again, I feel like we're thinking too small. We're just doing waitlists but what about the whole restaurant guest experience? Is there a bigger opportunity here we should go after?",
        workingDirectory: tmpDir,
        maxTurns: 5,
        allowedTools: ['Skill', 'Read', 'Bash', 'Glob', 'Grep'],
        timeout: 120_000,
        testName,
        runId,
      });

      const skillCalls = result.toolCalls.filter(tc => tc.tool === 'Skill');
      const actualSkill = skillCalls.length > 0 ? skillCalls[0]?.input?.skill : undefined;

      logCost(`journey: ${testName}`, result);
      recordRouting(testName, result, expectedSkill, actualSkill);

      expect(skillCalls.length, `Expected Skill tool to be called but got 0 calls. Claude may have answered directly without invoking a skill. Tool calls: ${result.toolCalls.map(tc => tc.tool).join(', ')}`).toBeGreaterThan(0);
      expect([expectedSkill], `Expected skill ${expectedSkill} but got ${actualSkill}`).toContain(actualSkill);
    } finally {
      fs.rmSync(tmpDir, { recursive: true, force: true });
    }
  }, 180_000);

  test.concurrent('journey-debug', async () => {
    const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'routing-debug-'));
    try {
      initGitRepo(tmpDir);
      installSkills(tmpDir);

      const run = (cmd: string, args: string[]) =>
        spawnSync(cmd, args, { cwd: tmpDir, stdio: 'pipe', timeout: 5000 });

      fs.mkdirSync(path.join(tmpDir, 'src'), { recursive: true });
      fs.writeFileSync(path.join(tmpDir, 'src/api.ts'), `
import express from 'express';
const app = express();

app.get('/api/waitlist', async (req, res) => {
  const db = req.app.locals.db;
  const parties = await db.query('SELECT * FROM parties WHERE status = $1', ['waiting']);
  res.json(parties.rows);
});

export default app;
`);
      fs.writeFileSync(path.join(tmpDir, 'error.log'), `
[2026-03-18T10:23:45Z] ERROR: GET /api/waitlist - 500 Internal Server Error
  TypeError: Cannot read properties of undefined (reading 'query')
    at /src/api.ts:5:32
    at Layer.handle [as handle_request] (/node_modules/express/lib/router/layer.js:95:5)
[2026-03-18T10:23:46Z] ERROR: GET /api/waitlist - 500 Internal Server Error
  TypeError: Cannot read properties of undefined (reading 'query')
`);

      run('git', ['add', '.']);
      run('git', ['commit', '-m', 'initial']);
      run('git', ['checkout', '-b', 'feature/waitlist-api']);

      const testName = 'journey-debug';
      const expectedSkill = 'investigate';
      const result = await runSkillTest({
        prompt: "The GET /api/waitlist endpoint was working fine yesterday but now it's returning 500 errors. The tests are passing locally but the endpoint fails when I hit it with curl. Can you figure out what's going on?",
        workingDirectory: tmpDir,
        maxTurns: 5,
        allowedTools: ['Skill', 'Read', 'Bash', 'Glob', 'Grep'],
        timeout: 60_000,
        testName,
        runId,
      });

      const skillCalls = result.toolCalls.filter(tc => tc.tool === 'Skill');
      const actualSkill = skillCalls.length > 0 ? skillCalls[0]?.input?.skill : undefined;

      logCost(`journey: ${testName}`, result);
      recordRouting(testName, result, expectedSkill, actualSkill);

      expect(skillCalls.length, `Expected Skill tool to be called but got 0 calls. Claude may have answered directly without invoking a skill. Tool calls: ${result.toolCalls.map(tc => tc.tool).join(', ')}`).toBeGreaterThan(0);
      expect([expectedSkill], `Expected skill ${expectedSkill} but got ${actualSkill}`).toContain(actualSkill);
    } finally {
      fs.rmSync(tmpDir, { recursive: true, force: true });
    }
  }, 150_000);

  test.concurrent('journey-qa', async () => {
    const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'routing-qa-'));
    try {
      initGitRepo(tmpDir);
      installSkills(tmpDir);

      fs.writeFileSync(path.join(tmpDir, 'package.json'), JSON.stringify({ name: 'waitlist-app', scripts: { dev: 'next dev' } }, null, 2));
      fs.mkdirSync(path.join(tmpDir, 'src'), { recursive: true });
      fs.writeFileSync(path.join(tmpDir, 'src/index.html'), '<html><body><h1>Waitlist App</h1></body></html>');
      spawnSync('git', ['add', '.'], { cwd: tmpDir, stdio: 'pipe', timeout: 5000 });
      spawnSync('git', ['commit', '-m', 'initial'], { cwd: tmpDir, stdio: 'pipe', timeout: 5000 });

      const testName = 'journey-qa';
      const expectedSkill = 'qa';
      const alternateSkills = ['qa-only', 'browse'];
      const result = await runSkillTest({
        prompt: "I think the app is mostly working now. Can you go through the site and test everything — find any bugs and fix them?",
        workingDirectory: tmpDir,
        maxTurns: 5,
        allowedTools: ['Skill', 'Read', 'Bash', 'Glob', 'Grep'],
        timeout: 60_000,
        testName,
        runId,
      });

      const skillCalls = result.toolCalls.filter(tc => tc.tool === 'Skill');
      const actualSkill = skillCalls.length > 0 ? skillCalls[0]?.input?.skill : undefined;
      const acceptable = [expectedSkill, ...alternateSkills];

      logCost(`journey: ${testName}`, result);
      recordRouting(testName, result, expectedSkill, actualSkill);

      expect(skillCalls.length, `Expected Skill tool to be called but got 0 calls. Claude may have answered directly without invoking a skill. Tool calls: ${result.toolCalls.map(tc => tc.tool).join(', ')}`).toBeGreaterThan(0);
      expect(acceptable, `Expected skill ${expectedSkill} but got ${actualSkill}`).toContain(actualSkill);
    } finally {
      fs.rmSync(tmpDir, { recursive: true, force: true });
    }
  }, 150_000);

  test.concurrent('journey-code-review', async () => {
    const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'routing-code-review-'));
    try {
      initGitRepo(tmpDir);
      installSkills(tmpDir);

      const run = (cmd: string, args: string[]) =>
        spawnSync(cmd, args, { cwd: tmpDir, stdio: 'pipe', timeout: 5000 });

      fs.writeFileSync(path.join(tmpDir, 'app.ts'), '// base\n');
      run('git', ['add', '.']);
      run('git', ['commit', '-m', 'initial']);
      run('git', ['checkout', '-b', 'feature/add-waitlist']);
      fs.writeFileSync(path.join(tmpDir, 'app.ts'), '// updated with waitlist feature\nimport { WaitlistService } from "./waitlist";\n');
      fs.writeFileSync(path.join(tmpDir, 'waitlist.ts'), 'export class WaitlistService {\n  async addParty(name: string, size: number) {\n    // TODO: implement\n  }\n}\n');
      run('git', ['add', '.']);
      run('git', ['commit', '-m', 'feat: add waitlist service']);

      const testName = 'journey-code-review';
      const expectedSkill = 'review';
      const result = await runSkillTest({
        prompt: "I'm about to merge this into main. Can you look over my changes and flag anything risky before I land it?",
        workingDirectory: tmpDir,
        maxTurns: 5,
        allowedTools: ['Skill', 'Read', 'Bash', 'Glob', 'Grep'],
        timeout: 120_000,
        testName,
        runId,
      });

      const skillCalls = result.toolCalls.filter(tc => tc.tool === 'Skill');
      const actualSkill = skillCalls.length > 0 ? skillCalls[0]?.input?.skill : undefined;

      logCost(`journey: ${testName}`, result);
      recordRouting(testName, result, expectedSkill, actualSkill);

      expect(skillCalls.length, `Expected Skill tool to be called but got 0 calls. Claude may have answered directly without invoking a skill. Tool calls: ${result.toolCalls.map(tc => tc.tool).join(', ')}`).toBeGreaterThan(0);
      expect([expectedSkill], `Expected skill ${expectedSkill} but got ${actualSkill}`).toContain(actualSkill);
    } finally {
      fs.rmSync(tmpDir, { recursive: true, force: true });
    }
  }, 150_000);

  test.concurrent('journey-ship', async () => {
    const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'routing-ship-'));
    try {
      initGitRepo(tmpDir);
      installSkills(tmpDir);

      const run = (cmd: string, args: string[]) =>
        spawnSync(cmd, args, { cwd: tmpDir, stdio: 'pipe', timeout: 5000 });

      fs.writeFileSync(path.join(tmpDir, 'app.ts'), '// base\n');
      run('git', ['add', '.']);
      run('git', ['commit', '-m', 'initial']);
      run('git', ['checkout', '-b', 'feature/waitlist']);
      fs.writeFileSync(path.join(tmpDir, 'app.ts'), '// waitlist feature\n');
      run('git', ['add', '.']);
      run('git', ['commit', '-m', 'feat: waitlist']);

      const testName = 'journey-ship';
      const expectedSkill = 'ship';
      const result = await runSkillTest({
        prompt: "This looks good. Let's get it deployed — push the code up and create a PR.",
        workingDirectory: tmpDir,
        maxTurns: 5,
        allowedTools: ['Skill', 'Read', 'Bash', 'Glob', 'Grep'],
        timeout: 60_000,
        testName,
        runId,
      });

      const skillCalls = result.toolCalls.filter(tc => tc.tool === 'Skill');
      const actualSkill = skillCalls.length > 0 ? skillCalls[0]?.input?.skill : undefined;

      logCost(`journey: ${testName}`, result);
      recordRouting(testName, result, expectedSkill, actualSkill);

      expect(skillCalls.length, `Expected Skill tool to be called but got 0 calls. Claude may have answered directly without invoking a skill. Tool calls: ${result.toolCalls.map(tc => tc.tool).join(', ')}`).toBeGreaterThan(0);
      expect([expectedSkill], `Expected skill ${expectedSkill} but got ${actualSkill}`).toContain(actualSkill);
    } finally {
      fs.rmSync(tmpDir, { recursive: true, force: true });
    }
  }, 150_000);

  test.concurrent('journey-docs', async () => {
    const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'routing-docs-'));
    try {
      initGitRepo(tmpDir);
      installSkills(tmpDir);

      const run = (cmd: string, args: string[]) =>
        spawnSync(cmd, args, { cwd: tmpDir, stdio: 'pipe', timeout: 5000 });

      fs.writeFileSync(path.join(tmpDir, 'README.md'), '# Waitlist App\nA simple waitlist management tool.\n');
      fs.mkdirSync(path.join(tmpDir, 'src'), { recursive: true });
      fs.writeFileSync(path.join(tmpDir, 'src/api.ts'), '// API code\n');
      run('git', ['add', '.']);
      run('git', ['commit', '-m', 'feat: ship waitlist feature']);

      const testName = 'journey-docs';
      const expectedSkill = 'document-release';
      const result = await runSkillTest({
        prompt: "We just shipped the waitlist feature. Can you go through the README and any other docs and make sure they match what we actually built?",
        workingDirectory: tmpDir,
        maxTurns: 5,
        allowedTools: ['Skill', 'Read', 'Bash', 'Glob', 'Grep'],
        timeout: 60_000,
        testName,
        runId,
      });

      const skillCalls = result.toolCalls.filter(tc => tc.tool === 'Skill');
      const actualSkill = skillCalls.length > 0 ? skillCalls[0]?.input?.skill : undefined;

      logCost(`journey: ${testName}`, result);
      recordRouting(testName, result, expectedSkill, actualSkill);

      expect(skillCalls.length, `Expected Skill tool to be called but got 0 calls. Claude may have answered directly without invoking a skill. Tool calls: ${result.toolCalls.map(tc => tc.tool).join(', ')}`).toBeGreaterThan(0);
      expect([expectedSkill], `Expected skill ${expectedSkill} but got ${actualSkill}`).toContain(actualSkill);
    } finally {
      fs.rmSync(tmpDir, { recursive: true, force: true });
    }
  }, 150_000);

  test.concurrent('journey-retro', async () => {
    const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'routing-retro-'));
    try {
      initGitRepo(tmpDir);
      installSkills(tmpDir);

      const run = (cmd: string, args: string[]) =>
        spawnSync(cmd, args, { cwd: tmpDir, stdio: 'pipe', timeout: 5000 });

      fs.writeFileSync(path.join(tmpDir, 'api.ts'), 'export function getParties() { return []; }\n');
      run('git', ['add', '.']);
      run('git', ['commit', '-m', 'feat: add parties API', '--date', '2026-03-12T09:30:00']);

      fs.writeFileSync(path.join(tmpDir, 'ui.tsx'), 'export function WaitlistView() { return <div>Waitlist</div>; }\n');
      run('git', ['add', '.']);
      run('git', ['commit', '-m', 'feat: add waitlist UI', '--date', '2026-03-13T14:00:00']);

      fs.writeFileSync(path.join(tmpDir, 'README.md'), '# Waitlist App\n');
      run('git', ['add', '.']);
      run('git', ['commit', '-m', 'docs: add README', '--date', '2026-03-14T16:00:00']);

      const testName = 'journey-retro';
      const expectedSkill = 'retro';
      const result = await runSkillTest({
        prompt: "It's Friday. What did we ship this week? I want to do a quick retrospective on what the team accomplished.",
        workingDirectory: tmpDir,
        maxTurns: 5,
        allowedTools: ['Skill', 'Read', 'Bash', 'Glob', 'Grep'],
        timeout: 120_000,
        testName,
        runId,
      });

      const skillCalls = result.toolCalls.filter(tc => tc.tool === 'Skill');
      const actualSkill = skillCalls.length > 0 ? skillCalls[0]?.input?.skill : undefined;

      logCost(`journey: ${testName}`, result);
      recordRouting(testName, result, expectedSkill, actualSkill);

      expect(skillCalls.length, `Expected Skill tool to be called but got 0 calls. Claude may have answered directly without invoking a skill. Tool calls: ${result.toolCalls.map(tc => tc.tool).join(', ')}`).toBeGreaterThan(0);
      expect([expectedSkill], `Expected skill ${expectedSkill} but got ${actualSkill}`).toContain(actualSkill);
    } finally {
      fs.rmSync(tmpDir, { recursive: true, force: true });
    }
  }, 150_000);

  test.concurrent('journey-design-system', async () => {
    const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'routing-design-system-'));
    try {
      initGitRepo(tmpDir);
      installSkills(tmpDir);

      const run = (cmd: string, args: string[]) =>
        spawnSync(cmd, args, { cwd: tmpDir, stdio: 'pipe', timeout: 5000 });

      fs.writeFileSync(path.join(tmpDir, 'package.json'), JSON.stringify({ name: 'waitlist-app' }, null, 2));
      run('git', ['add', '.']);
      run('git', ['commit', '-m', 'initial']);

      const testName = 'journey-design-system';
      const expectedSkill = 'design-consultation';
      const result = await runSkillTest({
        prompt: "Before we build the UI, I want to establish a design system — typography, colors, spacing, the whole thing. Can you put together brand guidelines for this project?",
        workingDirectory: tmpDir,
        maxTurns: 5,
        allowedTools: ['Skill', 'Read', 'Bash', 'Glob', 'Grep'],
        timeout: 60_000,
        testName,
        runId,
      });

      const skillCalls = result.toolCalls.filter(tc => tc.tool === 'Skill');
      const actualSkill = skillCalls.length > 0 ? skillCalls[0]?.input?.skill : undefined;

      logCost(`journey: ${testName}`, result);
      recordRouting(testName, result, expectedSkill, actualSkill);

      expect(skillCalls.length, `Expected Skill tool to be called but got 0 calls. Claude may have answered directly without invoking a skill. Tool calls: ${result.toolCalls.map(tc => tc.tool).join(', ')}`).toBeGreaterThan(0);
      expect([expectedSkill], `Expected skill ${expectedSkill} but got ${actualSkill}`).toContain(actualSkill);
    } finally {
      fs.rmSync(tmpDir, { recursive: true, force: true });
    }
  }, 150_000);

  test.concurrent('journey-visual-qa', async () => {
    const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'routing-visual-qa-'));
    try {
      initGitRepo(tmpDir);
      installSkills(tmpDir);

      const run = (cmd: string, args: string[]) =>
        spawnSync(cmd, args, { cwd: tmpDir, stdio: 'pipe', timeout: 5000 });

      fs.mkdirSync(path.join(tmpDir, 'src'), { recursive: true });
      fs.writeFileSync(path.join(tmpDir, 'src/styles.css'), `
body { font-family: sans-serif; }
.header { font-size: 24px; margin: 20px; }
.card { padding: 16px; margin: 8px; border: 1px solid #ccc; }
.button { background: #007bff; color: white; padding: 10px 20px; }
`);
      fs.writeFileSync(path.join(tmpDir, 'src/index.html'), `
<html>
<head><link rel="stylesheet" href="styles.css"></head>
<body>
  <div class="header">Waitlist</div>
  <div class="card">Party of 4 - Smith</div>
  <div class="card">Party of 2 - Jones</div>
</body>
</html>
`);
      run('git', ['add', '.']);
      run('git', ['commit', '-m', 'initial UI']);

      const testName = 'journey-visual-qa';
      const expectedSkill = 'design-review';
      const result = await runSkillTest({
        prompt: "Something looks off on the site. The spacing between sections is inconsistent and the font sizes don't feel right. Can you audit the visual design and fix anything that doesn't look polished?",
        workingDirectory: tmpDir,
        maxTurns: 5,
        allowedTools: ['Skill', 'Read', 'Bash', 'Glob', 'Grep'],
        timeout: 60_000,
        testName,
        runId,
      });

      const skillCalls = result.toolCalls.filter(tc => tc.tool === 'Skill');
      const actualSkill = skillCalls.length > 0 ? skillCalls[0]?.input?.skill : undefined;

      logCost(`journey: ${testName}`, result);
      recordRouting(testName, result, expectedSkill, actualSkill);

      expect(skillCalls.length, `Expected Skill tool to be called but got 0 calls. Claude may have answered directly without invoking a skill. Tool calls: ${result.toolCalls.map(tc => tc.tool).join(', ')}`).toBeGreaterThan(0);
      expect([expectedSkill], `Expected skill ${expectedSkill} but got ${actualSkill}`).toContain(actualSkill);
    } finally {
      fs.rmSync(tmpDir, { recursive: true, force: true });
    }
  }, 150_000);
});
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 467kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 467kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 467kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 193kUpdated Claude CodeCodexVerifiedInstall