Skip to content

olaafrossi/FoamPilot / .claude/skills/gstack

gstack

Flagged

Fast headless browser for QA testing and site dogfooding. Navigate pages, interact with elements, verify state, diff before/after, take annotated screenshots, test responsive layouts, forms, uploads, dialogs, and capture bug evidence. Use when asked to open or test a site, verify a deployment, dogfood a user flow, or file a bug with screenshots.

FileKindSize
.env.exampleother171 B
.github/actionlint.yamlconfig82 B
.github/docker/Dockerfile.ciother2.7 KB
.github/workflows/actionlint.ymlconfig176 B
.github/workflows/ci-image.ymlconfig1.0 KB
.github/workflows/evals-periodic.ymlconfig3.8 KB
.github/workflows/evals.ymlconfig8.1 KB
.github/workflows/skill-docs.ymlconfig1.2 KB
.gitignoreother243 B
AGENTS.mdmarkdown2.5 KB
ARCHITECTURE.mdmarkdown21.4 KB
BROWSER.mdmarkdown23.1 KB
CHANGELOG.mdmarkdown146.8 KB
CLAUDE.mdmarkdown19.2 KB
CONTRIBUTING.mdmarkdown17.6 KB
DESIGN.mdmarkdown4.4 KB
ETHOS.mdmarkdown7.3 KB
LICENSEother1.0 KB
README.mdmarkdown23.8 KB
SKILL.mdskill md26.1 KB
SKILL.md.tmplother7.8 KB
TODOS.mdmarkdown33.9 KB
VERSIONother9 B
actionlint.yamlconfig56 B
agents/openai.yamlconfig351 B
bun.lockother18.0 KB
conductor.jsonconfig87 B
design/prototype.tsscript5.4 KB
design/src/auth.tsscript1.7 KB
design/src/brief.tsscript1.9 KB
design/src/check.tsscript3.0 KB
design/src/cli.tsscript8.9 KB
design/src/commands.tsscript3.0 KB
design/src/compare.tsscript19.1 KB
design/src/design-to-code.tsscript3.0 KB
design/src/diff.tsscript3.3 KB
design/src/evolve.tsscript5.0 KB
design/src/gallery.tsscript7.0 KB
design/src/generate.tsscript4.0 KB
design/src/iterate.tsscript5.4 KB
design/src/memory.tsscript6.0 KB
design/src/serve.tsscript7.4 KB
design/src/session.tsscript1.8 KB
design/src/variants.tsscript8.4 KB
design/test/feedback-roundtrip.test.tsscript12.6 KB
design/test/gallery.test.tsscript5.0 KB
design/test/serve.test.tsscript12.6 KB
docs/designs/CHROME_VS_CHROMIUM_EXPLORATION.mdmarkdown4.0 KB
docs/designs/CONDUCTOR_CHROME_SIDEBAR_INTEGRATION.mdmarkdown3.5 KB
docs/designs/CONDUCTOR_SESSION_API.mdmarkdown3.9 KB
docs/designs/DESIGN_SHOTGUN.mdmarkdown22.5 KB
docs/designs/DESIGN_TOOLS_V1.mdmarkdown36.0 KB
docs/designs/ML_PROMPT_INJECTION_KILLER.mdmarkdown20.7 KB
docs/designs/SELF_LEARNING_V0.mdmarkdown5.5 KB
docs/images/github-2013.pngbinary61.9 KB
docs/images/github-2026.pngbinary59.1 KB
docs/skills.mdmarkdown45.3 KB
extension/background.jsscript8.1 KB
extension/content.cssother2.5 KB
extension/content.jsscript4.7 KB
extension/icons/icon-128.pngbinary2.8 KB
extension/icons/icon-16.pngbinary400 B
extension/icons/icon-48.pngbinary1.1 KB
extension/manifest.jsonconfig749 B
extension/popup.htmlother2.3 KB
extension/popup.jsscript1.7 KB
extension/sidepanel.cssother17.1 KB
extension/sidepanel.htmlother3.3 KB
extension/sidepanel.jsscript23.6 KB
lib/worktree.tsscript9.2 KB
package.jsonmanifest3.7 KB
pw-test-node.mjsscript147 B
scripts/analytics.tsscript5.3 KB
scripts/dev-skill.tsscript2.4 KB
scripts/discover-skills.tsscript1.2 KB
scripts/eval-compare.tsscript3.0 KB
scripts/eval-list.tsscript3.5 KB
scripts/eval-select.tsscript3.1 KB
scripts/eval-summary.tsscript6.5 KB
scripts/eval-watch.tsscript5.4 KB
scripts/gen-skill-docs.tsscript19.8 KB
scripts/resolvers/browse.tsscript4.3 KB
scripts/resolvers/codex-helpers.tsscript5.0 KB
scripts/resolvers/confidence.tsscript1.6 KB
scripts/resolvers/constants.tsscript2.7 KB
scripts/resolvers/design.tsscript44.8 KB
scripts/resolvers/index.tsscript3.1 KB
scripts/resolvers/learnings.tsscript3.9 KB
scripts/resolvers/preamble.tsscript27.7 KB
scripts/resolvers/review.tsscript47.1 KB
scripts/resolvers/testing.tsscript27.3 KB
scripts/resolvers/types.tsscript1.1 KB
scripts/resolvers/utility.tsscript15.5 KB
scripts/skill-check.tsscript7.1 KB
setupexecutablescript24.4 KB
supabase/config.shscript420 B
supabase/functions/community-pulse/index.tsscript4.1 KB
supabase/functions/telemetry-ingest/index.tsscript4.1 KB
supabase/functions/update-check/index.tsscript1.1 KB
supabase/migrations/001_telemetry.sqlother3.2 KB
supabase/migrations/002_tighten_rls.sqlother1.7 KB
supabase/verify-rls.shexecutablescript5.3 KB
test/analytics.test.tsscript10.2 KB
test/audit-compliance.test.tsscript3.9 KB
test/codex-e2e.test.tsscript8.0 KB
test/fixtures/coverage-audit-fixture.tsscript2.7 KB
test/fixtures/eval-baselines.jsonconfig388 B
test/fixtures/qa-eval-checkout-ground-truth.jsonconfig1.6 KB
test/fixtures/qa-eval-ground-truth.jsonconfig1.5 KB
test/fixtures/qa-eval-spa-ground-truth.jsonconfig1.6 KB
test/fixtures/review-eval-design-slop.cssother1.5 KB
test/fixtures/review-eval-design-slop.htmlother1.3 KB
test/fixtures/review-eval-enum-diff.rbscript986 B
test/fixtures/review-eval-enum.rbscript759 B
test/fixtures/review-eval-vuln.rbscript406 B
test/gemini-e2e.test.tsscript6.7 KB
test/gen-skill-docs.test.tsscript93.2 KB
test/global-discover.test.tsscript6.4 KB
test/helpers/codex-session-runner.tsscript9.4 KB
test/helpers/e2e-helpers.tsscript11.3 KB
test/helpers/eval-store.test.tsscript19.8 KB
test/helpers/eval-store.tsscript24.6 KB
test/helpers/gemini-session-runner.test.tsscript4.4 KB
test/helpers/gemini-session-runner.tsscript6.0 KB
test/helpers/llm-judge.tsscript4.2 KB
test/helpers/observability.test.tsscript9.9 KB
test/helpers/session-runner.test.tsscript3.7 KB
test/helpers/session-runner.tsscript11.8 KB
test/helpers/skill-parser.tsscript5.6 KB
test/helpers/touchfiles.tsscript18.7 KB
test/hook-scripts.test.tsscript13.6 KB
test/learnings.test.tsscript12.0 KB
test/review-log.test.tsscript2.7 KB
test/skill-e2e-bws.test.tsscript12.5 KB
test/skill-e2e-cso.test.tsscript8.7 KB
test/skill-e2e-deploy.test.tsscript17.6 KB
test/skill-e2e-design.test.tsscript24.4 KB
test/skill-e2e-learnings.test.tsscript5.1 KB
test/skill-e2e-plan.test.tsscript27.6 KB
test/skill-e2e-qa-bugs.test.tsscript8.1 KB
test/skill-e2e-qa-workflow.test.tsscript15.4 KB
test/skill-e2e-review.test.tsscript28.7 KB
test/skill-e2e-sidebar.test.tsscript9.6 KB
test/skill-e2e-workflow.test.tsscript21.0 KB
test/skill-e2e.test.tsscript131.7 KB
test/skill-llm-eval.test.tsscript36.7 KB
test/skill-parser.test.tsscript5.3 KB
test/skill-routing-e2e.test.tsscript25.8 KB
test/skill-validation.test.tsscript59.5 KB
test/telemetry.test.tsscript15.5 KB
test/touchfiles.test.tsscript11.9 KB
test/uninstall.test.tsscript6.2 KB
test/worktree.test.tsscript9.2 KB
ws-test.tsscript692 B

test/skill-e2e-qa-bugs.test.ts

View on GitHub
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { runSkillTest } from './helpers/session-runner';
import { outcomeJudge } from './helpers/llm-judge';
import { judgePassed } from './helpers/eval-store';
import {
  ROOT, browseBin, runId, evalsEnabled, selectedTests, hasApiKey,
  describeIfSelected, describeE2E, testConcurrentIfSelected,
  copyDirSync, setupBrowseShims, logCost, recordE2E, dumpOutcomeDiagnostic,
  createEvalCollector, finalizeEvalCollector,
} from './helpers/e2e-helpers';
import { startTestServer } from '../browse/test/test-server';
import { spawnSync } from 'child_process';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';

const evalCollector = createEvalCollector('e2e-qa-bugs');

// --- B6/B7/B8: Planted-bug outcome evals ---

// Outcome evals also need ANTHROPIC_API_KEY for the LLM judge
const describeOutcome = (evalsEnabled && hasApiKey) ? describe : describe.skip;

// Wrap describeOutcome with selection — skip if no planted-bug tests are selected
const outcomeTestNames = ['qa-b6-static', 'qa-b7-spa', 'qa-b8-checkout'];
const anyOutcomeSelected = selectedTests === null || outcomeTestNames.some(t => selectedTests!.includes(t));

let testServer: ReturnType<typeof startTestServer>;

(anyOutcomeSelected ? describeOutcome : describe.skip)('Planted-bug outcome evals', () => {
  let outcomeDir: string;

  beforeAll(() => {
    testServer = startTestServer();
    outcomeDir = fs.mkdtempSync(path.join(os.tmpdir(), 'skill-e2e-outcome-'));
    setupBrowseShims(outcomeDir);

    // Copy qa skill files
    copyDirSync(path.join(ROOT, 'qa'), path.join(outcomeDir, 'qa'));
  });

  afterAll(() => {
    testServer?.server?.stop();
    try { fs.rmSync(outcomeDir, { recursive: true, force: true }); } catch {}
  });

  /**
   * Shared planted-bug eval runner.
   * Gives the agent concise bug-finding instructions (not the full QA workflow),
   * then scores the report with an LLM outcome judge.
   */
  async function runPlantedBugEval(fixture: string, groundTruthFile: string, label: string) {
    // Each test gets its own isolated working directory to prevent cross-contamination
    // (agents reading previous tests' reports and hallucinating those bugs)
    const testWorkDir = fs.mkdtempSync(path.join(os.tmpdir(), `skill-e2e-${label}-`));
    setupBrowseShims(testWorkDir);
    const reportDir = path.join(testWorkDir, 'reports');
    fs.mkdirSync(path.join(reportDir, 'screenshots'), { recursive: true });
    const reportPath = path.join(reportDir, 'qa-report.md');

    // Direct bug-finding with browse. Keep prompt concise — no reading long SKILL.md docs.
    // "Write early, update later" pattern ensures report exists even if agent hits max turns.
    const targetUrl = `${testServer.url}/${fixture}`;
    const result = await runSkillTest({
      prompt: `Find bugs on this page: ${targetUrl}

Browser binary: B="${browseBin}"

PHASE 1 — Quick scan (5 commands max):
$B goto ${targetUrl}
$B console --errors
$B snapshot -i
$B snapshot -c
$B accessibility

PHASE 2 — Write initial report to ${reportPath}:
Write every bug you found so far. Format each as:
- Category: functional / visual / accessibility / console
- Severity: high / medium / low
- Evidence: what you observed

PHASE 3 — Interactive testing (targeted — max 15 commands):
- Test email: type "user@" (no domain) and blur — does it validate?
- Test quantity: clear the field entirely — check the total display
- Test credit card: type a 25-character string — check for overflow
- Submit the form with zip code empty — does it require zip?
- Submit a valid form and run $B console --errors
- After finding more bugs, UPDATE ${reportPath} with new findings

PHASE 4 — Finalize report:
- UPDATE ${reportPath} with ALL bugs found across all phases
- Include console errors, form validation issues, visual overflow, missing attributes

CRITICAL RULES:
- ONLY test the page at ${targetUrl} — do not navigate to other sites
- Write the report file in PHASE 2 before doing interactive testing
- The report MUST exist at ${reportPath} when you finish`,
      workingDirectory: testWorkDir,
      maxTurns: 50,
      timeout: 300_000,
      testName: `qa-${label}`,
      runId,
      model: 'claude-opus-4-6',
    });

    logCost(`/qa ${label}`, result);

    // Phase 1: browse mechanics. Accept error_max_turns — agent may have written
    // a partial report before running out of turns. What matters is detection rate.
    if (result.browseErrors.length > 0) {
      console.warn(`${label} browse errors:`, result.browseErrors);
    }
    if (result.exitReason !== 'success' && result.exitReason !== 'error_max_turns') {
      throw new Error(`${label}: unexpected exit reason: ${result.exitReason}`);
    }

    // Phase 2: Outcome evaluation via LLM judge
    const groundTruth = JSON.parse(
      fs.readFileSync(path.join(ROOT, 'test', 'fixtures', groundTruthFile), 'utf-8'),
    );

    // Read the generated report (try expected path, then glob for any .md in reportDir or workDir)
    let report: string | null = null;
    if (fs.existsSync(reportPath)) {
      report = fs.readFileSync(reportPath, 'utf-8');
    } else {
      // Agent may have named it differently — find any .md in reportDir or testWorkDir
      for (const searchDir of [reportDir, testWorkDir]) {
        try {
          const mdFiles = fs.readdirSync(searchDir).filter(f => f.endsWith('.md'));
          if (mdFiles.length > 0) {
            report = fs.readFileSync(path.join(searchDir, mdFiles[0]), 'utf-8');
            break;
          }
        } catch { /* dir may not exist if agent hit max_turns early */ }
      }

      // Also check the agent's final output for inline report content
      if (!report && result.output && result.output.length > 100) {
        report = result.output;
      }
    }

    if (!report) {
      dumpOutcomeDiagnostic(testWorkDir, label, '(no report file found)', { error: 'missing report' });
      recordE2E(evalCollector, `/qa ${label}`, 'Planted-bug outcome evals', result, { error: 'no report generated' } as any);
      throw new Error(`No report file found in ${reportDir}`);
    }

    const judgeResult = await outcomeJudge(groundTruth, report);
    console.log(`${label} outcome:`, JSON.stringify(judgeResult, null, 2));

    // Record to eval collector with outcome judge results
    recordE2E(evalCollector, `/qa ${label}`, 'Planted-bug outcome evals', result, {
      passed: judgePassed(judgeResult, groundTruth),
      detection_rate: judgeResult.detection_rate,
      false_positives: judgeResult.false_positives,
      evidence_quality: judgeResult.evidence_quality,
      detected_bugs: judgeResult.detected,
      missed_bugs: judgeResult.missed,
    } as any);

    // Diagnostic dump on failure (decision 1C)
    if (judgeResult.detection_rate < groundTruth.minimum_detection || judgeResult.false_positives > groundTruth.max_false_positives) {
      dumpOutcomeDiagnostic(testWorkDir, label, report, judgeResult);
    }

    // Phase 2 assertions
    expect(judgeResult.detection_rate).toBeGreaterThanOrEqual(groundTruth.minimum_detection);
    expect(judgeResult.false_positives).toBeLessThanOrEqual(groundTruth.max_false_positives);
    expect(judgeResult.evidence_quality).toBeGreaterThanOrEqual(2);
  }

  // B6: Static dashboard — broken link, disabled submit, overflow, missing alt, console error
  testConcurrentIfSelected('qa-b6-static', async () => {
    await runPlantedBugEval('qa-eval.html', 'qa-eval-ground-truth.json', 'b6-static');
  }, 360_000);

  // B7: SPA — broken route, stale state, async race, missing aria, console warning
  testConcurrentIfSelected('qa-b7-spa', async () => {
    await runPlantedBugEval('qa-eval-spa.html', 'qa-eval-spa-ground-truth.json', 'b7-spa');
  }, 360_000);

  // B8: Checkout — email regex, NaN total, CC overflow, missing required, stripe error
  testConcurrentIfSelected('qa-b8-checkout', async () => {
    await runPlantedBugEval('qa-eval-checkout.html', 'qa-eval-checkout-ground-truth.json', 'b8-checkout');
  }, 360_000);

});

// Module-level afterAll — finalize eval collector after all tests complete
afterAll(async () => {
  await finalizeEvalCollector(evalCollector);
});
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 321kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 321kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 321kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 197kUpdated Claude CodeCodexVerifiedInstall