Skip to content

alexandervashurin/semaphore / .claude/skills/gstack

gstack

Flagged

Fast headless browser for QA testing and site dogfooding. Navigate any URL, interact with elements, verify page state, diff before/after actions, take annotated screenshots, check responsive layouts, test forms and uploads, handle dialogs, and assert element states. ~100ms per command. Use when you need to test a feature, verify a deployment, dogfood a user flow, or file a bug with evidence. gstack also includes development workflow skills. When you notice the user is at these stages, suggest the appropriate skill: - Brainstorming a new idea → suggest /office-hours - Reviewing a plan (strategy) → suggest /plan-ceo-review - Reviewing a plan (architecture) → suggest /plan-eng-review - Reviewing a plan (design) → suggest /plan-design-review - Creating a design system → suggest /design-consultation - Debugging errors → suggest /investigate - Testing the app → suggest /qa - Code review before merge → suggest /review - Visual design audit → suggest /design-review - Ready to deploy / create PR → suggest /ship - Post-ship doc updates → suggest /document-release - Weekly retrospective → suggest /retro - Wanting a second opinion or adversarial code review → suggest /codex - Working with production or live systems → suggest /careful - Want to scope edits to one module/directory → suggest /freeze - Maximum safety mode (destructive warnings + edit restrictions) → suggest /guard - Removing edit restrictions → suggest /unfreeze - Upgrading gstack to latest version → suggest /gstack-upgrade If the user pushes back on skill suggestions ("stop suggesting things", "I don't need suggestions", "too aggressive"): 1. Stop suggesting for the rest of this session 2. Run: gstack-config set proactive false 3. Say: "Got it — I'll stop suggesting skills. Just tell me to be proactive again if you change your mind." If the user says "be proactive again" or "turn on suggestions": 1. Run: gstack-config set proactive true 2. Say: "Proactive suggestions are back on."

FileKindSize
.env.exampleother171 B
.github/workflows/skill-docs.ymlconfig648 B
.gitignoreother137 B
AGENTS.mdmarkdown2.5 KB
ARCHITECTURE.mdmarkdown20.7 KB
BROWSER.mdmarkdown16.5 KB
CHANGELOG.mdmarkdown73.3 KB
CLAUDE.mdmarkdown12.8 KB
CONTRIBUTING.mdmarkdown15.1 KB
ETHOS.mdmarkdown5.5 KB
LICENSEother1.0 KB
README.mdmarkdown21.5 KB
SKILL.mdskill md27.2 KB
SKILL.md.tmplother8.4 KB
TODOS.mdmarkdown21.9 KB
VERSIONother8 B
bin/dev-setupother2.4 KB
bin/dev-teardownother1.6 KB
bin/gstack-analyticsother6.3 KB
bin/gstack-community-dashboardother4.7 KB
bin/gstack-configother1.1 KB
bin/gstack-diff-scopeother2.4 KB
bin/gstack-review-logother409 B
bin/gstack-review-readother526 B
bin/gstack-slugother478 B
bin/gstack-telemetry-logother6.7 KB
bin/gstack-telemetry-syncother4.4 KB
bin/gstack-update-checkother6.9 KB
conductor.jsonconfig87 B
docs/images/github-2013.pngbinary61.9 KB
docs/images/github-2026.pngbinary59.1 KB
docs/skills.mdmarkdown44.2 KB
package.jsonmanifest3.1 KB
scripts/analytics.tsscript5.3 KB
scripts/dev-skill.tsscript2.4 KB
scripts/eval-compare.tsscript3.0 KB
scripts/eval-list.tsscript3.5 KB
scripts/eval-select.tsscript3.1 KB
scripts/eval-summary.tsscript6.4 KB
scripts/eval-watch.tsscript5.4 KB
scripts/gen-skill-docs.tsscript89.5 KB
scripts/skill-check.tsscript5.8 KB
setupother8.4 KB
supabase/config.shscript454 B
supabase/functions/community-pulse/index.tsscript1.7 KB
supabase/functions/telemetry-ingest/index.tsscript4.1 KB
supabase/functions/update-check/index.tsscript1.1 KB
supabase/migrations/001_telemetry.sqlother3.2 KB
test/analytics.test.tsscript10.2 KB
test/codex-e2e.test.tsscript7.5 KB
test/fixtures/eval-baselines.jsonconfig388 B
test/fixtures/qa-eval-checkout-ground-truth.jsonconfig1.6 KB
test/fixtures/qa-eval-ground-truth.jsonconfig1.5 KB
test/fixtures/qa-eval-spa-ground-truth.jsonconfig1.6 KB
test/fixtures/review-eval-design-slop.cssother1.5 KB
test/fixtures/review-eval-design-slop.htmlother1.3 KB
test/fixtures/review-eval-enum-diff.rbscript986 B
test/fixtures/review-eval-enum.rbscript759 B
test/fixtures/review-eval-vuln.rbscript406 B
test/gemini-e2e.test.tsscript6.3 KB
test/gen-skill-docs.test.tsscript37.3 KB
test/helpers/codex-session-runner.tsscript8.9 KB
test/helpers/e2e-helpers.tsscript9.5 KB
test/helpers/eval-store.test.tsscript19.8 KB
test/helpers/eval-store.tsscript23.6 KB
test/helpers/gemini-session-runner.test.tsscript4.4 KB
test/helpers/gemini-session-runner.tsscript6.0 KB
test/helpers/llm-judge.tsscript4.2 KB
test/helpers/observability.test.tsscript9.9 KB
test/helpers/session-runner.test.tsscript3.7 KB
test/helpers/session-runner.tsscript11.6 KB
test/helpers/skill-parser.tsscript5.6 KB
test/helpers/touchfiles.tsscript11.0 KB
test/hook-scripts.test.tsscript13.6 KB
test/skill-e2e-browse.test.tsscript11.8 KB
test/skill-e2e-deploy.test.tsscript11.2 KB
test/skill-e2e-design.test.tsscript24.4 KB
test/skill-e2e-plan.test.tsscript19.7 KB
test/skill-e2e-qa-bugs.test.tsscript8.1 KB
test/skill-e2e-qa-workflow.test.tsscript15.4 KB
test/skill-e2e-review.test.tsscript23.2 KB
test/skill-e2e-workflow.test.tsscript23.2 KB
test/skill-llm-eval.test.tsscript33.6 KB
test/skill-parser.test.tsscript5.3 KB
test/skill-routing-e2e.test.tsscript26.3 KB
test/skill-validation.test.tsscript54.5 KB
test/telemetry.test.tsscript10.3 KB
test/touchfiles.test.tsscript10.1 KB

test/helpers/observability.test.ts

View on GitHub
/**
 * Unit tests for E2E observability infrastructure.
 *
 * Tests heartbeat, progress.log, NDJSON persistence, savePartial(),
 * finalize() cleanup, failure transcript paths, watcher rendering,
 * and non-fatal I/O guarantees.
 */

import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import { sanitizeTestName } from './session-runner';
import { EvalCollector } from './eval-store';
import { renderDashboard } from '../../scripts/eval-watch';
import type { HeartbeatData, PartialData } from '../../scripts/eval-watch';

let tmpDir: string;

beforeEach(() => {
  tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'obs-test-'));
});

afterEach(() => {
  try { fs.rmSync(tmpDir, { recursive: true, force: true }); } catch {}
});

// --- Test 1: runDir created when runId set ---

describe('session-runner observability', () => {
  test('1: sanitizeTestName strips slashes and leading dashes', () => {
    expect(sanitizeTestName('/plan-ceo-review')).toBe('plan-ceo-review');
    expect(sanitizeTestName('browse-basic')).toBe('browse-basic');
    expect(sanitizeTestName('/qa/deep/test')).toBe('qa-deep-test');
    expect(sanitizeTestName('///leading')).toBe('leading');
  });

  test('2: heartbeat file path uses ~/.gstack-dev/e2e-live.json', () => {
    // Just verify the constant is correct — actual write is tested by E2E
    const expected = path.join(os.homedir(), '.gstack-dev', 'e2e-live.json');
    // Import the module and check HEARTBEAT_PATH exists in the file
    const sessionRunnerSrc = fs.readFileSync(
      path.resolve(__dirname, 'session-runner.ts'), 'utf-8'
    );
    expect(sessionRunnerSrc).toContain("'e2e-live.json'");
    expect(sessionRunnerSrc).toContain('atomicWriteSync');
  });

  test('3: heartbeat JSON schema has expected fields', () => {
    // Verify the heartbeat write code includes all required fields
    const src = fs.readFileSync(
      path.resolve(__dirname, 'session-runner.ts'), 'utf-8'
    );
    for (const field of ['runId', 'startedAt', 'currentTest', 'status', 'turn', 'toolCount', 'lastTool', 'lastToolAt', 'elapsedSec']) {
      expect(src).toContain(field);
    }
    // Should NOT contain completedTests (removed per plan)
    expect(src).not.toContain('completedTests');
  });

  test('4: progress.log format matches expected pattern', () => {
    // The progress line format is: "  [Ns] turn T tool #C: Name(...)"
    const src = fs.readFileSync(
      path.resolve(__dirname, 'session-runner.ts'), 'utf-8'
    );
    // Both stderr and progress.log use the same progressLine variable
    expect(src).toContain('progressLine');
    expect(src).toContain("'progress.log'");
    expect(src).toContain('appendFileSync');
  });

  test('5: NDJSON file uses sanitized test name', () => {
    const src = fs.readFileSync(
      path.resolve(__dirname, 'session-runner.ts'), 'utf-8'
    );
    expect(src).toContain('safeName');
    expect(src).toContain('.ndjson');
  });

  test('8: failure transcript goes to runDir when available', () => {
    const src = fs.readFileSync(
      path.resolve(__dirname, 'session-runner.ts'), 'utf-8'
    );
    // Should use runDir as primary, workingDirectory as fallback
    expect(src).toContain('runDir || path.join(workingDirectory');
    expect(src).toContain('-failure.json');
  });

  test('11: all new I/O is wrapped in try/catch (non-fatal)', () => {
    const src = fs.readFileSync(
      path.resolve(__dirname, 'session-runner.ts'), 'utf-8'
    );
    // Count non-fatal comments — should be present for each new I/O path
    const nonFatalCount = (src.match(/\/\* non-fatal \*\//g) || []).length;
    // Original had 2 (promptFile unlink + failure transcript), we added 4 more
    // (runDir creation, progress.log, heartbeat, NDJSON append)
    expect(nonFatalCount).toBeGreaterThanOrEqual(6);
  });
});

// --- Tests 6, 7: eval-store savePartial() and finalize() ---

describe('eval-store observability', () => {
  test('6: savePartial() writes valid JSON with _partial: true', () => {
    const evalDir = path.join(tmpDir, 'evals');
    const collector = new EvalCollector('e2e', evalDir);

    collector.addTest({
      name: 'test-one',
      suite: 'test',
      tier: 'e2e',
      passed: true,
      duration_ms: 1000,
      cost_usd: 0.05,
      exit_reason: 'success',
    });

    const partialPath = path.join(evalDir, '_partial-e2e.json');
    expect(fs.existsSync(partialPath)).toBe(true);

    const partial = JSON.parse(fs.readFileSync(partialPath, 'utf-8'));
    expect(partial._partial).toBe(true);
    expect(partial.tests).toHaveLength(1);
    expect(partial.tests[0].name).toBe('test-one');
    expect(partial.tests[0].exit_reason).toBe('success');
    expect(partial.schema_version).toBe(1);
    expect(partial.total_tests).toBe(1);
    expect(partial.passed).toBe(1);
  });

  test('6b: savePartial() accumulates multiple tests', () => {
    const evalDir = path.join(tmpDir, 'evals');
    const collector = new EvalCollector('e2e', evalDir);

    collector.addTest({
      name: 'test-one', suite: 'test', tier: 'e2e',
      passed: true, duration_ms: 1000, cost_usd: 0.05,
    });
    collector.addTest({
      name: 'test-two', suite: 'test', tier: 'e2e',
      passed: false, duration_ms: 2000, cost_usd: 0.10,
      exit_reason: 'timeout', timeout_at_turn: 5, last_tool_call: 'Bash(ls)',
    });

    const partialPath = path.join(evalDir, '_partial-e2e.json');
    const partial = JSON.parse(fs.readFileSync(partialPath, 'utf-8'));
    expect(partial.tests).toHaveLength(2);
    expect(partial.total_tests).toBe(2);
    expect(partial.passed).toBe(1);
    expect(partial.failed).toBe(1);
    expect(partial.tests[1].exit_reason).toBe('timeout');
    expect(partial.tests[1].timeout_at_turn).toBe(5);
    expect(partial.tests[1].last_tool_call).toBe('Bash(ls)');
  });

  test('7: finalize() preserves partial file alongside final', async () => {
    const evalDir = path.join(tmpDir, 'evals');
    const collector = new EvalCollector('e2e', evalDir);

    collector.addTest({
      name: 'test-one', suite: 'test', tier: 'e2e',
      passed: true, duration_ms: 1000, cost_usd: 0.05,
    });

    const partialPath = path.join(evalDir, '_partial-e2e.json');
    expect(fs.existsSync(partialPath)).toBe(true);

    await collector.finalize();

    // Partial file preserved for observability — never cleaned up
    expect(fs.existsSync(partialPath)).toBe(true);

    // Final eval file should also exist
    const files = fs.readdirSync(evalDir).filter(f => f.endsWith('.json') && !f.startsWith('_'));
    expect(files.length).toBeGreaterThanOrEqual(1);
  });

  test('EvalTestEntry includes diagnostic fields', () => {
    const evalDir = path.join(tmpDir, 'evals');
    const collector = new EvalCollector('e2e', evalDir);

    collector.addTest({
      name: 'diagnostic-test', suite: 'test', tier: 'e2e',
      passed: false, duration_ms: 5000, cost_usd: 0.20,
      exit_reason: 'error_max_turns',
      timeout_at_turn: undefined,
      last_tool_call: 'Write(review-output.md)',
    });

    const partialPath = path.join(evalDir, '_partial-e2e.json');
    const partial = JSON.parse(fs.readFileSync(partialPath, 'utf-8'));
    const t = partial.tests[0];
    expect(t.exit_reason).toBe('error_max_turns');
    expect(t.last_tool_call).toBe('Write(review-output.md)');
  });
});

// --- Tests 9, 10: watcher dashboard rendering ---

describe('eval-watch dashboard', () => {
  test('9: renderDashboard shows completed tests and current test', () => {
    const heartbeat: HeartbeatData = {
      runId: '20260314-143022',
      startedAt: '2026-03-14T14:30:22Z',
      currentTest: 'plan-ceo-review',
      status: 'running',
      turn: 4,
      toolCount: 3,
      lastTool: 'Write(review-output.md)',
      lastToolAt: new Date().toISOString(), // recent — not stale
      elapsedSec: 285,
    };

    const partial: PartialData = {
      tests: [
        { name: 'browse basic', passed: true, cost_usd: 0.07, duration_ms: 24000, turns_used: 6 },
        { name: '/review', passed: true, cost_usd: 0.17, duration_ms: 63000, turns_used: 13 },
      ],
      total_cost_usd: 0.24,
      _partial: true,
    };

    const output = renderDashboard(heartbeat, partial);

    // Should contain run ID
    expect(output).toContain('20260314-143022');

    // Should show completed tests
    expect(output).toContain('browse basic');
    expect(output).toContain('/review');
    expect(output).toContain('$0.07');
    expect(output).toContain('$0.17');

    // Should show current test
    expect(output).toContain('plan-ceo-review');
    expect(output).toContain('turn 4');
    expect(output).toContain('Write(review-output.md)');

    // Should NOT show stale warning (lastToolAt is recent)
    expect(output).not.toContain('STALE');
  });

  test('10: renderDashboard warns on stale heartbeat', () => {
    const staleTime = new Date(Date.now() - 15 * 60 * 1000).toISOString(); // 15 min ago

    const heartbeat: HeartbeatData = {
      runId: '20260314-143022',
      startedAt: '2026-03-14T14:30:22Z',
      currentTest: 'plan-ceo-review',
      status: 'running',
      turn: 4,
      toolCount: 3,
      lastTool: 'Write(review-output.md)',
      lastToolAt: staleTime,
      elapsedSec: 900,
    };

    const output = renderDashboard(heartbeat, null);

    expect(output).toContain('STALE');
    expect(output).toContain('may have crashed');
  });

  test('renderDashboard handles no active run', () => {
    const output = renderDashboard(null, null);
    expect(output).toContain('No active run');
    expect(output).toContain('bun test');
  });

  test('renderDashboard handles partial-only (heartbeat gone)', () => {
    const partial: PartialData = {
      tests: [
        { name: 'browse basic', passed: true, cost_usd: 0.07, duration_ms: 24000 },
      ],
      total_cost_usd: 0.07,
      _partial: true,
    };

    const output = renderDashboard(null, partial);
    expect(output).toContain('browse basic');
    expect(output).toContain('$0.07');
  });
});
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 468kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 468kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 468kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall