Skip to content

notque/vexjoy-agent / skills/meta/do

Classify user requests and route to the correct skill, pipeline, and stack. Primary entry point for all delegated work.

references/semantic-first-ab-results.md

View on GitHub
# Semantic-First Routing: Unrun A/B Protocol

**Status: UNRUN.** `scripts/routing-ab-results/answers/README.md` states that the live routing run was never executed. No accuracy, cost, bucket, regression, or prompt-fix result from this protocol is evidence.

## Hypothesis

Test whether model-first intent routing with deterministic force-route safeguards performs at least as well as deterministic-first routing while reducing false positives from metaphorical git language.

## Planned protocol

- Corpus: 49 benchmark and paraphrase cases.
- Arm A: deterministic pre-router first, semantic router on fallthrough.
- Arm B: semantic router first, deterministic safety-critical override second.
- Blind scoring: de-identify arms, judge each route against the registered expected route, and record strict and partial accuracy.
- Safety gate: no new misses in force-route, false-positive-guard, git paraphrase, or security paraphrase cases.
- Artifacts expected after execution: answer files for both arms, judge output, scoreboard, and a gate verdict.

## Current decision basis

The shipped semantic-first structure is an architectural choice grounded in the toolkit's plain-English interface and deterministic safety-net design. This unrun protocol does not validate that choice. Use completed experiments in `docs/router-ab-runbook.md` for measured routing claims.

## Execution requirement

Do not add numeric results to this file until both arm outputs and the blind scoreboard exist under `scripts/routing-ab-results/`. When executed, preserve this protocol, link the immutable run directory, and report observed results separately from hypotheses.
writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 322kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall