stevesolun/ctx / docs
skill-router
FlaggedRepo-aware recommendation manager for ctx. Scans the active repository, identifies stack and workflow signals, recommends a capped set of skills, agents, and MCP servers, and unloads helpers that no longer match the current work after user confirmation. Harnesses are recommended by the custom-model onboarding flow or loop adapters and then attach to the same recommendation layer.
ctx-fit/PRD.md
View on GitHub# CTX Fit — Product Requirements Document
Status: draft v1, 2026-08-09. Owner: coordinator. Supersedes the earlier
"evolution, preserve the surface" framing.
## 1. One sentence
> Point CTX Fit at your repository. It finds the cheapest AI coding setup that
> reliably works on your codebase, writes the winning configuration into your
> working tree, and hands you a pull-request body to open yourself.
Headline: **Find the cheapest AI coding setup that actually works on your repo.**
CTX Fit runs no git *write* commands: it reads history to derive tasks, but creates no branch, commits nothing, pushes nothing and never merges. `ctx fit --pr` prints a PR body and a suggested
branch name; it creates no branch, commits nothing and never merges.
Three words carry the whole product. *Cheapest* is the objective. *Reliably* is
the constraint that makes cheapest meaningful. *Configuration* is the
deliverable — the product is not finished when it produces a report.
## 2. The problem
Teams adopting AI coding agents choose their setup — agent, model,
instructions, skills, MCP servers — from generic benchmarks, blog posts, and
opinion. None of that is evidence about *their* repository. Meanwhile CTX
itself has become a system that requires explanation before it can be
appreciated: 44 public commands, a 79,958-node graph, an entity taxonomy, and
a benchmark research framework.
Both problems have the same fix: put the complexity behind a single answer.
## 3. Users
| User | Wants | Success looks like |
| --- | --- | --- |
| Developer adopting AI coding tools | To stop guessing which setup to use | Runs one command, gets a defensible answer and the config to apply it |
| Engineering leader | Evidence before standardizing a stack for a team | A report with verified task outcomes and real cost, not vendor claims |
| Platform/DevEx engineer | To know whether a repo is even ready for agents | A readiness assessment with prioritized, actionable fixes |
## 4. The product experience
Three levels, increasing in cost and commitment. Level 1 must be safe, fast,
read-only, and free.
```text
ctx fit understand — readiness + what to improve (free)
ctx fit --test verify — controlled experiments (costs money, gated)
ctx fit --apply act — generate the configuration (reviewable)
```
Supporting flags, as shipped: `--budget USD`, `--dry-run`, `--pr`, `--yes`,
`--json`, `--max-depth`. Diagnostics live in `ctx doctor`. Anything else must
justify its existence; run `ctx fit --help` for the current surface.
## 5. Scope
### In scope for V1
- Repository profile with per-property provenance.
- AI agent readiness assessment with dimensions, evidence, blockers, and
prioritized fixes.
- Bounded candidate generation (2–5) with an explicit selection reason each.
- Controlled execution against a mandatory baseline, isolated per execution.
- Repository-native verification with distinct `attempted / completed /
verified / failed / inconclusive` states.
- Honest cost accounting where unknown stays unknown.
- Deterministic recommendation policy plus a Pareto view.
- Artifact generation with preview, apply, and rollback.
### Explicitly out of scope for V1
Simultaneous optimization of every dimension; MCP experimental optimization
(observe and recommend only); organization management, multi-tenant SaaS,
billing, SSO, RBAC; marketplaces, leaderboards, social features; a generic
agent platform, workflow builder, or second knowledge graph; CTX Continuous.
## 6. Non-negotiable product rules
These are correctness requirements, not preferences. Each maps to a test.
1. **Verification is repository-native.** An agent's self-report never counts
as success.
2. **Unknown cost stays unknown.** A configuration must never look cheaper
because its usage data was missing.
3. **A baseline is mandatory.** No improvement claim without one, run under
identical conditions.
4. **No fabricated numbers.** Every figure shown is measured, or labelled
estimate/inference/unknown.
5. **"No improvement found" is a valid, first-class result.** The product never
invents a recommendation to justify its own name.
6. **Recommendation is deterministic.** An LLM may *explain* a result; it may
never *decide* it.
7. **Nothing expensive happens by surprise.** Spend requires an explicit budget
and confirmation, with a dry run available.
8. **Honest scope language.** "Strongest among the configurations tested in this
experiment", never "the best AI setup for this repository".
9. **Graceful degradation.** Missing credentials, network, tests, or graph
reduce capability; they never produce a fatal error where partial value was
possible.
10. **The graph is invisible.** It is an internal optimizer, never the UX.
## 7. The objective function
**Cheapest that reliably works.** Selection is lexicographic, not a weighted
score (ADR-014):
```text
1. FILTER verified success rate across repeated trials >= reliability floor
2. MINIMIZE total attributable cost among the qualifying candidates
3. TIE-BREAK prefer the simpler configuration
```
*Reliably* is a constraint, not a metric: a candidate that passes two runs out
of three has not been shown to work, and cheapness among unreliable candidates
is meaningless. *Cheapest* is the objective, which makes cost completeness
load-bearing — a candidate whose cost is partial or unknown is reported
unranked rather than allowed to win by having less data (ADR-004).
Quality, latency, retries, and confidence remain separately measured and are
shown alongside the winner (`BEST QUALITY` / `FASTEST` / `RECOMMENDED`), but
that presentation never determines the choice.
If no candidate clears the floor more cheaply than the baseline, the correct
answer is **keep your current setup** — a success, not a failure.
## 8. Milestones
| M | Name | Outcome | Cost to user |
| --- | --- | --- | --- |
| M0 | Audit | Quarry classified; architecture chosen | none |
| M1 | Product skeleton | `ctx fit` exists and owns the product surface | none |
| M2 | Repository profile | Stack, verification, existing AI config | none |
| M3 | Readiness | Scored dimensions, blockers, prioritized fixes | none |
| M4 | Candidate selection | 2–5 explained candidates | none |
| M5 | Experiment planning | Plan + `--dry-run` + cost estimate | none |
| M6 | Execution | Isolated, reproducible runs with usage | money |
| M7 | Verification | Structured evidence per candidate/task | money |
| M8 | Recommendation | First complete Fit result | money |
| M9 | Apply | Generate config, preview, rollback | none |
| M10 | PR | PR body and suggested branch, with evidence | none |
**M1 through M10 have all shipped**: each milestone has landed code with tests
(`src/ctx/fit/`, `src/ctx/cli/fit.py`, `src/tests/fit/` — 273 tests passing).
M10 shipped as *prepare* a pull request rather than *open* one: CTX Fit runs no
git commands. Individual follow-up tasks remain open, but the per-task board in
[`planned_tasks.md`](planned_tasks.md) is a stale M0 snapshot and is not a
record of which — read the code.
## 9. Definition of done
Per task: implementation **+** tests **+** independent review **+** evidence.
Code existing is not done.
Per milestone: the four reviews in §11 all pass, including Product, which may
reject a technically complete milestone that does not improve the outcome.
V1 overall: a developer installs, runs `ctx fit` in a real repository, gets
useful readiness output; runs `ctx fit --test --budget 10` and receives an
honest recommendation with verified evidence and real cost; optionally runs
`ctx fit --apply` to generate the configuration.
## 10. Success criteria
Product: time to first useful result; fit completion rate; test-mode adoption;
apply rate; repeat use; whether users trust the recommendation.
Engineering: **public commands removed** (from 44 toward ~1); modules and LOC
removed; duplicate paths eliminated; experiment reproducibility; provider
adapter reliability. A successful rebuild ends with substantially *less* code.
## 11. Review model
Every milestone passes four independent reviews. Reviewers are expected to
reject; review is not ceremonial.
| Role | Asks |
| --- | --- |
| **Product Manager** | Did this make the product simpler, or merely add capability? Can a new user understand it? Does it produce a decision rather than more information? |
| **Architecture (CTO)** | Did we reuse valuable CTX code, avoid duplication, avoid accidental abstraction? Can the architecture be smaller? Are we building a second CTX? |
| **Security** | Credentials, repository boundaries, command execution, workspace isolation, artifact storage, telemetry, cleanup. The engine executes code; treat it accordingly. |
| **QA** | Test quality and coverage, integration across the full path, adversarial cases, regression, flakiness, reproducibility. |
Milestone reports are stored under `docs/ctx-fit/milestones/`, which currently
holds the M0 audit and the M1 spike friction log.
## 12. Risks
| Risk | Mitigation |
| --- | --- |
| Task derivation is the hardest unsolved problem; invalid tasks produce confident nonsense | `red_failure_contains` — a task that does not start red is not a task. Start with the narrowest defensible source. |
| Combinatorial evaluation cost | Bounded candidate generation is the whole point; hard budget enforced before spend. |
| Building a second CTX alongside the first | Architecture review has explicit authority to reject; prefer deletion over wrapper chains. |
| Cost dishonesty via missing usage | Completeness is a first-class field; incomplete records are never compared as if complete. |
| Optimizing our own benchmark instead of real outcomes | Held-out repositories never used for tuning. |
| Deleting something load-bearing | Evidence required before removal: dependents, tests, docs, CLI/API usage, integration points. |
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |