cgallic/kai-cmo-harness / harness/skills/kai-taste
kai-taste
VerifiedAudit or design generative AI interfaces against three diagnostic pillars (deterministic-stochastic balance, interaction density, visual cohesion). Treats taste as a measurable control system, not subjective preference. Use when: 'taste audit', 'score this UI', 'design quality', 'interaction density', 'visual cohesion', 'refiner layer', 'correction cost', 'why does this feel off', 'polish this', 'design review', or building any user-facing AI product.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 14.1 KB |
| references/pillar-rubrics.md | markdown | 7.5 KB |
| references/refiner-protocol.md | markdown | 10.8 KB |
| references/theory-foundations.md | markdown | 9.9 KB |
references/refiner-protocol.md
View on GitHub# The Refiner Layer: 10-Step Protocol
The Refiner Layer sits between raw model output and the user-facing artifact. It converts stochastic generation into deterministic, authored behavior. This is where taste becomes repeatable.
The model is not the product. The model is a probabilistic component inside a deterministic refinement system. The Refiner is what makes output feel inevitable rather than generated.
---
## Step 1: Define the Taste Contract as Constraints, Not Adjectives
Translate "brand voice" and "quality" into testable constraints:
- Reading level (e.g., Flesch-Kincaid 8th grade)
- Verbosity ceiling (max tokens/words per section)
- Required sections and structure (headings, summary, steps, caveats, citations)
- Prohibited moves (never hide uncertainty, never use banned phrases, never omit sources)
- Formatting grammar (markdown conventions, component types allowed)
- Interaction rules (when to ask questions vs proceed, when to show alternatives)
**Why:** Adjectives like "clean" and "professional" are not enforceable. Constraints are. A taste contract is a design system expressed as rules.
**Bad signal:** The contract says "make it good" or "be creative" — these are not constraints, they are wishes.
---
## Step 2: Separate Divergence from Convergence
Make the pipeline explicitly two-phase:
**Divergent phase (exploration):**
- Multiple candidates allowed
- Higher entropy/temperature
- Broader sampling strategies (nucleus/top-p)
- Goal: surface range of possibilities
**Convergent phase (refinement):**
- Low variance, strong constraints
- Schema-bound outputs
- Deterministic formatting
- Goal: lock down the final artifact
Never mix these phases. Exploring while refining creates volatility. Refining while exploring kills creativity.
**Why:** Iterative self-feedback with phase separation improves preference outcomes vs one-shot generation. The system needs room to explore before it commits.
**Bad signal:** The system generates a single candidate and immediately renders it as final output. Or: the system keeps generating alternatives after the user has committed to a direction.
---
## Step 3: Generate a Structured Intent Representation
Before drafting any prose or UI, derive a structured intent object:
```
{
goal: "what the user is trying to accomplish",
constraints: ["hard limits from the taste contract"],
audience: "who will consume this output",
deliverable_type: "answer | plan | code | document | UI component",
risk_level: "low | medium | high (determines verification depth)",
uncertainty_zones: ["areas where the model is not confident"]
}
```
This intent object becomes the stable substrate of the artifact. In a Canvas/Artifact model, it persists across turns. In Chat, it would otherwise be buried in conversation history.
**Why:** Without structured intent, the model guesses what the user wants from ambiguous natural language. Structured intent is a compact control vector that reduces downstream entropy.
**Bad signal:** The system jumps straight to prose without establishing what it's building, for whom, or under what constraints.
---
## Step 4: Draft into Intermediate Representation, Not Final UI
Have the model produce an IR — not polished prose or rendered UI:
```
IR = {
sections: [
{ type: "summary", claims: [...], confidence: 0.9 },
{ type: "steps", items: [...], dependencies: [...] },
{ type: "caveats", items: [...], severity: [...] },
{ type: "citations", sources: [...], verified: true/false }
],
ui_components: ["code_block", "table", "callout"],
uncertainty_map: { section_id: confidence_score }
}
```
The IR separates content from presentation. It can be projected into Chat, Canvas, or any other surface.
**Why:** Drafting directly into final UI couples content to presentation. When you need to change the rendering (different device, different context), you lose the semantic structure. IR makes the system surface-agnostic.
**Bad signal:** The model outputs a wall of markdown that is simultaneously the content and the UI. Changing the format requires regenerating everything.
---
## Step 5: Run a Critic Pass with Explicit Rubrics
Use a second model call (or the same model in critic mode) to grade the IR against:
- **Correctness:** Are claims supported? Are there contradictions?
- **Constraint compliance:** Does the output meet the taste contract from Step 1?
- **Tone violations:** Does it match the target voice? Any banned words or AI slop?
- **Formatting violations:** Does the structure match the required schema?
- **Affordance hazards:** Are calls-to-action ambiguous? Can users tell what's clickable?
- **Uncertainty gaps:** Are low-confidence claims marked? Or smoothed over?
The critic catches errors before the user pays the correction cost.
**Why:** The system corrects itself before the user has to. This is "support efficient correction" applied internally. Every error the critic catches is one less edit the user makes.
**Bad signal:** No self-review. The first draft goes directly to the user. Or: the critic exists but uses vague rubrics ("is this good?") instead of testable criteria.
---
## Step 6: Refine via Diffs, Not Rewrites
Output a patch/diff against the IR, not a complete rewrite:
```diff
- section.summary.claims[2]: "Revenue grew 40%"
+ section.summary.claims[2]: "Revenue grew 40% (source: Q3 earnings report, p.12)"
```
Diffs are explainable, controllable, and auditable. Rewrites are volatile — the user can't tell what changed or why.
**Why:** Full rewrites destroy user trust because the output shifts unpredictably. Diffs let users accept/reject individual changes, preserving agency. This also reduces outcome volatility (a key failure metric).
**Bad signal:** Every refinement regenerates the entire output. The user notices wording changed in sections they already approved.
---
## Step 7: Anchor Truth with Tools, Then Re-Render
When factuality matters, route to tools — don't let the model guess:
- **Retrieval:** Search knowledge base, documentation, or web for source material
- **Calculation:** Use a calculator for numeric claims
- **Database:** Query live data instead of recalling training data
- **Verification:** Cross-check claims against authoritative sources
After tool results return, regenerate only the dependent sections of the IR.
**Why:** Tool calls are determinism anchors. They convert "the model thinks X" into "the system verified X." This is the boundary where taste enforces "don't guess when you can check."
**Bad signal:** The model states facts with high confidence but never verifies them. Or: tool results are available but the model ignores them in favor of its own generation.
---
## Step 8: Compile to UI Using Component Grammar
Render the IR through a constrained set of components, not raw text:
| IR Type | Component | Rules |
|---------|-----------|-------|
| Summary | `<Summary>` | Max 3 sentences, bold key claim |
| Steps | `<StepList>` | Numbered, verb-first, one action per step |
| Warning | `<Callout type="warning">` | Yellow/orange, icon, never hidden |
| Code | `<CodeBlock lang="x">` | Syntax highlighted, copy button |
| Citation | `<Citation>` | Inline link + footnote, verified badge if checked |
| Table | `<DataTable>` | Sortable headers, aligned numbers |
| Uncertainty | `<ConfidenceBand>` | Visual encoding: blur, opacity, or explicit range |
Every component inherits design tokens (typography, spacing, color) from the product's design system.
**Why:** Raw LLM output should never directly become UI. Compiling through a component grammar ensures visual cohesion, affordance protection, and consistent behavior across all outputs. This is where Pillar 3 (Visual Cohesion) becomes mechanically enforced.
**Bad signal:** Generated content renders as unstyled markdown. Or: every output looks slightly different because there's no component grammar governing rendering.
---
## Step 9: Guarantee Correction Affordances at the Surface
Expose correction as primary interaction, not a fallback:
- **Edit-in-place:** Click any section to modify directly
- **Scoped refinement:** "Refine this section" without regenerating everything
- **Accept/reject diffs:** Per-change approval, not all-or-nothing
- **Dismiss:** Clear way to suppress unwanted AI interventions
- **Undo:** Reversible at every step
- **Fork:** Branch from any point to explore alternatives without losing current state
Correction affordances are not optional features. They are the primary interaction surface for uncertain systems.
**Why:** AI will be wrong. The product must make correction cheap, not treat it as user failure. "Support efficient correction" is a first-class design requirement, not a nice-to-have.
**Bad signal:** The only way to change output is to re-prompt from scratch. Or: the "edit" button exists but regenerates the entire artifact.
---
## Step 10: Instrument Taste as Feedback Loops
Log everything needed to close the loop:
| Signal | What It Measures | How to Use It |
|--------|-----------------|---------------|
| Time-to-first-value | How quickly users reach useful output | Optimize onboarding and first-run experience |
| Edit operations per session | Correction density | Identify sections that consistently need cleanup |
| Regeneration count | Output quality misses | Find patterns in what triggers regeneration |
| Diff rejection rate | Refinement quality | If users reject diffs, the critic pass is miscalibrated |
| Dismissal events | Relevance and intrusion | If users dismiss > 30% of suggestions, reduce proactivity |
| Abandonment after render | Output fails to meet intent | Investigate what was shown vs what was expected |
| Session graph structure | Workflow efficiency | Loops = thrashing; short paths = good taste |
Feed these signals back into:
- Critic rubric calibration (Step 5)
- Taste contract updates (Step 1)
- Component grammar refinement (Step 8)
- RLDF training data (preference pairs from edit traces)
**Why:** Taste is not a one-time decision. It's a closed-loop system. Without instrumentation, you're flying blind — guessing what users want instead of measuring it.
**Bad signal:** No telemetry. The team debates taste in design reviews based on personal preference instead of behavioral data.
---
## What the Refiner Buys You
1. **The model is not the product.** The model is a probabilistic component inside a deterministic system. Brand and experience stability come from the Refiner, not from hoping the model behaves consistently.
2. **No retraining required for taste changes.** Constraints, critique rubrics, component grammar, and rendering rules do most of the visible work. Updating taste = updating the Refiner config, not fine-tuning a foundation model.
3. **Chat-to-Canvas becomes an architecture shift.** The artifact (IR + rendered components) is the primary state. Conversation becomes one control surface among many. The Refiner makes this possible by maintaining structured state that survives across interaction modes.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |