ZenUml/conf-app / .claude/skills/ai-repair-improvement
ai-repair-improvement
VerifiedExtract original DSL from production AI Repair failures, save a private corpus, coordinate fixes, and complete L2→L3→L4 validation. Use for ongoing work on newly discovered real failures. For regression-only runs on saved historical DSL, use ai-repair-regression; for single-case validation, use ai-repair-case-validation.
AI Repair Continuous Improvement
This skill processes a user-selected number of failures: find real failures → preserve original cases → assign work → fix → validate each layer → save the final L4 preview → optionally run regression with ai-repair-regression. Model calls have no preset count, token, or cost cap; make the calls needed for the selected cases and record actual usage. When the request is only for simple or full regression on saved historical error DSL, use that standalone skill directly; do not enter this skill's production data retrieval flow. When invoked again, resume from the saved corpus and unfinished cases. Do not present a single run as a scheduled automated task.
At the start, locate both the conf-app and diagramly.ai checkouts and verify their remotes and project instructions. Relative paths below are based on conf-app; Diagramly paths are identified explicitly. If either checkout is missing, report the blocker instead of guessing its path.
First report counts for the requested period, then resolve three choices
Use the user's stated time range. If none was provided, default to the seven completed calendar days before today in Asia/Shanghai, ending at today's 00:00 exclusive; calculate the actual dates at run time and report them. Run the read-only aggregate query for production Jobs described in Production intake, using its programmatic database API path by default. Report the number of failed jobs, the number of distinct failed DSL inputs, and the number of identifiable COMPLETED no-ops in that range. Do not export DSL bodies, create case tasks, or call AI at this step. Tell the user the time range, filtering criteria, and query time. If production is inaccessible, say the counts are unknown; do not substitute Mixpanel event counts for DSL counts.
After obtaining the counts, resolve the three choices below. For a small batch (no more than 10 newly failed DSLs and no more than 20 pre-existing historical cases), this skill defaults to: process every newly failed DSL, use only openai/gpt-5.6-luna through local Diagramly → OpenRouter for the selected original DSL/error/Prompt, and run simple L2 regression on all pre-existing historical cases after every selected case reaches L4 PASS. Record that default and its scope in the run. If the batch exceeds either bound, the destination/model changes, or the operator has not authorized that data destination, ask all unresolved choices together; never infer authorization for a new destination. L4 test-site data still requires its own confirmation. A user can override any default for a particular run.
- Of the newly failed DSLs just counted, how many may this run process at most? If the private index is readable, also report the number of existing pending cases. Match newly found inputs against saved input SHA-256 values in both
private/customer-data/ai-repair-regression/cases/and the local private corpus; skip any case markedexcludeFromFutureRegression: truein its source or expected metadata unless the user explicitly re-includes it. Report these exclusions separately from pending repair cases. Stop selecting new cases at the user's case limit and leave the rest in the candidate queue. Do not ask for a model-call count, token limit, or cost limit. Make model calls as needed for the selected cases and record each call's actual usage and cost source. - Is there user authorization to send each selected case's complete original DSL, actual parser error, and constructed AI Repair Prompt to the named AI repair service and model during this run, either from this run's answer or an applicable standing preference? Identify the actual service, model, routing path, and authorization source. If authorization is absent or declined, data retrieval, saving, and offline analysis may continue, but all L2/L3/L4 model calls and regression calls remain BLOCKED (no_model_authorization). Authorization to one destination/model does not cover another.
- After all fixes are complete, should the latest AI Repair Prompt be regression-tested? Use the applicable standing preference or ask the user to choose simple regression (L2 only) or full regression (L2 → L3 → L4), and confirm whether to cover all pre-existing historical cases or a user-specified maximum.
ai-repair-regressionruns model calls as needed; full regression also requires authorization to use L4 test-site data. If no, record SKIPPED (user_declined). If not all cases are fixed, record that the trigger condition was not met; do not call partial testing final regression.
Create this run's record at private runs//run.json. Record the time range, aggregate counts and criteria, all three answers, selected regression mode, case IDs, actual model/destination, task IDs and statuses, regression scope, and model-call usage. Do not overwrite a root-level run.json or a previous batch. Parallel tasks report each call to the main session for one shared usage ledger. If the authorization scope for item 2 is unclear, verify the destination and data scope before calling the model. Model-call authorization does not authorize commits, pushes, deployments, page publishing, or scheduled tasks.
1. Find and preserve failures
After all three answers are clear, use Production intake to inspect Mixpanel lifecycle signals and Diagramly production Jobs. Mixpanel contains neither DSL nor jobId; use Job.input.diagramCode as the source text. Include FAILED jobs, and inspect COMPLETED jobs that returned the original input or whose candidate still cannot be parsed/rendered. Distinguish frontend failures that have no backend Job. Production access must remain read-only; use the already-counted time window and bounded pagination, stopping at the user's case limit.
Save each selected original DSL and any available actual errorMessage byte-for-byte under conf-app's private/local-data/ai-repair-regression/. If an error is missing, mark the evidence gap; do not invent one. Follow the existing corpus.json and separate case-directory structure. Append the source job, time, language, model, status, input SHA-256, and selection rationale. Preserve source records for repeated identical inputs with different errors or attempts; never overwrite originals or historical results. Save candidate fixes separately from original inputs. This directory is a persistent local regression corpus; it must not enter the public repository or be committed to the private submodule. If the directory is unavailable, stop at a safe local private path and report that migration is needed; do not save the data in the public repository.
2. Coordinate task sessions
Follow Task assignment and acceptance to write a clear task for each independent case or group of cases sharing a root cause. Specify the fixed original DSL/error, source record, authorized model, permitted files, acceptance evidence, and private artifact directory. The main session owns the corpus index, shared Prompt changes, final decisions, and L4 browser. Delegate simple retrieval, organization, and deterministic checks to lower-cost subagents; the main session handles deeper diagnosis and integration.
Create and coordinate multiple Codex task sessions only when the user explicitly requests them; otherwise divide work among subagents within the current task. Each session may access only its assigned customer cases. Do not paste DSL into the public repository, PRs, task titles, or ordinary messages; hand off using absolute local private paths. Tasks must not concurrently edit the same Prompt/file, share a browser, or compete for the same local service. Queue conflicting tasks and integrate them one at a time in the main session. Do not create Git branches or worktrees without user confirmation. Preserve existing workspace changes.
3. Fixes and layer gates
For each case, use the default L2 → L3 → L4 flow in ai-repair-case-validation, including its L2 browser rendering procedure for Mermaid and offline PlantUML previews. A strong-model retry tests L2 only by default; continue further only when explicitly requested. Before running, read the current model configuration in src/utils/aiModelConfig.ts. Generate the actual Prompt from the current Diagramly checkout; do not copy it by hand. Record both repositories' HEAD/dirty state, Prompt hash, parameters, each raw output, parsing/rendering evidence, and cost source. For no-ops, empty output, syntax errors, lost anchors, or job/UI failures, improve the earliest failing layer and retest. If a layer has not passed, mark the next layer SKIPPED unless the user explicitly requests diagnostic execution of that layer; label such results separately and never carry PASS forward from a failed gate. Log each model call and its usage after completion. A successful model response or unit test does not replace UI evidence.
L4 is determined only by an actually visible browser result. Putting production customer DSL into a test Confluence macro sends it to another data destination. Before L4, tell the user the specific site and whether a draft will remain, and obtain separate confirmation; authorization for item 2 does not cover it. Without confirmation, record L4 as BLOCKED (no_ui_data_authorization). After confirmation, reuse the current logged-in browser tab under the project browser rules. If it cannot be controlled, use the Codex built-in browser at the standard desktop viewport. Run cases serially. Save error states, diff states, the real preview screenshot from the last successful repair after Apply Code, and network/model evidence under that case's l4//. Also write final-preview.json pointing to the final PASS screenshot and recording the repaired DSL SHA-256, model, Prompt hash, time, and corresponding l4/results.jsonl record. Keep earlier failure screenshots; do not overwrite history. Without a real preview, do not write final-preview or mark L4 PASS. By default, leave only a macro draft; do not publish a Confluence page.
4. Final regression and delivery
Only when every case selected for this batch has reached L4 PASS, and the answer to item 3 is yes with a regression mode selected, run ai-repair-regression on the user-approved historical scope using the final code/Prompt. Hand that skill the selected cases, current code/Prompt version, this run's model and data-authorization scope, uncapped call policy, and before/after evidence. If historical cases fall outside the existing data authorization, obtain the relevant authorization first. Preserve evidence and return to the fix loop if regression finds a regression. Do not report “entire corpus passed” unless all selected historical cases pass.
Deliver the private corpus index, each case's status/task session, repair changes, L2/L3/L4 repair-validation evidence, and final preview paths. If final regression ran, also include the independent skill's regression evidence and covered/uncovered lists. A historical successful L4 preview is not L4 regression evidence for the new Prompt. Report only observed results; do not infer an overall success rate from a small sample. Unless separately and explicitly requested, do not commit, push, deploy, send external messages, or schedule recurring runs.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |