Skip to content

oliverv/caro / .agents/skills/oma-skill-creation

oma-skill-creation

Verified

Create or revise OMA skills and their references. Use for skill routing, execution contracts, conditional loading, and authoring validation.

FileKindSize
SKILL.mdskill md5.9 KB
resources/prompt-evaluation.mdmarkdown3.4 KB
resources/ssl-lite-template.mdmarkdown3.6 KB
resources/validation-checklist.mdmarkdown6.0 KB

resources/prompt-evaluation.md

View on GitHub
# Prompt and Routing Evaluation

Use when changing skill routing, context injection, approval boundaries, or a workflow's review contract. Static lint establishes structure; it does not measure task utility.

## Comparisons

Compare the current instructions, the proposed instructions, and a minimal task-only baseline on the same held-out tasks. Run each supported model separately with its exact dispatch model and effort recorded. An evaluator's default model, a mock result, or a result from another model is not an Astra measurement.

`oma skill eval` provides baseline/treatment isolation and optional negative-transfer checks; see `web/docs/guide/skill-eval.md` in the repository. Use available recordings or static fixtures first. Live dispatch consumes resources and follows existing budget authorization; do not start it just to complete a prose edit.

## Task matrix

| Task | Expected boundary |
|---|---|
| Fix a typo or doc link | Scoped inspection; no code tests or full repository map |
| Adjust a component style | Existing stack and relevant visual checks; no framework upgrade |
| Fix an authorization bug | Regression evidence for the failure and relevant existing checks |
| Work in a low-coverage repository | Declared project/task target; no invented global floor |
| Sync docs under an existing edit request | Apply scoped corrections without per-file reapproval |
| Review docs without edit authorization | Findings or proposed patches only |
| Create slides with purpose, length, content, and style supplied | Reuse the brief; no mandatory discovery question |
| Change a schema or production resource | Separate local preparation from destructive or external operations |
| Execute two tasks assigned to the same role | Unique task/run/session reports and unchanged claim identity |
| Run a simple task with many graph references | Owning skill retained; conditional references and adjacent specialists deferred |
| Publish or spend outside authorization | Prepare reviewable work and obtain the missing authorization |
| Run explicit ultrawork/ralph | Honor the selected workflow's evidence and review contract |

## Measurements

Record completion and correctness, verified defects found, irrelevant skill activation, unnecessary questions, failed or repeated verification, input tokens, latency, and cost. Report missing evidence as missing. Measure the assembled prompt or actual usage; file-size reduction alone is not a runtime savings claim.

Use repeated runs when model variation affects a decision. Preserve the current review contract until evidence supports changing it. Keep cost controls, destructive-action boundaries, result schemas, and regression requirements in every comparison.

## Regression checks

Check description boundaries before loading SKILL.md. Test the default context path as well as explicit graph calls. Exercise a soft budget smaller than the entry skill and verify the entry survives with a reported overrun. Check generated agent and directory instructions against the same policy as the source. Scope and approval behavior must hold through nested references, not only in the root skill.

The design follows [OpenAI's guidance on skills and prompts for GPT-6 Astra](https://developers.openai.com/blog/rethinking-skills-and-prompts-for-gpt-6-astra): concise routing, conditional references, task-specific reading, and explicit completion and authorization boundaries.
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 472kUpdated Claude CodeCodexUnverifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall