Skip to content

stark-ai-de/agent-skills / incubator/skills/skill-maintenance/skillopt-setup

skillopt-setup

Flagged

Set up Microsoft SkillOpt for Agent Skills improvement. Use when installing SkillOpt, preparing per-skill train/val/test eval splits, configuring Codex CLI or provider-backed execution, running SkillOpt against SKILL.md bodies, reviewing best_skill.md, or importing optimized skill changes. Do not use for ordinary skill authoring or manual repo validation.

FileKindSize
SKILL.mdskill md12.9 KB
assets/agent-skills-benchmark/README.mdmarkdown1.7 KB
assets/agent-skills-benchmark/adapter.py.templateother14.4 KB
assets/agent-skills-benchmark/codex_cli_reflector.py.templateother27.5 KB
assets/agent-skills-benchmark/config.codex-cli-all.yamlconfig1.5 KB
assets/agent-skills-benchmark/config.hybrid-codex-target.yamlconfig1.7 KB
assets/agent-skills-benchmark/config.native-provider.yamlconfig1.3 KB
assets/agent-skills-benchmark/dataloader.py.templateother6.6 KB
assets/agent-skills-benchmark/evaluator.py.templateother62.5 KB
assets/agent-skills-benchmark/prompts/analyst_error.mdmarkdown193 B
assets/agent-skills-benchmark/prompts/analyst_error_full_rewrite.mdmarkdown215 B
assets/agent-skills-benchmark/prompts/analyst_success.mdmarkdown183 B
assets/agent-skills-benchmark/prompts/analyst_success_full_rewrite.mdmarkdown157 B
assets/agent-skills-benchmark/prompts/codex_cli_reflect_failure.mdmarkdown202 B
assets/agent-skills-benchmark/prompts/codex_cli_reflect_success.mdmarkdown186 B
assets/agent-skills-benchmark/rollout.py.templateother143.7 KB
assets/agent-skills-benchmark/schemas/codex_cli_patch.schema.jsonconfig1.2 KB
assets/agent-skills-benchmark/schemas/codex_cli_rollout.schema.jsonconfig854 B
references/adapter-contract.mdmarkdown5.8 KB
references/adoption-policy.mdmarkdown1.0 KB
references/codex-cli-runner.mdmarkdown3.3 KB
references/credential-modes.mdmarkdown5.8 KB
references/data-schema.mdmarkdown7.7 KB
references/local-artifact-audit.mdmarkdown1.9 KB
references/local-openai-gateway.mdmarkdown5.1 KB
references/official-best-practices.mdmarkdown4.1 KB
references/runbook.mdmarkdown20.0 KB
references/troubleshooting.mdmarkdown5.8 KB
scripts/apply-skillopt-best.mjsscript13.9 KB
scripts/audit-skillopt-local-artifacts.mjsscript22.0 KB
scripts/check-skillopt-readiness.mjsscript99.6 KB
scripts/codex-local-openai-chat-gateway.mjsscript51.6 KB
scripts/prepare-local-skillopt-adapter.mjsscript25.4 KB
scripts/prepare-skillopt-split.mjsscript22.9 KB
scripts/probe-codex-cli.mjsscript4.3 KB
scripts/probe-openai-compatible-endpoint.mjsscript7.9 KB
scripts/setup-skillopt-local.mjsscript40.9 KB
scripts/summarize-skillopt-run.mjsscript17.9 KB
scripts/verify-skillopt-run-artifacts.mjsscript7.4 KB

references/codex-cli-runner.md

View on GitHub
# Codex CLI Runner

Codex CLI mode is for users who are already authenticated locally and want SkillOpt target rollouts to run through `codex exec`.

## Probe

Use the bundled probe before Codex-backed runs:

```bash
node <skill-root>/scripts/probe-codex-cli.mjs --json
```

Resolve `<skill-root>` to the loaded `skillopt-setup` skill directory, such as `.agents/skills/skillopt-setup` for a repo-local install.

The probe asks Codex to return exactly `CODEX_READY` and stores only redacted readiness files under `.agents/skillopt-work/_readiness/`.

## Default Execution Settings

Generated configs should set:

```yaml
target_backend: codex_exec
codex_exec_path: codex
codex_exec_use_sdk: cli
codex_exec_full_auto: false
codex_exec_reasoning_effort: none
codex_exec_sandbox: workspace-write
codex_exec_network_access: false
codex_exec_web_search: false
codex_exec_approval_policy: never
tool_rollout_for_visual_assertions: true
require_drawio_cli_for_visual_rollouts: true
visual_exec_timeout: 120
visual_eval_policy: auto
```

Use `read-only` for probes. `codex_exec_sandbox` remains in generated configs for compatibility with older local adapters, but current target rollouts do not rely on legacy sandboxes because they permit host-wide reads.

Every Codex target, judge, and reflection launch selects a strict permission profile with `:minimal` platform reads, a narrow read-only grant for the resolved Codex executable or `@openai/codex` package, no inherited trainer secrets, and network disabled. Non-visual cases receive no workspace read or write grant and remain final-response-only. A case with `visual_assertions` may additionally use bounded rollout-workspace edits and shell commands for copied helper scripts, draw.io XML, deterministic validators, and requested PNG/SVG export; its protected control-output directory remains denied. `--strict-config` makes incompatible Codex versions fail closed instead of falling back to `workspace-write`. Strict readiness verifies this with an actual bounded config-parse launch that deliberately fails on a missing local output schema before session creation, model access, or network access. Browser, hosted, install, network, and out-of-workspace access remain disabled. When `visual_eval_policy: auto` and no `drawio`/`diagrams.net` executable is on PATH, generated configs should use `data-text-only`; that fallback bypasses draw.io only and still requires the strict Codex isolation probe. Provider-backed targets cannot create visual artifacts and must select `data-text-only` whenever source cases contain visual assertions. When visual cases remain active and `require_drawio_cli_for_visual_rollouts` is enabled, the adapter should fast-fail with `visual_rollout_blocker` before launching nested Codex.

## Usage Limits

Codex CLI runs through a ChatGPT plan may avoid direct API-key setup for target rollouts, but they still consume local plan allowance and can hit rate limits. Keep exploratory runs small and record target backend, optimizer backend, and Codex CLI version in public summaries.

## Slow/Meta Boundary

`codex-cli-all` is provider-free only while `optimizer.use_slow_update` and `optimizer.use_meta_skill` stay disabled. SkillOpt's upstream slow update and meta skill functions call the provider-backed optimizer path, so use `hybrid-codex-target` or `native-provider` when those mechanisms are required.
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall