Skip to content

agentsope/SkillAlchemy / skills/agentsop-code-execution-decision

agentsop-code-execution-decision

Verified

Decision rubric for when an LM agent should write-and-run code (Program-of-Thought / code interpreter) versus reason in natural language: classify each step as deterministic- computable (emit + execute code, feed the result back) vs judgment (stay in prose). Use when designing or debugging an agent step that does arithmetic/parsing/data transforms, when prose reasoning hallucinates a computation (under-coding), or when a sandbox round- trip is wasted on a judgment task (over-coding). Search keywords: code interpreter, agent does math wrong, calculator hallucination, when to run code vs reason, program of thought, PoT, tool vs reasoning.

FileKindSize
README.mdmarkdown1.9 KB
SKILL.mdskill md25.5 KB
intermediate/operation_candidates.jsonconfig4.2 KB
references/R1-source-evidence.mdmarkdown4.6 KB
# code-execution-decision

**Enhancement-overlay coder skill — T2: Code execution decision (emit-code-vs-reason).**

When should an LM agent write-and-run code (Program-of-Thought / code interpreter) versus
reason in natural language? This skill is the **decision rubric**. It overlays the `dspy` skill,
which ships the `ProgramOfThought` *mechanism* but not the *when*.

## Core claim

> LMs are unreliable calculators but reliable coders. Emit code when the answer needs
> determinism/precision (arithmetic, exact data manipulation, deterministic transforms);
> reason in prose when it needs judgment.

- **Under-coding** (reasoning through arithmetic) fails *silently* — a plausible wrong number.
- **Over-coding** (sandboxing a judgment task) fails *loudly and cheaply* — a wasted round-trip.

## The gate (one line)

> Is there a single verifiable answer a program could check?
> **YES → emit + run code → feed result back into the LM. NO → reason in prose. MIXED → decompose.**

## Contents

| File | Purpose |
|---|---|
| `SKILL.md` | The skill: 7 sections (activate / mental model / SOP / operations / dilemma cases / anti-patterns / cross-framework). |
| `references/R1-source-evidence.md` | Source evidence map — claims traced to DSPy docs, PoT/PAL papers, framework tool docs. |
| `intermediate/operation_candidates.json` | Candidate operations distilled during authoring (provenance for §4). |

## Cross-links

- `[[agentsop-output-format-by-model]]` — sibling: once you decide to emit code, how to serialize it (never nest code in JSON).
- `[[agentsop-test-fix-loop]]` — the bounded execute → error → retry loop this skill defers to.
- `dspy` — supplies `ProgramOfThought`; this overlay supplies the decision rubric it lacks.

## Framework mechanisms covered

DSPy `ProgramOfThought` · OpenAI Code Interpreter · Anthropic code-execution tool · LangChain `PythonREPLTool` (note: unsandboxed).

`name: code-execution-decision` · `version: 0.1.0`
brainstormingobra/superpowersYou MUST use this before any creative work - creating features, building components, adding functionality, modifying behavior, or planning anything new, in software or out of it (a talk, a business, a renovation).Stars: 321kUpdated Claude CodeCodexVerifiedInstall
brand-guidelinesanthropics/skillsApplies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. Use it when brand colors or style guidelines, visual formatting, or company design standards apply.Stars: 197kUpdated Claude CodeCodexVerifiedInstall
frontend-designanthropics/skillsGuidance for distinctive, intentional visual design when building new UI or reshaping an existing one. Helps with aesthetic direction, typography, and making choices that don't read as templated defaults.Stars: 197kUpdated Claude CodeCodexVerifiedInstall
mcp-builderanthropics/skillsGuide for creating high-quality MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools. Use when building MCP servers to integrate external APIs or services, whether in Python (FastMCP) or Node/TypeScript (MCP SDK).Stars: 197kUpdated Claude CodeCodexVerifiedInstall