Skip to content

dse120071750/m8m-harness-builder / skills/m8m/m8m-harness-builder

m8m-harness-builder

Flagged

Build and edit M8M harnesses that coordinate existing tools and FlowSteps, validate milestone outputs, and track progress and resume. Reuse working implementations in place. Compile workflow changes without runtime packaging; build portable packages when requested and deliver server installations through the dedicated M8M server installer.

FileKindSize
.gitattributesother112 B
.gitignoreother208 B
LICENSEother673 B
README.mdmarkdown29.8 KB
SKILL.mdskill md1.3 KB
agents/audit_complete.yamlconfig1.6 KB
agents/flow_generated.yamlconfig2.2 KB
agents/harness_validated.yamlconfig2.3 KB
agents/openai.yamlconfig4.6 KB
agents/skill_shipped.yamlconfig2.1 KB
agents/toolbox_ready.yamlconfig1.6 KB
conftest.pyscript275 B
contracts/conformance/expectation_first_runtime/candidate-cases.jsonconfig4.1 KB
contracts/conformance/expectation_first_runtime/common-linear.jsonconfig2.7 KB
contracts/conformance/expectation_first_runtime/schemas/judge-decision.schema.jsonconfig373 B
contracts/conformance/expectation_first_runtime/schemas/result.output.schema.jsonconfig589 B
contracts/conformance/expectation_first_runtime/schemas/source.output.schema.jsonconfig500 B
contracts/conformance/flowstep_flow_v4/invalid/judge-loop-unbound.jsonconfig483 B
contracts/conformance/flowstep_flow_v4/invalid/malformed-execution.jsonconfig571 B
contracts/conformance/flowstep_flow_v4/invalid/no-required-output.jsonconfig455 B
contracts/conformance/flowstep_flow_v4/invalid/nonjudge-bound.jsonconfig763 B
contracts/conformance/flowstep_flow_v4/invalid/unknown-field.jsonconfig469 B
contracts/conformance/flowstep_flow_v4/valid/authoring-union.jsonconfig4.3 KB
contracts/conformance/flowstep_flow_v4/valid/platform-dag.jsonconfig1.9 KB
contracts/file_ref_v2.schema.jsonconfig484 B
contracts/flow_sequence_action_v2.schema.jsonconfig1.5 KB
contracts/flowstep_flow_v4.schema.jsonconfig13.6 KB
contracts/flowstep_output_v2.schema.jsonconfig2.8 KB
contracts/flowstep_output_v3.schema.jsonconfig3.0 KB
contracts/flowstep_skill_audit_v1.schema.jsonconfig5.1 KB
contracts/m8m_builder_configuration_v1.schema.jsonconfig252 B
contracts/m8m_builder_input_v1.schema.jsonconfig183 B
contracts/m8m_builder_installation_receipt_v2.schema.jsonconfig2.9 KB
contracts/m8m_builder_output_v1.schema.jsonconfig453 B
contracts/m8m_builder_request_v3.schema.jsonconfig707 B
contracts/m8m_builder_result_v3.schema.jsonconfig422 B
contracts/m8m_builder_source_audit_v2.schema.jsonconfig6.1 KB
contracts/m8m_builder_source_bundle_v2.schema.jsonconfig8.6 KB
contracts/m8m_builder_toolbox_manifest_v2.schema.jsonconfig2.6 KB
contracts/m8m_builder_validation_report_v2.schema.jsonconfig3.4 KB
contracts/m8m_cache_receipt_v1.schema.jsonconfig1.2 KB
contracts/m8m_candidate_cache_entry_v1.schema.jsonconfig2.8 KB
contracts/m8m_chosen_output_v1.schema.jsonconfig2.3 KB
contracts/m8m_context_capsule_v1.schema.jsonconfig1.9 KB
contracts/m8m_contract_bundle_lock_v1.schema.jsonconfig648 B
contracts/m8m_goal_ledger_v1.schema.jsonconfig1.8 KB
contracts/m8m_milestone_agent_v1.schema.jsonconfig13.8 KB
contracts/m8m_milestone_expectation_v1.schema.jsonconfig1.9 KB
contracts/m8m_milestone_judge_receipt_v1.schema.jsonconfig1.7 KB
contracts/m8m_milestone_judge_request_v1.schema.jsonconfig3.0 KB
contracts/m8m_run_context_v1.schema.jsonconfig994 B
contracts/m8m_run_context_v2.schema.jsonconfig1.2 KB
contracts/m8m_run_storage_contract_v1.schema.jsonconfig894 B
contracts/m8m_runtime_lock_v1.schema.jsonconfig1.7 KB
contracts/m8m_runtime_release_v1.schema.jsonconfig2.2 KB
contracts/m8m_skill_canvas_v1.schema.jsonconfig4.0 KB
contracts/m8m_source_asset_manifest_v1.schema.jsonconfig1.2 KB
contracts/m8m_workflow_package_archive_v1.schema.jsonconfig1.1 KB
contracts/m8m_workflow_source_bundle_proof_v1.schema.jsonconfig1.6 KB
contracts/m8m_workflow_source_bundle_v1.schema.jsonconfig19.5 KB
contracts/tool_vs_intelligence_table_v1.schema.jsonconfig1.3 KB
docs/m8m-chart.jpgbinary172.2 KB
docs/m8m-task-builder-design.zh-HK.mdmarkdown66.9 KB
examples/article_infographic/README.mdmarkdown598 B
examples/article_infographic/planning/m8m-flowchart.jpgbinary186.0 KB
examples/article_infographic/planning/m8m-flowchart.mdmarkdown4.2 KB
flows/m8m_build_v2.yamlconfig9.9 KB
flowsteps/tools/audit_complete_judge/input.schema.jsonconfig190 B
flowsteps/tools/audit_complete_judge/output.schema.jsonconfig477 B
flowsteps/tools/audit_complete_judge/tests/test_tool.pyscript1.1 KB
flowsteps/tools/audit_complete_judge/tool.pyscript1.2 KB
flowsteps/tools/audit_skill/input.schema.jsonconfig250 B
flowsteps/tools/audit_skill/output.schema.jsonconfig136 B
flowsteps/tools/audit_skill/tests/test_tool.pyscript1.9 KB
flowsteps/tools/audit_skill/tool.pyscript2.9 KB
flowsteps/tools/flow_generated_judge/input.schema.jsonconfig190 B
flowsteps/tools/flow_generated_judge/output.schema.jsonconfig475 B
flowsteps/tools/flow_generated_judge/tests/test_tool.pyscript1.1 KB
flowsteps/tools/flow_generated_judge/tool.pyscript1.5 KB
flowsteps/tools/generate_from_audit/input.schema.jsonconfig562 B
flowsteps/tools/generate_from_audit/output.schema.jsonconfig137 B
flowsteps/tools/generate_from_audit/tests/test_tool.pyscript2.2 KB
flowsteps/tools/generate_from_audit/tool.pyscript3.0 KB
flowsteps/tools/harness_validated_judge/input.schema.jsonconfig193 B
flowsteps/tools/harness_validated_judge/output.schema.jsonconfig488 B
flowsteps/tools/harness_validated_judge/tests/test_tool.pyscript1.1 KB
flowsteps/tools/harness_validated_judge/tool.pyscript1.3 KB
flowsteps/tools/install_local_skill/input.schema.jsonconfig733 B
flowsteps/tools/install_local_skill/output.schema.jsonconfig144 B
flowsteps/tools/install_local_skill/tests/test_tool.pyscript2.1 KB
flowsteps/tools/install_local_skill/tool.pyscript3.0 KB
flowsteps/tools/install_toolbox/input.schema.jsonconfig400 B
flowsteps/tools/install_toolbox/output.schema.jsonconfig140 B
flowsteps/tools/install_toolbox/tests/test_tool.pyscript1.9 KB
flowsteps/tools/install_toolbox/tool.pyscript3.0 KB
flowsteps/tools/skill_shipped_judge/input.schema.jsonconfig189 B
flowsteps/tools/skill_shipped_judge/output.schema.jsonconfig484 B
flowsteps/tools/skill_shipped_judge/tests/test_tool.pyscript1.1 KB
flowsteps/tools/skill_shipped_judge/tool.pyscript1.3 KB
flowsteps/tools/toolbox_ready_judge/input.schema.jsonconfig189 B
flowsteps/tools/toolbox_ready_judge/output.schema.jsonconfig490 B
flowsteps/tools/toolbox_ready_judge/tests/test_tool.pyscript1.3 KB
flowsteps/tools/toolbox_ready_judge/tool.pyscript1.6 KB
flowsteps/tools/validate_harness/input.schema.jsonconfig563 B
flowsteps/tools/validate_harness/output.schema.jsonconfig141 B
flowsteps/tools/validate_harness/tests/test_tool.pyscript2.1 KB
flowsteps/tools/validate_harness/tool.pyscript3.0 KB
planning/upgrade.mdmarkdown16.2 KB
references/architecture.mdmarkdown28.1 KB
references/audit_complete.mdmarkdown739 B
references/builder-authoring.mdmarkdown9.8 KB
references/flow_generated.mdmarkdown1.2 KB
references/flowstep-development.mdmarkdown4.3 KB
references/harness-invocation.mdmarkdown2.9 KB
references/harness_validated.mdmarkdown786 B
references/master-prompts.mdmarkdown14.0 KB
references/milestone.mdmarkdown11.4 KB
references/output-lineage.mdmarkdown4.4 KB
references/runtime-packaging.mdmarkdown35.2 KB
references/server-delivery.mdmarkdown6.8 KB
references/skill_shipped.mdmarkdown798 B
references/tool-vs-intelligence.mdmarkdown10.0 KB
references/toolbox_ready.mdmarkdown612 B
requirements.txtmanifest150 B
scripts/audit_harness.pyscript80.7 KB
scripts/candidate_cache.pyscript27.6 KB
scripts/coordinate_workflow.pyscript4.3 KB
scripts/execution_identity.pyscript1.0 KB
scripts/flowchart_jpg.pyscript16.1 KB
scripts/flowstep_instruction.pyscript14.1 KB
scripts/flowstep_runtime.pyscript108.6 KB
scripts/flowstep_tools.pyscript5.7 KB
scripts/gem_text.pyscript5.7 KB
scripts/generate_harness.pyscript61.4 KB
scripts/humanize_chart.pyscript5.6 KB
scripts/humanize_chart_zh.pyscript5.2 KB
scripts/import_flow_v4.pyscript62.3 KB
scripts/m8m_build_steps.pyscript143.2 KB
scripts/m8m_cache.pyscript1.7 KB
scripts/m8m_delivery_common.pyscript3.8 KB
scripts/m8m_deploy.pyscript5.6 KB
scripts/m8m_factory.pyscript10.1 KB
scripts/m8m_flowchart.pyscript16.6 KB
scripts/m8m_local_bundle.pyscript4.5 KB
scripts/migrate_skill_source.pyscript34.2 KB
scripts/milestone_expectation.pyscript3.3 KB
scripts/milestone_pair.pyscript3.1 KB
scripts/package_archive.pyscript7.1 KB
scripts/project_imports.pyscript10.8 KB
scripts/run_flow.pyscript153.8 KB
scripts/run_goal.pyscript14.1 KB
scripts/run_m8m.pyscript3.8 KB
scripts/runtime_release.pyscript45.0 KB
scripts/schema_gate.pyscript5.6 KB
scripts/session_layout.pyscript74.4 KB
scripts/skill_source.pyscript65.3 KB
scripts/source_bundle.pyscript87.7 KB
scripts/teaching_contracts.pyscript8.9 KB
scripts/tool_vs_intelligence.pyscript9.3 KB
scripts/toolbox_plan.pyscript6.1 KB
scripts/validate_harness.pyscript24.1 KB
scripts/verify_platform_contract_parity.pyscript2.9 KB
seeds/branch_receipt/input.schema.jsonconfig1.0 KB
seeds/branch_receipt/output.schema.jsonconfig359 B
seeds/branch_receipt/tests/test_tool.pyscript2.0 KB
seeds/branch_receipt/tool.pyscript3.7 KB
seeds/cycle_receipt/input.schema.jsonconfig1.0 KB
seeds/cycle_receipt/output.schema.jsonconfig319 B
seeds/cycle_receipt/tests/test_tool.pyscript1.5 KB
seeds/cycle_receipt/tool.pyscript2.0 KB
seeds/hash_bind/input.schema.jsonconfig213 B
seeds/hash_bind/output.schema.jsonconfig288 B
seeds/hash_bind/tests/test_tool.pyscript814 B
seeds/hash_bind/tool.pyscript527 B
seeds/ledger_receipt/input.schema.jsonconfig278 B
seeds/ledger_receipt/output.schema.jsonconfig355 B
seeds/ledger_receipt/tests/test_tool.pyscript842 B
seeds/ledger_receipt/tool.pyscript747 B
seeds/ok_receipt/input.schema.jsonconfig280 B
seeds/ok_receipt/output.schema.jsonconfig280 B
seeds/ok_receipt/tests/test_tool.pyscript667 B
seeds/ok_receipt/tool.pyscript682 B
seeds/schema_validate/input.schema.jsonconfig265 B
seeds/schema_validate/output.schema.jsonconfig196 B
seeds/schema_validate/tests/test_tool.pyscript1.0 KB
seeds/schema_validate/tool.pyscript844 B
seeds/slot_write/input.schema.jsonconfig558 B
seeds/slot_write/output.schema.jsonconfig346 B
seeds/slot_write/tests/test_tool.pyscript1.2 KB
seeds/slot_write/tool.pyscript1015 B
tests/__init__.pyscript0 B
tests/support.pyscript668 B
tests/test_audit_harness.pyscript23.5 KB
tests/test_branch.pyscript14.4 KB
tests/test_builder3_templates.pyscript2.4 KB
tests/test_builder_install_integrity.pyscript30.6 KB
tests/test_builder_native_snapshot.pyscript5.0 KB
tests/test_builder_requirements.pyscript26.8 KB
tests/test_builder_self_dogfood.pyscript7.0 KB
tests/test_builder_self_source.pyscript10.3 KB

references/tool-vs-intelligence.md

View on GitHub
# Tool vs intelligence

A **tool** is a pre-made function in `flowsteps/tools/`. A **FlowStep**
is that tool used *inside* a **milestone** (see `milestone.md`).
Intelligence is how a milestone may be reached, not a third canvas node
and not a FlowStep.

Classify every **tool** and every **milestone’s intelligence** before
generate. The class is not a label for the YAML. It decides whether the
work is a **repo function** or a **model call that must still use those
functions**.

This split is the one used by the highest-star agent/workflow systems:

| Source | Stars (order) | Their split |
| --- | --- | --- |
| [n8n](https://github.com/n8n-io/n8n) | ~201k | Workflow **nodes** (HTTP, DB, transform) vs an **AI Agent node** that *calls* those nodes |
| [LangChain / LangGraph](https://github.com/langchain-ai/langchain) | ~120k+ | Typed `@tool` functions / deterministic **graph nodes** vs an **agent node** that chooses tools |
| [CrewAI](https://github.com/crewAIInc/crewAI) | ~57k | “Keep application logic in regular Python”; wrap agents in deterministic workflows |
| [OpenAI Agents SDK](https://github.com/openai/openai-agents-python) | ~19–27k | **Function tools** (Pydantic-validated Python) vs **agents-as-tools** (intelligence callable without handing off the flow) |
| [Anthropic, Building effective agents](https://www.anthropic.com/engineering/building-effective-agents) | — | **Workflow** = predefined code path; **agent** = model directs the path. Tools are an agent-computer interface (ACI), designed like HCI |

They agree on three rules we adopt at the **milestone** grain:

1. Prefer a **workflow of checkpoints** (our driver) over an open agent
   loop *between* milestones.
2. A **tool** is a prewritten, schema-bound function the FlowStep prefers.
   The builder should develop it (fetch, MCP, crop). It is not compulsory.
3. If that tool fails, recover **like a normal agent** still aimed at the
   milestone candidate. The judge commits its declared named outputs;
   intelligence does not pick the next **milestone**.

Tool work is in-process product code. It is not a subprocess, shell command,
secondary workflow, or runtime-folder discovery routine. Read
`flowstep-development.md` for the closed execution and approval boundary.

Read this file once per new flow. Do not restate the catalog in `SKILL.md`.

The catalog describes possible capabilities, not required FlowSteps. Default
milestones to `loop: none`. Hashes, revision chains, proof graphs, and automatic
image reviews are not milestone prerequisites. Add a checksum or review only
for an explicit user request or an existing external API contract.

## The test (use this, not vibes)

A step is a **tool** if **all** of these are true:

1. **Same input → same action.** Given the same typed payload, it performs
   the same query, crop, hash, or write (allowing clock/IDs in receipts).
2. **Unit-testable with a fixture.** You can test it with a file, a mock
   row, or a PNG. No model key is required.
3. **The next step needs a receipt, not an opinion.** IDs, hashes, pixel
   size, row counts, file refs.
4. **A junior engineer could implement it from the schema alone.**

If any of 1–4 fail, it is **intelligence**. It still gets
`steps/<id>/tool.py`. That tool validates I/O and may return `NEED_MODEL`.
It must not `return draft`.

A third thing is **not a step class**: **orchestration**. The driver already
owns **milestone** order. Intelligence must not pick the next milestone.
Inside a milestone, FlowStep order is a table guide.

## What a tool is

A tool is a **function with a contract** (LangChain `@tool`, OpenAI
`FunctionTool`, n8n node, MCP tool):

- lives in `<codebase>/flowsteps/<flow_id>/steps/<id>/tool.py`
- `class: tool`, `model: none`
- input schema in, output schema out
- side effects allowed (DB, HTTP, disk) but **named and fail-closed**
- documented like a junior-dev API (Anthropic ACI / poka-yoke: obvious
  parameters, absolute paths, hard-to-misuse enums)

The FlowStep **prefers** the tool. If the tool fails, the session may
find a way (like a normal skill). Prefer fixing or adding
`flowsteps/tools/<id>/` rather than inventing a one-off every run.

### Tool catalog (expand here, not in chat)

| Family | Examples | Why it is a tool |
| --- | --- | --- |
| Fetch / IO | MCP `get_case`, SQL by primary key, HTTP GET of a known URL, read a UTF-8 file | Structured read; same id → same record |
| Transform | crop 4:3→4:5, letterbox, resize to 1080×1350, EXIF strip | Pixel math; fixture PNG in, transformed PNG out |
| Bind | sha256, `file_ref_v2`, locale normalize, whitespace collapse | Pure function |
| Render | HTML shell → Chromium screenshot, DXF emit, contact sheet | Generator + fixed viewport |
| Validate | JSON Schema, language/mojibake gate, dimension check | Pass/fail from rules |
| Package | zip pages, write manifest, copy captions from an approved plan | Assembly of already-typed inputs |
| Write | exact-id patch, upload with a frozen path, Firestore write behind an IO skill | Side effect with a receipt |
| Extract (library) | regex, date parse, Playwright a11y dump, OCR **if** it is a pinned engine | Deterministic enough to fixture |

Studio-shaped examples that **must** be tools:

- fetch CaseRecordV2 / style bundle by key
- crop a 4:3 source to native 4:5
- check that a generated PNG exists
- render seven 1080×1350 cards from plan + images
- materialize Instagram/Threads copy that already exists on the plan
- quote line-item math from a ratebook

## What intelligence is

Intelligence is **judgment or invention** the schema cannot compute:

- choose which lesson / page / crop *meaning* to keep
- draft wording, prompts, or a plan
- judge PASS/FAIL from evidence when the rule is not a pixel measurement
- decide among plausible alternatives

`class: intelligence` plus `model: completion|image|judge` and a real
`model_justification`. The repo tool still owns the contract. The model
only fills a **draft** that the tool admits.

OpenAI’s name for this wrap is **agents as tools**: intelligence is
callable, but it does not take over the workflow. Anthropic’s
evaluator-optimizer is the same idea: generate, then a separate typed
gate.

Studio-shaped examples that **are** intelligence (still wrapped):

- pick three educational highlights from a candidate pool
- write zh-Hant-HK caption wording
- compile a scene prompt that invents camera/wardrobe
- release-judge whether a card *teaches* (not whether footer pixels exist)

Footer geometry, token presence, and 920×978 size are **tools**, even
when a judge later reads them.

## Borderline cases

| Work | Class | Why |
| --- | --- | --- |
| OCR / speech-to-text via a pinned engine | tool | Engine is the implementation; fixture the transcript |
| “Is this Traditional Chinese?” via character set + denylist | tool | Rule |
| “Is this good Hong Kong copy?” | intelligence | Taste |
| Prompt = fill a frozen template with plan fields | tool | Substitution |
| Prompt = invent composition and lighting | intelligence | Invention |
| Route “url vs file vs text” by request keys | tool | Branch on data |
| Route “which worker should handle this customer?” | intelligence if the rule is fuzzy; tool if it is a table |
| Image **generate** | intelligence (`model: image`) | Model produces bytes; check the declared output schema and actual file |
| Image **crop / composite** | tool | No model |

When unsure, ship a **tool** first. Anthropic: start with the simplest
path; add a model only when a fixture test cannot express the work.

## Forbidden (milestone harness)

- Generating product tools under `~/.codex/skills` or `~/.claude/skills`
- Skipping required named outputs because a tool failed or a model “approved”
- Marking fetch, crop, hash, render, package, or schema-validate as the
  *first* move of a FlowStep when a repo tool exists
- Candidate intelligence choosing the next **milestone**, returning a semantic judge decision, or writing dedicated branch/cycle control fields (the separate worker owns those)
- Implementing a product FlowStep with `subprocess`, `Popen`, `os.system`,
  `os.popen`, PowerShell, batch, shell, or another workflow/CLI launcher
- Recursive discovery over the execution root, including `rg`,
  `Get-ChildItem -Recurse`, `os.walk`, `Path.rglob`, or globstar traversal
- Hand-writing chosen manifests, runtime judge/cache receipts, progress,
  roster, or ledger state from a FlowStep
- Separate stability-fetch or receipt-generation commands, or post-approval
  FlowSteps other than one declared in-process `finalize`

Not forbidden: recovering like a normal agent **inside** a milestone after
the preferred tool failed, as long as the candidate passes structural
admission and, when declared, the separate semantic judge returns PASS before
the runtime commits a chosen output bundle.

## How the highest-star systems strengthen us

Apply these as engineering gates, not extra classes:

- **n8n:** integrations are nodes. The AI node only *selects* nodes. Our
  `class: tool` is a node. Our driver is the canvas.
- **LangGraph:** deterministic nodes and LLM nodes share one state
  schema. Our output schema is that state.
- **OpenAI Agents SDK:** every tool has a Pydantic schema; wrap
  intelligence as a tool, do not hand off the whole flow.
- **CrewAI:** business logic stays in Python. Do not put ratebook math
  or crop math in a prompt.
- **Anthropic ACI:** spend time on tool signatures (absolute paths,
  enums, examples). A bad tool format is why models “write code on the
  fly.”

## Classification table (required before generate)

Fixed schema: `contracts/tool_vs_intelligence_table_v1.schema.json`.
Columns are `id`, `class`, `test`, `why`. Audit and generate emit this
object. See the sample rows in the GitHub README.

If a name contains `fetch`, `crop`, `hash`, `render`, `package`,
`resize`, `normalize`, `query`, `upload`, or `parse`, write it as a
**tool** on the table. That is a hint, not a refusal to draw.
planning-and-task-breakdownaddyosmani/agent-skillsBreaks work into ordered tasks. Use when you have a spec or clear requirements and need to break work into implementable tasks. Use when a task feels too large to start, when you need to estimate scope, or when parallel work is possible.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
screenpipethedotmack/claude-memSearch screen and audio history recorded by Screenpipe and optionally save selected context into claude-mem. Use when the user asks to recall desktop activity, a meeting, or something seen in another app through Screenpipe.Stars: 118kUpdated Claude CodeCodexVerifiedInstall
contact-widgetnexu-io/open-designSelf-contained floating chat widget with welcome screen, social links, meeting button, and message input. Single HTML file, zero dependencies.Stars: 108kUpdated Claude CodeCodexVerifiedInstall
meeting-notesnexu-io/open-designMeeting notes page — title bar with attendees, agenda checklist, decisions block, action items table with owners + dates, and a "next meeting" footer. Use when the brief mentions "meeting notes", "minutes", "1:1 notes", "all-hands recap", or "会议纪要".Stars: 108kUpdated Claude CodeCodexVerifiedInstall