Skip to content

opensiro/vsm-harness-skills / skills/assess-vsm-harness

assess-vsm-harness

Verified

Produce a standalone evidence-backed VSM assessment of one autonomous AI agent harness at a pinned repository revision, including first-party ownership modes.

Assess a Harness as a Viable System

Methodology version: 0.3.8

Use the bundled VSM Harness Profile as the sole definition of S1-S5, S3*, recursion, autonomy, variety, homeostasis, and algedonic signalling. The bundled Profile for this methodology is v0.2.4. The output is a standalone repository assessment, not a cross-catalog comparison.

Read assessment-format.md for the required artifact and autonomy-states.md for local ownership notation. For —/? adjudication also read negative-state-calibration.md. For illustrative positive-evidence counterexamples (non-normative; no new thresholds), see positive-evidence-calibration.md. The same Methodology release also governs cohort synthesis and deterministic ranking through the repository-level SYNTHESIS.md.

Workflow

  1. Pin the repository revision and declare system-in-focus, purpose, environment, standard-distribution boundary, first-party operating/deployment modes considered, recursion level, review date, Profile version, and Methodology version. On first creation, also record the immutable generation-origin pair: generated_profile_version and generated_assessment_procedure_version.
  2. Describe repository architecture before VSM mapping: runtime, agent loop, state, tools, delegation, persistence, scheduling, evaluation, human involvement, maintainer/contributor governance, self-hosted/operator modes, and external integration when material.
  3. Identify operational outcomes and real S1 units.
  4. For each VSM function, first establish the organizational function from behavior and relationships. State the disturbance or variety being regulated.
  5. For S2 specifically, identify the distinct S1 units, the specific actual or structurally evidenced inter-S1 interference/conflict/oscillation, the coordination relation that attenuates it, and the feedback/closure path into subsequent S1 behaviour. Generic messaging, shared state, sequencing, routing, or speaker selection is not enough by itself.
  6. Identify the decisive decision right or feedback path that closes the function.
  7. Identify who owns that right in every first-party mode being claimed: agent, deterministic runtime, developer/configuration, human, parent system, or an explicitly described distributed arrangement.
  8. Record supporting/enforcement mechanisms separately. A scheduler, budget monitor, policy engine, queue, database, kill switch, interrupt, or deterministic controller does not inherit ownership merely because it executes the decision.
  9. Record the closure path where the result must affect subsequent regulation or operation. For parent-governed S3/S4/S5 modes, show the relevant matter reaching legitimate parent authority and the returned decision/feedback changing subsequent operation.
  10. When ownership is ambiguous, apply the Profile's counterfactual owner test: remove the candidate owner conceptually while leaving supporting machinery in place and ask whether materially the same discretionary organizational decision still occurs.
  11. For every material mapping record primary evidence, basis (explicit, structural, inferred, or unknown), confidence, and caveat.
  12. For negative/unknown candidates, perform negative-state adjudication: first test positive and eligible parent-governed modes, then trace plausible first-party function-specific paths within the declared supported operating boundary. Publish — only with the existing defensible three-part Absence scope; use ? when a specific plausible in-boundary path or material coverage gap prevents either positive or bounded absence judgment, and record the uncertainty. An abstract hypothetical future extension alone does not require ?. See negative-state-calibration.md.
  13. For every A(P) or C(P), record the autonomous/constructor base mode and the parent-governed mode separately in the standard mode matrix so a second reviewer can reconstruct each decisive owner and closure path independently.
  14. Assign the local state only after function, decisive right, ownership, support, closure, and any required negative/composite evidence are separated. For S3/S4/S5, encode a separately evidenced first-party parent mode with (P) on A or C, or standalone P when no first-party A/C autonomous mode is established.
  15. Record recursion, variety, escalation, and unresolved evidence gaps separately from the six-state vector.
  16. Write assessments/<harness_id>.md for index work. Do not generate a cohort-relative signature in this skill.

Methodology 0.3.8 positive-evidence calibration patch

This compatible patch adds an informative R1–R10 positive-evidence calibration guide and its frozen external-harness sample manifest. It illustrates existing function-first tests with runtime-boundary reachability, actual owner/decision, witness identity, and feedback/closure. It introduces no mandatory source-execution trace, positive threshold, required field, altered Profile semantics, ranking rule, or automatic Index regrade. Do not infer a gold label from an unresolved Index correction or revise a frozen experiment's original Methodology.

Methodology 0.3.7 negative-state calibration patch

This is a clarification of the existing —/? evidence distinction, not a change to state meanings or proof thresholds. A documented absence is bounded by the assessed first-party standard distribution; it does not require universal impossibility. A specific unresolved shipped function path or materially incomplete review can still justify ?. Consult negative-state-calibration.md and record any concrete uncertainty under existing Evidence gaps.

No Profile semantics, state set, required artifact field, parent mode, positive threshold, synthesis/ranking rule, or immutable provenance field changes. Do not alter historical Index vectors automatically: corrections require separate same-ref source review and Index gates.

Methodology 0.3.6 Profile-sync patch

Methodology 0.3.6 synchronizes the bundled normative Profile to compatible Profile 0.2.4, whose release declares assessment_impact: none. The Profile clarification makes the algedonic signal/channel boundary explicit: an exceptional signal path is not an additional S-function, and a queue/status/current-work surface is not itself positive S3/S4/S5 evidence.

This Methodology patch introduces no new assessment field, ownership symbol, function threshold, synthesis rule, or ranking rule. The structural requirements introduced in 0.3.5 remain in force unchanged. Existing assessments do not advance their current Profile/Methodology provenance merely because this patch exists; normal successful revalidation is still required, and immutable generation provenance is never rewritten.

Methodology 0.3.5 boundary-provenance patch

Methodology 0.3.5 makes the already-required system boundary reconstructable in the assessment artifact. It does not redefine any VSM function or ownership state.

Before crediting a positive state, partition first-party evidence into the supported assessed mode versus adjacent systems. Repository co-location is not enough. In particular, development/dogfood agents, contributor workflows, CI/release machinery, benchmark/evaluation infrastructure, tests/examples and governance surfaces may inform interpretation without owning a function in the assessed distribution.

For every new or revalidated 0.3.5 or later compatible assessment:

  • Review boundary records Credited operating / distribution surfaces and Adjacent first-party surfaces excluded from ownership;
  • every positive state records Boundary reachability, explaining why the decisive owner/path used for that state is reachable in the declared supported mode;
  • same-repository dogfood may corroborate a constructor path but does not upgrade C to A for another system-in-focus;
  • if a positive path depends on an adjacent actor that is not packaged/wired into the assessed mode, classify ownership without borrowing that actor.

The structural completion oracle checks only that these surfaces are recorded. It still does not decide whether the boundary claim is semantically true.

Methodology 0.3.3 reproducibility patch

Methodology 0.3.3 is a procedural PATCH over 0.3.1. It does not redefine any VSM Harness Profile function, add or remove an autonomy symbol, change ranking semantics, or require a vector change for an already conforming assessment merely because the patch exists. Version 0.3.2 was not published; the release line advances directly to 0.3.3 by maintainer decision.

Negative-state proof

— is an evidence-backed no-material-path conclusion at the reviewed boundary, not the default result of failing to find a positive witness. Every — produced or revalidated under 0.3.3 must record an Absence scope with:

  • Surfaces inspected — the first-party runtime/docs/tests/configuration/governance surfaces actually reviewed;
  • Plausible first-party paths checked — the concrete places where the missing function could reasonably have closed;
  • Why no material first-party path remains — the boundary-relative reason the reviewed standard distribution supports — rather than ?.

If the evidence boundary is too narrow to support that negative conclusion, use ?.

Composite-mode reconstruction

Every A(P) or C(P) claim must include the standard mode matrix:

ModeDecisive ownerTriggerClosureEvidence
Base (A or C)............
Parent (P)............

The matrix is a reproducibility surface only. It does not alter the meaning of composite notation or imply simultaneous dual ownership.

Structural completion oracle

Assessments newly produced or successfully revalidated under Methodology 0.3.3 or a later compatible patch must pass:

python scripts/check_assessment_contract.py assessments/<harness_id>.md

The checker validates artifact completeness only: required fields, function-specific witness labels, negative-state absence scope, composite-mode reconstruction, and boundary-reachability fields where applicable. It deliberately does not decide whether cited evidence is true or whether a VSM mapping is semantically correct. Existing assessments that retain an earlier current Methodology version are not required to be reformatted solely because a later compatible patch was released.

Generation provenance

Every assessment newly created by Methodology v0.2.1 or later must preserve which semantic/tooling versions produced the original artifact:

generated_profile_version: 0.2.4
generated_assessment_procedure_version: 0.3.8

These two fields are immutable origin metadata. Do not change them during reassessment, same-ref correction, methodology migration, or later Methodology upgrades.

The separate fields:

profile_version: ...
assessment_procedure_version: ...

record the semantics/procedure under which the current canonical classification was most recently successfully produced or revalidated. They may therefore advance while the generated_* pair remains unchanged.

assessment_procedure_version and generated_assessment_procedure_version remain compatibility storage-key names. Their values identify the VSM Harness Methodology version as applied to the assessment.

For legacy artifacts whose original generation versions were never recorded, do not guess or backfill them. Absence means generation provenance is unknown.

Methodology 0.3 ownership notation

Methodology 0.3.x keeps the Profile's function semantics unchanged and changes only how first-party ownership arrangements are published for S3, S4, and S5.

Allowed publication symbols are:

A  A(P)  C  C(P)  P  —  ?

For S3/S4/S5:

  • A(P) = an autonomous A mode is established and a distinct first-party parent-governed mode for the same function is also operationally closed;
  • C(P) = a constructor C mode is established and a distinct first-party parent-governed mode for the same function is also operationally closed;
  • P = an operationally closed parent-governed mode is established, but no first-party A or C autonomous mode is established at the reviewed boundary.

The parent modifier is multi-mode capability notation, not simultaneous dual ownership. A concrete deployment/run still has one reconstructable decisive owner for the function at a time.

Intentional publication boundary

The Methodology is not a complete ontology of every human-owned VSM arrangement that can exist in a real organization. Its assessment target is an autonomous AI agent harness, so the publication notation intentionally focuses on ownership topologies useful at that boundary.

  • S1: no P publication state. Ordinary operational ownership is expected to be agent-owned for an included autonomous harness; otherwise the harness may fail the inclusion boundary rather than receive S1=P.
  • S2: no P publication state. Inter-S1 coordination may be A, C, —, or ?; ordinary human coordination is not promoted to a parent-mode harness state.
  • S3: parent mode is an explicit supervisory/current-control exception. P, A(P), and C(P) are available when the S3 parent loop is first-party and operationally closed.
  • S3:* no parent-mode modifier in the 0.3.x line. Classify complementary audit through independence, audit judgment, ownership, and feedback without a separate P publication mode.
  • S4: parent mode is an explicit adaptation exception. P, A(P), and C(P) are available when the S4 parent adaptation loop is first-party and operationally closed.
  • S5: parent governance is the canonical P case because identity / ultimate-policy authority may legitimately remain at a parent recursion while the return-to-operation loop is complete.

This boundary is methodological, not ontological. Human ownership of S1, S2, or S3* can exist in the world; this Methodology intentionally does not encode those cases with P for autonomous-harness publication.

This is especially relevant to self-hosted OSS harnesses and distributed open-source organizations. A harness may deliberately support both autonomous and operator-governed S3/S4/S5 modes. Conversely, a non-human organization such as a swarm may close S3/S4/S5 autonomously without exposing any qualifying parent mode; plain A remains correct in that case.

Open-source contributors may also run agents in private/local execution contexts that are not centrally visible. Missing centralized access does not imply that every human decision is S5. Current-control decisions may be S3, adaptation decisions may be S4, and identity/ultimate-policy decisions may be S5, each mapped by function first and ownership second.

A contributor's local intervention does not automatically establish organization-level (P). The reviewer must reconstruct the relevant function and closure at the declared recursion. If only a lower-recursion contribution cell is evidenced, classify only that boundary.

Hard distinctions

  • Delegation, routing, sequencing, or handoff alone is not S2; require evidence of a specific actual or structurally evidenced interference/conflict/oscillation among distinct operational units plus a path that attenuates it and feeds the result back into later S1 behaviour.
  • A generic mailbox, shared task state, queue, graph edge, speaker selector, or lifecycle API does not receive S2=C merely because a developer could build coordination from it. The S2 function itself must first be established.
  • Methodology 0.3.x does not apply (P) or standalone P to S1, S2, or S3*.
  • A manager is not S3; require a whole-system current view plus authority over shared resources, commitments, priorities, constraints, accountability, synergy, or intervention. Any S3 parent-mode claim additionally requires legitimate parent ownership and returned closure.
  • Runtime enforcement is not S3 ownership. A hard budget, concurrency gate, scheduler, or kill mechanism may enforce an S3 decision while the decisive choice belongs to an agent, developer, human, or another actor.
  • A routine verifier is not S3*; require complementary and sufficiently independent access to operational reality.
  • Deterministic verdict parsing/gating does not automatically own S3* if the independent audit judgment is made elsewhere; record both separately.
  • Planning, learning, self-improvement, or event reaction alone is not S4; require an external-and-prospective adaptation loop whose options can affect current capability. Any S4 parent-mode claim additionally requires parent ownership of the decisive adaptation right and return into current capability.
  • A prompt, static policy, guardrail, or approval step alone is not S5; require runtime identity or ultimate-policy closure.
  • Generic human approval is not (P). The cited human/parent path must close the specific established S3, S4, or S5 function at the declared recursion.
  • A(P) does not mean parent assistance occurs during every autonomous run; it means the standard distribution exposes both evidenced first-party ownership configurations.
  • C(P) does not mean P is halfway from C to A; C and the parent mode must each be established independently.
  • Spawning or nesting is not VSM recursion.
  • Missing evidence is ?, not —.

A second reviewer must be able to reconstruct every positive state from cited primary evidence, including the function, decisive right, owner, support/enforcement split, closure path, declared boundary, and every ownership mode encoded by the symbol.

reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall