Skip to content

ObneyAI/orc / orc-skills/orc-model-sio

orc-model-sio

Verified

Configure ORC models and structured I/O, inspect SIO or native decision-client requests, and diagnose provider or parsing failures. Use when model choice or what was sent to a model matters.

references/complex-model-portability.md

View on GitHub
# Qualify model swaps on complex contracts

Use the same authored graph, typed contracts, inputs and request profile when comparing model names. Increase completion headroom for reasoning models when needed and record that difference. Do not add a consumer parser, key cleanup or fabricated defaults for a particular model. Verify the loaded ORC, SIO and adapter sources; a configured option or advertised provider capability is not wire evidence.

Capture the exact request, response channel and raw response, then compare decoded values with canonical per-node reads/writes and delegated outputs. Validate raw structured content against the actual sent schema as well as the original native contract. Keep required-null, optional absence, false, zero, punctuation-bearing keys, JSON-looking strings and nested provenance distinct. Inspect finite decision options and actual generated calls, not only final success. Model content accuracy, valid API use, schema acceptance, authorized host execution and effect completion are separate judgments.

A private final-source checkpoint ran **56 actual calls across eight model families**: Luna, Haiku, Gemini, DeepSeek, GLM, MiniMax, MiMo and Qwen. Every raw response validated against its exact sent strict JSON schema. Supplied native input values were retained and input-contract meanings reached the requests. GLM used an inline researcher trajectory; the other seven used an emitted specialist. Six cases produced exact canonical task results. There were two useful content failures:

- MiniMax changed evidence ordering in its generated specialist instruction. The final semantic validator rejected the resulting order.
- MiMo first generated an unmatched delimiter in a redundant nested schema, corrected it through normal researcher feedback, then added a literal backslash to an ownership identifier in final synthesis. The actual sent input was correct, the canonical output preserved the provider's altered string, and the exact-value validator rejected it.

These are not all-task-success claims. In particular, valid structured code fields do not guarantee executable generated code, and valid string fields do not guarantee source fidelity. Preserve first errors and normal recovery; do not add per-model cleanup or retry until a favorable sample appears. Keep immutable identifiers host-owned where the application does not require model authorship; where copying is necessary, validate exact equality through the canonical result.

All eight cases used one hash-manifested private library revision. Earlier wider tool/shape and strict-function cohorts retain their own source pins. The deterministic read-only lookup in this checkpoint is not autonomous model tool selection, and these runs do not prove outage failover or many-user overlap. Hosted checkpoint cost was USD0.1339067505 from its own 56 response usages; that figure is not the historical campaign total or unresolved spend exposure.

The shared input-description repair is now present in the private candidate. ORC automatically uses SIO's request guide; no consumer helper is required. A labeled declared-contract supplement preserves nested optional-child, empty-item and nullable-container meanings for supplied inputs, while the shown-value guide and native values remain unchanged. Absent top-level inputs remain absent. Common compiled/registered/ref schemas receive accurate types and finite references; custom opaque/regex structures are explicitly not expanded. See [input meanings and source limits](input-field-guide.md). Output-schema descriptions are not a substitute for this input-contract evidence.

Source status matters: these proofs use an immutable, hash-manifested private ORC/SIO/client snapshot with integrated local repairs. They do not establish released/mainline availability. Compare loaded namespace resources and dependency pins before adopting the behavior; do not copy temporary dependency overrides or research capture plumbing into normal application code.

Current-model availability, privacy admission and token limits remain external facts. A paid sibling does not prove an unavailable free route. Retain explicit policy/capacity blocks and unknown spent calls; do not change account policy merely to make a comparison green.

## Authored workspace comparison on the merged stack

A later matched authored-tree checkpoint uses merged ORC `4966c8e35e3bb5939570bfc645eac52a8d18eb6e`, the same described contracts, ordinary tools and a 32,768-token completion cap. Five matched arms retain 391 physical posts: 388 complete responses match their exact sent strict schemas and 1,316 canonical native output fields. Two other responses are MiniMax `length` failures with incomplete JSON and long trailing whitespace; neither became a canonical native value. One MiMo post has no captured response and retains its unknown-cost reservation. The original raw captures remain available for this checkpoint, unlike the lost historical cohort described in the broader portability reference.

| Model | Complete / physical responses | Canonical fields matched | Task outcome on the frozen tree |
| --- | --- | --- | --- |
| Luna | 71 / 71 | 245 | Both workspace revisions passed; delivered a qualified incomplete answer because the customer's exact reproduction inputs were unavailable. |
| Haiku | 98 / 98 | 330 | Baseline passed; business passes ended before acceptance on the changed revision. |
| MiniMax | 97 / 99 | 326 | Made no source edit; baseline remained 14/15. Its plan added exhaustive diagnostics and consumed the business passes. Both truncated nodes recovered through ordinary same-budget retries. |
| DeepSeek | 66 / 66 | 224 | Both revisions passed 19/19, but a local host admission/fencing error prevented final delivery. No provider error was observed on its 66 responses. |
| MiMo | 56 / 57 | 191 | Applied both repairs; current protected acceptance improved from 14/19 to 17/19. A conservative local input guard then refused a request before dispatch. No final answer was delivered. |

MiMo’s graceful-close check was withheld after a cancelled HTTP future. Its canonical audit was independently recovered from a cold SQLite/WAL copy: all original evidence files remained unchanged, no commands/appends/poller ran, and the current protected receipt is 17/19. This adds contract evidence without qualifying the original close or task. The missing-response attempt remains unknown. The refused request was 244,420 UTF-8 bytes, plus 2,048 admission-overhead units, above the host’s conservative 229,375-unit bound. This is not a measured model context-limit rejection. Recovered audit (historical local experiment; private archive not bundled) A separate exact MiniMax request replay at 65,536 tokens completed with valid strict output and three exact canonical fields, using 6,297 tokens at $0.00722427. Only the cap changed in the request; the actual route changed to Nebius. Since the result fits within the old allowance, no causal headroom benefit is established. Replay audit (historical local experiment; private archive not bundled) Preserve that configuration difference and the original failed attempts. The larger allowance is a bounded experiment, not a new universal minimum or evidence of reasoning-token exhaustion.

The DeepSeek case demonstrates why a generic propagated provider-error label is insufficient diagnosis. The host reserved final-answer capacity, refused the next business request, then closed the same dispatch gate needed by the reserved writer. An isolated consumer fix separates business exhaustion from unexpected capture failure. A real ORC/SQLite proof using the shared hosted request wrapper passed 42 assertions. A later separately admitted final-only successor reused the exact 22-pass repaired state and original goal, with no new planning, repairs or tests: one DeepSeek/Wafer call delivered six exact strict/native/canonical fields at $0.00927596. The successor succeeded and closed; the original failed root, its 11,750-event prefix and all 693 archived canonical files remain unchanged. The captured log has 13,616 events and six new completions. This proves that specific delivery continuation and real capture/settlement path, not general automatic recovery. The report matches the current 19/19 receipt; customer-specific reproduction remains unproven, and the read-only contract incorporates the dependency extension by notice rather than restating it. Delivery evidence (historical local experiment; private archive not bundled) This finding requires no model parser or library workaround.

These results establish the observed request/decoder/canonical contract path. Different planning, supported conclusions, tool selections and task completion remain separate measurements. See the Luna/Haiku audit (historical local experiment; private archive not bundled), MiniMax audit (historical local experiment; private archive not bundled) and DeepSeek audit (historical local experiment; private archive not bundled).
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall