ObneyAI/orc / orc-skills/orc-model-sio
orc-model-sio
VerifiedConfigure ORC models and structured I/O, inspect SIO or native decision-client requests, and diagnose provider or parsing failures. Use when model choice or what was sent to a model matters.
ORC model and SIO boundary
For new model-switching work, start with the current authoring and source checkpoint. It gives the common strict-output recipe, verified model set, dependency status and limits. No per-model application parser is required on that repaired stack.
The structured-contract repairs are integrated in the merged ORC source snapshot 250de6e6de2ad95647ef6da4c69220c2bd631279. Read structured contracts and provider observations before applying historical parser/metadata limitations below. It covers generated model-visible output meanings, paired strict parsing, null preservation and execution-owned provider observations. Verify the consuming ORC and pinned SIO/client revisions; older checkouts may lack these APIs, and strict output remains opt-in.
Read the target checkout's docs/ORC-PRINCIPLES.md, components/llm/src/ai/obney/orc/llm/core.clj, and components/orc-service/src/ai/obney/orc/orc_service/core/executor.clj before changing a model path. components/llm/deps.edn pins SIO and litellm-clj; inspect those exact commits rather than assuming upstream HEAD or an old guide is current.
For new trees, run contract preflight before paid execution. Executor mocks do not cover strict provider-schema compatibility. Also record each generation node's completion allowance and enclosing deadlines using reasoning and completion budgets; a valid schema does not reserve space for the answer after reasoning.
For cross-model portability, read the model-contract investigation recipe. It distinguishes the authored schema, compiled wire contract, endpoint admission, response channels, paired decoding and canonical writes. It also separates published fallback source, additional private campaign repairs, historical rich-schema limitations and remaining unsupported cases.
For valid-but-wrong records after a model switch, read portability proof boundaries. Distinguish schema validity, source fidelity and semantic support before changing parsing or prompts.
When a complex journey fails after otherwise valid calls, distinguish local admission/context growth from a provider or decoding failure. Read model context size before changing parsing. A shared typed context design is application composition, not per-model output adaptation.
Configure deliberately
For missing, renamed or rejected output fields, use structured-output boundaries. Verify setup and the exact adapter request before changing schemas or declaring an engine/provider defect; local validation and upstream schema enforcement are distinct guarantees.
- List every LLM-capable node (
sheet/llm,sheet/llm-condition,sheet/llm-decision, and anysheet/repl-researcherin scope) with its task, provider alias, requested:model, required capabilities, input/output contract, and expected cost/latency. Deterministic conditions compare typed facts.sheet/codecan invoke an injected native model client; this uses the client's explicit model configuration, not ORC's built-in LLM executor. Set a built-in LLM node's:modelwhen it needs a per-request override; the registered:llm-provideralias resolves provider credentials and supplies the default model when the node has no explicit:model. This default selection is not outage failover. Verify the actual selected model from response metadata or provider capture, not merely the DSL declaration. - Let blackboard Malli schemas and descriptions define the SIO signature. ORC serializes complex input values to JSON for LLM leaves; explicit
[:map ...]outputs are flattened to fields and reassembled. A description in that signature is not proof it reaches every request mode. Check input and output description visibility separately in the actual message/tool schema, as well as field names, optionality and parsed results, usingbuild-module/gather-inputsand a real or fake provider call. - Record the effective output mode for every ordinary generation node. On the documented structured-contract repair stack,
:output-mode :json-schemacan be selected by node options or inherited from provider configuration; see strict-mode configuration. Without strict mode, ordinary ORC leaves default to marker-based output (:use-function-calling? false). Opt a node into function calling with:options {:use-function-calling? true}only after checking the provider and schema. A directllm/predictcaller has its own provider-capability default, so do not conflate the two paths.:force-tool-choice?is a separate opt-in; its tool choice can be problematic for[:map-of ...]output schemas. ORC does not silently retry a failed function request as a marker request. - Verify required fields and semantic quality. A transport success, parse success, and useful domain answer are different claims. Inspect
:failure-kindand sanitized:provider-evidencefor structured failures, then compare the model's response, schema, and projected blackboard. Never infer meaning or quality from words/phrases matched in human or AI messages, or manufacture useful answers with canned response templates. Use semantic assessment; code parses explicit literal protocols and validates typed values.
Declare a concise, described rationale output when the consuming task or downstream assessment needs one, such as [:route-reasoning :routing]; it is not universally required or a proven quality improvement. Preserve any explicit consuming contract, and keep concurrent/per-item rationale ownership separate. Request an explanation supporting the task, not access to a model's hidden reasoning. Finite decision clients that generate no prose instead retain their native assessment and host policy/reason codes; do not invent a generated rationale for them.
Read reasoning and completion budgets for provider reasoning versus a declared rationale, reasoning-token headroom, incomplete responses, and what the current adapter actually retains. A small completion cap can be consumed by reasoning before valid output is finished; repeating field names will not fix that. Provider reasoning can exist even when no blackboard rationale is requested.
Choose request options for the selected model/provider. The pinned client requires a double for an optional :temperature and an integer for :max-tokens: decimal values such as 0.0, 0.2, and 0.7 satisfy that local numeric type, but supported sampling settings depend on the selected model. If choosing zero temperature on a compatible model, use :options {:temperature 0.0}, not integer 0; configure the completion allowance separately for that node's reasoning and output needs. Omit unsupported options at the node, then inspect the final wire body: client bc65561e emits absent sampling options as null, so DSL omission does not establish wire omission. The model-contract campaign has a separately verified local adapter repair that omits these null properties; it is not part of that published client pin. See option forwarding and repair status. Valid node input/output schemas do not establish valid provider options. Request validation can fail before any HTTP call; see live boundary findings.
At ORC 1851212396cced2fc4533fda30b824bd4b691390, the condition metadata bug is fixed. Conditions and decisions participate in budget/deadline/provider policy and durable assessment traces. llm-decision has exactly one finite answer write; its structured conversational request defaults to function calling, unlike ordinary generation. Do not apply the ordinary rationale-first write requirement to it. Read live boundary findings for historical evidence and current distinctions.
Opt-in prompt caching
At merged ORC 250de6e6de2ad95647ef6da4c69220c2bd631279, a generation node may set :options {:prompt-cache? true :generic-tool-description? true}. Both controls default off. For function-calling and strict JSON requests, :prompt-cache? puts a nonblank instruction in its own text part with cache_control {:type "ephemeral"}, ahead of variable inputs; marker-mode layout stays unchanged. In function mode, :generic-tool-description? removes the duplicated instruction from the generated tool description while retaining the instruction in the message and preserving output field descriptions. ORC consumes both controls before transport.
Source anchors are llm/core.clj (request-options, message-content, function-tool, function-request, json-schema-request) and docs/DSL-REFERENCE.md. The latter records that the pinned OpenRouter adapter preserves the message marker but drops the tool-level marker ORC constructs. Inspect the consuming adapter's serialized body and provider usage: cache support, minimum prefix size and savings depend on provider/model, and a marker alone proves no cache hit. This is merged source guidance, not a new live cache measurement.
For reuse across nodes, delegates, models and users, read prompt caching. Its source-qualified evidence covers hosted native trees, warming economics, actual cache read/write counters, message-boundary diagnostics and unresolved provider behavior. Released ORC 4966c8e3 uses one user message with two content parts. A private, unpublished candidate separates those parts into two USER messages under the existing flag; verify actual source/adoption before relying on that layout. No per-model application parser or per-execution session API is introduced by that candidate.
Native decision clients
Provider alias versus model identity
For an ordinary llm node, put the registered provider keyword in execution and poller context as :llm-provider. Omit node :model to inherit that provider's configured model, or supply an actual provider model ID such as "deepseek/deepseek-v4.1-flash". Do not put the ordinary provider alias, such as "authored-chat", in node :model: at the inspected pin it is forwarded as a literal model override. The special resolution of a registered native decision name below does not apply to ordinary chat aliases. A fake of llm/predict can hide this mistake; verify the serialized request's model through the real adapter before a live workflow.
Consider orc-decision-models for finite routes, tool arguments, applicability or work checks. Jev uses a native state/questions/answers protocol; assigning its model ID to an unchanged chat-completions request is insufficient. For supported boolean/Choice outputs register :protocol :decision, then use its provider name as the node :model. The adapter constructs questions from instructions and finite output descriptions; input values form state. Native Score or other unsupported surfaces still use an explicit custom client. Essential input meaning must appear in instructions/described data; inspect actual serialization. Native finite decisions cannot generate arbitrary replies or explanations; keep writing in separate LLM actions.
Registered native decisions use normal model usage and family budget integration. For all paths verify request assembly, decoded outputs, resolved model, policy/evidence versions, deadlines and attempts. Custom code clients outside registration still need their own LLM accounting seam. Read Jev setup for provider-specific protocol and validation.
See what actually went to the provider
For described tool selection, generated arguments and researcher contract visibility, read orc-tools. Capture the actual capability descriptions/schemas and relevant prior tool observations as well as the parsed proposal.
Separate four artifacts: (a) ORC node instruction and blackboard values; (b) ORC's SIO spec and pre-adapter request (:messages, optional :tools, options); (c) the provider adapter's serialized HTTP body; and (d) the provider's response. Only (c) is the exact outgoing request body. The ORC execution trace and :raw-response do not contain (c), and a provider may transform inputs internally beyond ORC's visibility.
To diagnose a call, capture the request at a test or instrumented HTTP transport boundary after litellm-clj's provider-specific transformation and serialization. Associate the captured body with tick/node/execution-context/attempt through an independently verified correlation seam and the provider endpoint; see structural request ownership. Final bodies do not automatically carry ORC identities, and matching output text is ambiguous for identical parallel work or missing responses; record its body bytes or parsed JSON in a controlled, access-limited artifact. Redact authorization headers and secrets, and avoid sharing raw user data or image payloads in reports. A mock router/completion proves only the ORC-to-router request, not the HTTP wire body; label it accordingly. For live QA, use a secure transport hook or controlled local endpoint and compare capture with the trace and projection. If this seam is unavailable, say that the exact wire body was not verified. HTTP exception messages can embed the complete request, including authorization headers. Capture only allowlisted error fields such as exception class and numeric status; never persist exception messages, whole ex-data, or throwable dumps from the transport. An adapter may already decode responses to JSON: label those as parsed responses, preserving model content verbatim without claiming raw response bytes.
The source-pinned request paths are sio/spec->prompt for markers, sio/outputs->tool-definition for submit_response, and the native decision adapter's state/questions construction. The historical input-field-guide stack carries shown input meanings. The separately verified private candidate automatically supplements supplied inputs with their declared nested contract, retaining empty-item and absent-child meanings without inventing evidence; verify loaded revisions before relying on it. Read input field guide and output parsing before applying historical workarounds. Keep meanings in described schemas, inspect the actual loaded ORC/SIO dependencies and serialized requests, and add explicit instruction context only for a verified remaining gap.
An input-description fix does not prove output parsing is correct. The historical input-field-guide stack reproduced an optional string whose marker prompt requested JSON but whose parsed value retained quotes. A model retry repeated the compliant output. Trace wire schema → response → parser → blackboard before blaming reasoning; on an unrepaired checkout, choose a verified supported request mode. The qualified repair path above corrects the parser rather than changing task prompts. Do not strip arbitrary quotation marks or weaken identity checks.
Model failover and provider failure evidence
Configured fallback is integrated in merged ORC 4966c8e35e3bb5939570bfc645eac52a8d18eb6e: use :provider, :fallback-models and :fallback-policy on existing llm, llm-condition, llm-decision and repl-researcher nodes. The merged dependency manifest pins SIO e61e11a9 and litellm-clj f5fa99d8. Read configured model fallback for the API, inherited researcher roles, durable selection, retry limits and stream boundary. It supersedes the earlier llm-fallback wrapper. Earlier open-PR/private-source status statements in detailed evidence are historical checkpoints; use the current adoption checkpoint and inspect the consuming pins. Read retained fallback profiles for the separately scoped retention/replay evidence and researcher recovery limits. No application parser or separate fallback node is required. PR #50 additionally contains a transport-timeout classification correction and fixture alignment; it remains unmerged at this checkpoint and is not part of the merged source above.
Keep model availability recovery separate from semantic or task recovery. Typed temporary failures admit a candidate under the default policy; broader transport admission is explicit. A valid negative answer, malformed output, cancellation or spent budget does not authorize another model. The prior scoped alternatives remain historical consumer examples, not the recommended new-node authoring API.
Review result
Report model/provider for each LLM-capable node, selected structured-output mode, where input/output descriptions actually appear, observed request boundary, response and parsed outputs, validation/failure evidence, and whether the tick's event projection agrees. Label simulated, router-level, and HTTP-level observations precisely. Never present a guessed prompt as verbatim wire content.
Source anchors
Historical verification against ORC origin/main 490816267b76b0428c53437052980a25f5905209; SIO d6d27f9bd59e2ac071396fc7942eabcdd4eabd12 (src/sio/core.clj); litellm-clj 0570e9459b8751d4a0c3dc1cd48595804426a84d (src/litellm/schemas.clj, src/litellm/core.clj, src/litellm/router.clj, src/litellm/providers/openrouter.clj). Re-check pins in the target checkout.
Judge request boundaries
Read orc-judges when configuring evaluators. Built-ins request a discrete band and derive its score in code; custom LLM judges should validate the complete band/evidence/feedback contract. A native Jev Score client uses typed ordered criteria and an application accounting seam. Capture actual instructions, input values, output schema and raw response. Description visibility is dependency-specific: consult the input-field-guide reference and capture the actual evaluator request before duplicating semantic definitions in instructions. Field order does not prove hidden model reasoning.
Configured versus resolved model identity
At the merged revision, native decision completions retain actual resolved model identity. Ordinary generation completion events instead keep the configured/requested node model (core/todo_processors.clj, completion command assembly). Preserve that event faithfully as configured identity, and capture the predictor metadata or actual provider response separately for resolved identity. Never overwrite either with a cohort label. A real provider can resolve a different model even when the request is accepted. Read current decision evidence.
Researcher model boundaries
Use orc-repl-researcher for prime, inline and emitted model roles. The prime sees previews/inspection metadata and final-write contracts; full source values and read descriptions may appear only in the relevant subcall. Capture each actual serialized request. Older source snapshots dropped generated per-leaf options; the fallback revision and subsequent local finite-node repair have their own evidence. Verify each actual role and option on the loaded pin rather than applying the historical finding globally; see researcher model contracts and source status. For classification/R-inject inspect pattern reuse, including all reranker roles and repeated guidance placement. Temperature remains model-supported and adjustable, never universally fixed at zero.
For complex model-name comparisons, empty-input descriptions and separation of protocol from content/API behavior, read complex model portability.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 193k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 193k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 193k | Updated | Claude CodeCodex | Verified | Install |