Skip to content

ObneyAI/orc / orc-skills/orc-prompt-caching

orc-prompt-caching

Verified

Configure and verify provider prompt caching in ORC nodes and composed trees, including shared delegates, model changes, concurrent users, warming and cache misses. Use for reusable prompt prefixes and inference cost; application result caches are a different mechanism.

references/qualified-evidence.md

View on GitHub
# Qualified evidence and remaining limits

## Source checkpoint

Hosted authored-tree evidence uses released ORC `4966c8e35e3bb5939570bfc645eac52a8d18eb6e`, SIO `e61e11a9abc6fdad4e19f772d9e32e2edd2dcc3b` and client `f5fa99d8df32d0830e972382e8fa6db233622022`. These exact sources emit the ONE-user/two-content-part enabled layout. Inspect the consuming namespace resources and dependencies rather than assuming a branch name selects them.

The two-USER layout is a private candidate in an isolated branch at this draft; it is not committed/published or an automatically adopted dependency. The source helper `request-messages` replaces the enabled nonblank-instruction layout under the existing `:prompt-cache?` flag. No new public layout selector or model-specific parser is established. The candidate now has focused public-boundary proof, exact-byte captured-response replays and ten new hosted SQLite ticks as detailed below. It remains unpublished; the original 296-call complex hosted cohort still used the released layout.

## Hosted ORC / SQLite cohort

296 physical responses, all successful. Independent raw audit validates all 296 against their ACTUAL sent strict JSON schemas; decoded outputs match all 296 canonical node outputs, including the compiler's flattened rare-report fields. Every physical response ID joins a public provider-attempt observation and the corresponding canonical node/child/root/user. There are 37 independent roots, eight generation calls each, eight authored prompt families; shared analysis executes three times/root. The task is a concrete described nested record/vector/enum/boolean contract, not every rich schema or arbitrary model task.

| Phase | Calls | Calls reading cache | Calls writing cache | Actual cost USD |
|---|---:|---:|---:|---:|
| One owner visits a,b,rare |24|16|8|0.011762550|
| Twelve different users, same warm cohort |96|96|0|0.018215260|
| Twelve users, separate fresh cohort |96|59|37|0.051333725|
| Common idle refresh, late |8|2|6|0.006976765|
| Rare revisit, late |8|7|1|0.002465655|
| Default routing/shared session |16|9|7|0.009391095|
| Default routing/separate sessions |16|16|0|0.002987630|
| Default routing/no session |16|16|0|0.003032830|
| Mixed role models |16|9|9|0.008420765|

Total actual cohort cost USD 0.114586275, not the campaign's historical exposure or unresolved holds. All 296 responses have both read/write counter fields. Totals 230 read-calls and 68 write-calls overlap: a response can read a prefix AND write additional content. Actual read tokens 1,835,620; write tokens 518,579.

The warm twelve-user wave reached 12 simultaneous HTTP bodies. Every new work/user had distinct typed identity and exact canonical results. Its per-role reads were:

| Prompt family | Calls | Cached tokens per call | Serving model/provider |
|---|---:|---:|---|
| Route |12|7721|Haiku / Anthropic|
| Method a |4|7693|Haiku / Anthropic|
| Method b |4|7694|Haiku / Anthropic|
| Rare method |4|7803|Haiku / Anthropic|
| Shared analysis |36|8067|Haiku / Anthropic|
| Direct left |12|8073|Haiku / Anthropic|
| Direct right |12|8073|Haiku / Anthropic|
| Synthesis |12|8070|Haiku / Anthropic|

Each role's warm cached-token count exactly equals that role's original primer write size. This is reuse of the whole measured cacheable prefix, not a claim that only a smaller common contract fragment hit. The input guide and current facts remain after the marker; the entire prompt is not implied cached.

This demonstrates one user's priming followed by reuse across other users on this account/route. It does not prove application authorization, cross-account/workspace/region sharing, unbounded capacity or a controlled throughput/latency improvement. Public synthetic static material was deliberately long for eligibility; do not copy that padding into production.

The fresh-cohort wave was NOT 96 independent uncached calls: it included 59 reads, including reuse developing within the burst and potentially shared earlier schema prefixes. Do not make zero-read a necessary cold-control oracle.

The mixed phase used ordinary node model/options: route+synthesis on Luna/OpenAI (4 calls), other work on Haiku/Anthropic (12 calls). Both Luna roles read on their second call while also reporting new writes. This supports model-local caching in one composed tree, not KV sharing across models or guaranteed warmth after idle.

The default-routing session cells ran sequentially on Google serving Haiku. First shared traffic wrote 7 prefixes; later separate and absent traffic read 16/16. Order and cache warming confound any claim that one session policy is better. Different app sessions did not establish isolated caches in this cohort. Both native idle phases have false timing qualifications: nominal 3/6-minute targets were missed. Preserve actual late observations and do not use these as a controlled TTL result. At the rare revisit, seven calls read; only the rare role rewrote its 7803-token prefix.

## Dedicated default-TTL control

Five request-level Haiku/Anthropic responses were strict-valid against the actual sent schemas, matched native decoded outputs and preserved typed source facts. Both distinct A/B prefixes initially wrote 8059 tokens. A refreshed at 180.1278 seconds and read again at 360.2834 seconds from the schedule origin, each reading 8059 with 0 writes. Untouched B at 364.3942 seconds read 0/wrote 8059. Actual A refresh→read interval was 180.1582055 seconds; B prime→read was 360.187814125 seconds. Maximum schedule lateness was 4.3942 seconds; monotonic runner brackets and actual POST timings are retained in the audit. Total actual cost USD 0.003690805.

This qualified control supports per-prefix expiry/refresh under this route's default five-minute profile: refreshing A retained A while leaving B idle did not retain B. It does not measure the exact expiry threshold or establish Gemini/other-provider TTLs, one-hour retention, durable tick recovery or every authored role's retention. The separate native complex-tree 3/6-minute timing qualifications remain false. Read/write counters must be interpreted jointly; the broader native cohort includes responses that both read and write.

## Private candidate qualification

The candidate keeps `:prompt-cache?` and ordinary model/options; it introduces no layout selector, model-specific parser, schema relaxation or system-role promotion. Independent focused tests passed 14 tests / 164 assertions. Allium check/analyse retained the accepted 39-warning / 113-information / zero-error baseline; this is not a clean structural check. The narrow public-boundary obligation is 1 covered / 0 uncovered; a full repository suite is not claimed. Implementation provenance includes a process deviation: the source agent drafted the specification amendment despite the repository condition reserving that work; the primary independently reviewed and accepted the amendment and reran the scoped checks. This is not represented as compliance with that condition.

Ten new hosted public SQLite ticks ran on the candidate, two changed-source requests for each of five models. All ten raw responses validated against their exact sent strict schemas, matched native outputs/source facts and canonical node writes, and joined response IDs across capture, durable observations and accounting. All root projections succeeded. Total actual cost was $0.0062856143333333334; no new unknowns arose. These are single-leaf model-switch qualifications, not ten newly hosted complex trees.

| Model | Cache read A → B | Cache write A → B |
|---|---:|---:|
| Luna | 0 → 5010 | 5374 → 364 |
| DeepSeek | 0 → 4992 | 0 → 0 |
| MiMo | 0 → 0 | 0 → 0 |
| MiniMax | 0 → 4864 | 0 → 0 |
| Gemini | 5057 → 5057 | 5057 → 0 |

A positive read alongside a new write is not by itself proof of sharing the intended prefix; preserve both counters. MiMo's zero pair is an observed miss, not a structured-output failure or evidence that caching is impossible. The earlier MiMo positive control remains separate.

The candidate also replayed six historical Haiku/GLM/Qwen captures through exact constructor requests and native decoding. A separate complex replay ran 12 sequential SQLite roots using 96 captured real responses with 596 passing checks. Each of the 96 actual request bodies matched its expected body after ONLY the split-message transformation. Neither replay is new hosted inference or new concurrent-load proof. All 46 frozen source hashes were verified. This qualifies the measured constructor/native contracts without claiming cache hits for every role, model or researcher mode.

## Request/predictor controls, separate from durable trees

- Haiku A→B→A→B interleave: 0,0,8055,8055 read tokens and 8055,8055,0,0 writes. Two prefixes remained usable between requests; not by itself simultaneous or native delegate proof.
- Haiku perturbations: 32 complete responses strict-valid and replayed through actual ORC/SIO parsing/native contracts; one earlier DNS failure remains unknown, unreplayed. Changed authored node names, runtime identities and nested input descriptions reused 8056 tokens in the qualified controls; output contract/description/type/mode or early-instruction changes wrote new prefixes. The 663-token short arm still wrote cache; it is NOT a below-threshold proof. Cache flags off changed layout here but is not a universal implicit-cache disable switch.
- GLM/Z.AI and Qwen/Alibaba exact full repeats read 5184/5484 tokens; changed runtime suffix read 0 with absent or stable session IDs. Both routes demonstrably have some cache capability.
- Separate-system/user diagnostic: Qwen changed-input reads 5102 in three matched cohorts; GLM 0 in all three. Higher instruction authority changed, so this is not the recommended library recipe.
- Same-authority two-USER control: Haiku 0→8062, Qwen 0→5111, GLM 0→0 across one pair per route. Six raw outputs were strict-valid/source-faithful. These six returned responses were request-level controls; they later replayed through the private candidate, separately from its ten newly hosted SQLite ticks. They do not qualify every mode/role.
- Eight-model request baseline includes Haiku, Luna, DeepSeek/Wafer, MiMo/Xiaomi, MiniMax/Inferact, GLM/Z.AI, Gemini/Google AI Studio and Qwen/Alibaba. MiMo had 0,0,4096 warm reads and MiniMax 0,4864,4864; neither establishes guaranteed hits. Later new-layout MiMo's single zero pair does not erase its earlier positive. Gemini first/new-prefix calls reported read AND write; its return removed the write. Interpret actual costs and both counters.

Caching failures did not establish structured-output failures in these complete responses. The known MiniMax 502, DNS/transport unknowns and unused preparations remain preserved separately; a passed sibling or paid route does not erase them. Free Nemotron/privacy-blocked routes and native decision protocols are not passed members.

## Gateway debug boundary

Six streaming debug samples (Haiku, GLM, Qwen pairs) were HTTP 200 with genuine terminal usage and strict-valid reassembled content. Gateway-reported upstream bodies showed only the later variable text changing; static instruction, schema and earlier text were unchanged. Haiku/Qwen retained the marker; GLM dropped it. No message flattening was shown. Qwen therefore did NOT fail because the visible gateway lost its marker. GLM's marker removal is an observed disconnect, but implicit caching and unseen routing/provider internals prevent a complete causal attribution.

Raw SSE/debug echoes remain restricted private artifacts. Reassembled content is labelled a protocol-level concatenation, never replacement raw bytes. These samples do not establish blocking gateway equivalence, native packet capture, role-cache behavior for all models or a universal parser fix.

## Fresh-context guidance check

A fresh agent used the draft to propose a multi-role architecture and identify adoption/measurement gaps without assuming the unpublished candidate was installed. The follow-up operational example passed 16 deterministic checks with two fully controlled HTTP responses, independently rerun by the primary agent; no inference occurred. This is design/usability review plus a tested model-only warm example, not a newly built live service, a fresh hosted complex tree or complete application adoption. The operational reference now supplies source/version gates, role inventory and measurement guidance; consumer-specific wiring remains necessary.

## Not yet qualified at this draft

Private candidate release/adoption and a new hosted rerun of its complete complex-role tree; native Jev caching; every researcher prime/inline/emitted role; attached judges; arbitrary tool/schema arrangements; expiry/refresh beyond the scoped Haiku/Anthropic default-TTL control; one-hour TTL; restart recovery and exactly-once effects; cross-account/region cache sharing; a fresh agent's live rebuilt-service adoption. Parent review precedes skill installation. The scoped design/operational forward check above is complete.

## Evidence availability

The observations above were recorded in source-pinned private research campaigns. Raw requests/responses, canonical records, independent audit reports and original hashes remain in the private research archive and are not bundled here. This public collection preserves the technical conclusions and their proof limits; importing the documentation does not independently reproduce those experiments.

Use [source status](../../SOURCE-STATUS.md) and the consuming implementation to distinguish the mainline cache layout from the unpublished candidate. Requalify provider/model behavior with bounded, authorized measurements for your application.
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall