Skip to content

DevNullLtd/Yuzu / .codex/skills/ci-cache

ci-cache

Unverified

Canonical patterns for caching in Yuzu CI workflows. Two snippets — one for ephemeral GHA-hosted runners (split actions/cache/restore + actions/cache/save, never `save-always: true`) and one for self-hosted runners (local filesystem cache under `runner.tool_cache`, no GHA cache round-trip). Use when adding a new vcpkg/ccache/dependency cache step to any workflow under `.github/workflows/`, or when reviewing a PR that touches `actions/cache@`.

ci-cache

Canonical caching patterns for Yuzu CI workflows. CI eats more time than dev — get this right the first time.

TL;DR — two patterns

Runner kindCache mechanismSnippet
GHA-hosted (ubuntu-24.04, macos-15, windows-latest)actions/cache/restore + paired actions/cache/saveGHA-hosted
Self-hosted (yuzu-bigtam-linux, yuzu-weetam-windows, yuzu-bigmags-macos)Local filesystem under runner.tool_cacheSelf-hosted

Hard rule (enforced by .github/workflows/zizmor.yml): never use save-always: true on actions/cache@. The input is deprecated by the actions/cache maintainers as "does not work as intended and will be removed in a future release." The Guard — no save-always on actions/cache step in zizmor.yml fails any workflow audit if ^[[:space:]]*save-always: reappears as a YAML key. Don't try to silence the guard — rewrite the cache step using the snippets below.

Reference: https://github.com/actions/cache/tree/main/save#always-save-cache.


GHA-hosted (ephemeral runners)

Each job starts on a fresh disk. The cache lives in GitHub's blob storage and is moved over the network on restore + save. Use the split restore + always-save pattern:

- name: Restore <thing>
  id: cache-<thing>
  uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
  with:
    path: <path>
    key: <primary-key-with-strong-uniqueness>
    restore-keys: |
      <progressively-shorter-fallback-prefixes>

# ... build / test / install steps that produce the cache content ...

- name: Save <thing>
  # `if: always()` is the documented replacement for the deprecated
  # `save-always: true` input — runs even on Build/Test failure, which
  # is the only behaviour `save-always` was meant to deliver. NOTE: this
  # is right for an ADDITIVE cache only — see "Additive vs
  # coherent-artifact caches" below before copying it onto a tree. The
  # `cache-hit != 'true'` gate skips the upload when the primary key
  # already matched on restore (no new content to save, no point
  # paying the upload cost).
  if: always() && steps.cache-<thing>.outputs.cache-hit != 'true'
  uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
  with:
    path: <path>
    key: ${{ steps.cache-<thing>.outputs.cache-primary-key }}

Additive vs coherent-artifact caches — always() is not always right:

The if: always() above is correct for an additive cache, where every object saved is independently valid and a partial save is merely a smaller cache. ccache is the type case.

It is wrong for a coherent-artifact cache — vcpkg_installed, or any installed-prefix tree — where the entry only means anything as a complete set. A run cancelled or failed part-way saves a half-populated tree, and because the key inputs (manifest + baseline) still claim it is complete, the next run restores it as an exact hit, rebuilds everything anyway, and then skips its own save because cache-hit == 'true' — so the good tree is discarded and the poison persists. GHA cache keys are immutable, so nothing can overwrite it; only out-of-band deletion recovers. This is not hypothetical: it cost the canary ~65 min per run until #3229.

Note the split is about CONTENT VALIDITY, not about key occupancy. The cache-hit != 'true' skip means ANY entry — additive or not — locks its key for as long as it lives, so a thin entry saved by a cancelled run suppresses later saves in both cases. For an additive cache that costs a colder cache; for a coherent artifact it hands out a tree that reads as complete. Only the second is a correctness problem, which is why only the second is gated here — but do not read "additive" as "immune".

Two consequences follow, and they are different questions:

Correctness — only coherent artifacts need it. Gate the save on the producing step's outcome, not the job's.

Key occupancy — both classes need thought. A thin entry written under an exact key is hit by every later run with the same inputs, which then skips its own save, and the key is immutable. For an additive cache that costs a permanently colder cache rather than a wrong answer, so the gate is looser: save when the producing step RAN (pass or fail — a failed build's objects are still worth keeping), skip only when it was cancelled or never started. ci.yml's canary ccache save is the worked example.

Key inputs must be stable. hashFiles globs the whole workspace and does not honour .gitignore, so a bare **/*.cpp-style glob also hashes anything an earlier restore step materialised — dependency trees included. The key then differs between a cold and a warm run and the namespace splits. Scope the glob to the source roots you actually compile (#3270).

For coherent artifacts, additionally gate on the producing step's outcome, not the job's:

- name: Install deps
  id: install-deps
  run: ...

- name: Save <tree>
  # always() is still required — see the invariant above; without it the
  # implicit success() would also skip the save when only a LATER step
  # (e.g. Build) failed, even though the tree is complete and worth keeping.
  if: always() && steps.install-deps.outcome == 'success' && steps.cache-<thing>.outputs.cache-hit != 'true'

Key construction (D3 of the CI overhaul plan):

For vcpkg: include manifest + every overlay triplet + the pinned baseline. Example:

key: vcpkg-<triplet>-${{ env.VCPKG_COMMIT }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'triplets/*.cmake') }}
restore-keys: |
  vcpkg-<triplet>-${{ env.VCPKG_COMMIT }}-
  vcpkg-<triplet>-

For ccache: hash all .cpp / .hpp / .h source. Cascading restore-keys keep the cache mostly warm across source changes:

key: ccache-<leg>-${{ hashFiles('**/*.cpp', '**/*.hpp', '**/*.h') }}
restore-keys: |
  ccache-<leg>-

Branch-scope gotcha (the canary cold-start lesson from PR #740):

GHA cache scope is branch-isolated with PR runs scoped to refs/pull/<N>/merge. A PR job can read caches from (a) its own ref, (b) the repo's default branch, (c) the PR's base branch. Sibling-PR caches are unreadable. If a job only ever runs on pull_request, its cache never lands on refs/heads/main and every PR cold-starts.

Fix: also trigger the job on push to the branch PRs actually base on. The canary job in ci.yml does this via detect-ci-changes firing on pull_request plus push to refs/heads/main and refs/heads/dev. Warming only the default branch is not enough here and silently stopped working: this repo integrates on dev, main moves only at release time, and with no main push for five weeks the warm scope was simply empty — every PR then saved its own private ~843 MB copy of a byte-identical tree, six of them live at once against a 10 GB repo cap (#3233). Pick the branch by scope rule (c), not by convention, and check it is actually pushed to.


Self-hosted runners

Self-hosted runners (the yuzu-bigtam-linux, yuzu-weetam-windows and yuzu-bigmags-macos pools) keep state across jobs. Skip GHA's blob-storage cache entirely — round-tripping ~1 GB of vcpkg binaries through GHA storage when the same data already lives on the runner's local disk is an unforced loss. Pin caches to runner.tool_cache (a stable per-runner path that survives between jobs but is owned by the runner setup, not the workspace):

# Self-hosted: no actions/cache step at all. Set the env var that
# the build tool consumes and let it manage its own on-disk cache.
- name: Configure local cache paths (self-hosted)
  if: ${{ !contains(runner.name, 'GitHub') && runner.environment == 'self-hosted' }}
  run: |
    echo "VCPKG_DEFAULT_BINARY_CACHE=${{ runner.tool_cache }}/yuzu-vcpkg-binary-cache-${{ matrix.os }}" >> "$GITHUB_ENV"
    echo "CCACHE_DIR=${{ runner.tool_cache }}/yuzu-ccache-${{ matrix.os }}" >> "$GITHUB_ENV"
    mkdir -p "${{ runner.tool_cache }}/yuzu-vcpkg-binary-cache-${{ matrix.os }}"
    mkdir -p "${{ runner.tool_cache }}/yuzu-ccache-${{ matrix.os }}"

(See .github/workflows/codeql.yml for the existing VCPKG_DEFAULT_BINARY_CACHE → runner.tool_cache wiring on the Windows self-hosted runner — that's the pattern.)

Why runner.tool_cache and not ~/.cache/...:

  • runner.tool_cache is a stable, runner-managed path. It survives actions/checkout workspace cleanup (which can wipe vcpkg_installed/ if clean: true is set, see lesson from session 2026-04-28).
  • It's per-runner (not per-job), so back-to-back jobs on the same runner share warm caches.
  • It's outside ${{ github.workspace }}, so concurrent jobs can't race on the same file.

Per-triplet (not per-matrix-leg) scoping:

If two matrix legs build the same triplet (e.g., gcc-13 debug and gcc-13 release both build x64-linux), share the cache directory keyed on the triplet, not the matrix axis. The CI overhaul commit on 2026-04-28b establishes this — cumulative first-warm cost dropped ~5 h → ~2 h 20 min by collapsing per-matrix scopes into per-triplet scopes.

Sentinel-driven invalidation:

Self-hosted caches stay until something invalidates them. scripts/ci/vcpkg-triplet-sentinel.sh is the canonical invalidator: it computes the cache key from vcpkg.json + vcpkg-configuration.json + overlay triplets + VCPKG_COMMIT, compares against the stored sentinel, and wipes the triplet tree (and the orphaned-registry vcpkg/info/ directory — see #741) on drift. Always run the sentinel before vcpkg install on self-hosted.

Future self-hosted macOS:

When the self-hosted Mac lands (replacing or augmenting the macos-15 GHA runner), it should follow the self-hosted pattern: drop the actions/cache step from the macOS leg, set CCACHE_DIR=${{ runner.tool_cache }}/yuzu-ccache-macos-${{ matrix.build_type }} and VCPKG_DEFAULT_BINARY_CACHE=${{ runner.tool_cache }}/yuzu-vcpkg-binary-cache-arm64-osx, and rely on the runner's local SSD. ccache deltas on macOS are the same shape as Linux (~30 G with CCACHE_MAXSIZE=30G); the GHA-hosted cap of 10 GB total per repo no longer applies.


Reading the existing tree

When in doubt, read these as worked examples (all up-to-date as of v0.12.0-rc0):

FilePattern shown
.github/workflows/ci.yml macOS legs (lines 564–675)GHA-hosted split restore + paired save
.github/workflows/ci.yml canary (job canary:; line numbers drift, grep the job name)GHA-hosted split restore + paired save + push-to-main/dev warming
.github/workflows/release.yml (lines 703–751)Restore-only (release builds consume cache, never produce — see comment block)
.github/workflows/codeql.ymlSelf-hosted VCPKG_DEFAULT_BINARY_CACHE → runner.tool_cache
scripts/ci/vcpkg-triplet-sentinel.shSentinel invalidation logic, including the orphaned-registry self-heal

When to add a new cache

Three questions to answer before you add actions/cache/restore:

  1. What's the cache key? Must be uniquely determined by the inputs that change the cache content. Source-file hash for ccache; manifest + triplet + baseline for vcpkg; pip-tools requirements file hash for ~/.cache/pip. If the key collapses two materially-different states into one entry, the cache is poisoned.
  2. Where does it save? GHA-hosted → blob storage with branch-scope rules. Self-hosted → local runner.tool_cache. Pick one based on the runner kind, never both.
  3. Will it ever warm via a push to main or dev? If only pull_request writes the cache, every PR cold-starts. Either accept that or add a push trigger on the branch PRs base on that produces the cache (see canary in ci.yml).

If any of those three feels wrong, the cache step is wrong. Re-read this skill before pushing.

planning-and-task-breakdownaddyosmani/agent-skillsBreaks work into ordered tasks. Use when you have a spec or clear requirements and need to break work into implementable tasks. Use when a task feels too large to start, when you need to estimate scope, or when parallel work is possible.Stars: 118kUpdated Claude CodeCodexVerifiedInstall
screenpipethedotmack/claude-memSearch screen and audio history recorded by Screenpipe and optionally save selected context into claude-mem. Use when the user asks to recall desktop activity, a meeting, or something seen in another app through Screenpipe.Stars: 117kUpdated Claude CodeCodexVerifiedInstall
contact-widgetnexu-io/open-designSelf-contained floating chat widget with welcome screen, social links, meeting button, and message input. Single HTML file, zero dependencies.Stars: 108kUpdated Claude CodeCodexVerifiedInstall
meeting-notesnexu-io/open-designMeeting notes page — title bar with attendees, agenda checklist, decisions block, action items table with owners + dates, and a "next meeting" footer. Use when the brief mentions "meeting notes", "minutes", "1:1 notes", "all-hands recap", or "会议纪要".Stars: 108kUpdated Claude CodeCodexVerifiedInstall