Skip to content

Diogenesoftoronto/keating / video/keating-surface-tour/.agents/skills/media-use

media-use

Flagged

Agent Media OS, the single skill for every media need in a HyperFrames project. Resolve BGM, SFX, image, icon, brand logo, voice, color grade, or LUT into a frozen local file or paste-ready block + ledger record (one verb, `resolve`); generate via TTS / music / image models when the catalog misses; produce voiceover, transcription, captions, and background removal through one shared audio engine; operate on media (cut / reframe / transform); and reuse assets across projects. Also use for vague feedback that real footage looks dark, flat, boring, should feel retro/camcorder/print/ASCII, needs privacy, or needs a media reveal.

FileKindSize
.gitignoreother17 B
SKILL.mdskill md7.8 KB
audio/assets/sfx/CREDITS.mdmarkdown1.2 KB
audio/assets/sfx/chime.mp3binary26.7 KB
audio/assets/sfx/click-soft.mp3binary11.4 KB
audio/assets/sfx/click.mp3binary11.4 KB
audio/assets/sfx/error.mp3binary50.6 KB
audio/assets/sfx/glitch-1.mp3binary82.4 KB
audio/assets/sfx/glitch-2.mp3binary109.5 KB
audio/assets/sfx/glitch-3.mp3binary96.8 KB
audio/assets/sfx/impact-bass-1.mp3binary66.1 KB
audio/assets/sfx/impact-bass-2.mp3binary81.0 KB
audio/assets/sfx/key-press.mp3binary3.8 KB
audio/assets/sfx/manifest.jsonconfig3.6 KB
audio/assets/sfx/notification.mp3binary76.7 KB
audio/assets/sfx/ping.mp3binary25.8 KB
audio/assets/sfx/pop.mp3binary22.5 KB
audio/assets/sfx/riser.mp3binary313.5 KB
audio/assets/sfx/sparkle.mp3binary56.3 KB
audio/assets/sfx/typing.mp3binary26.2 KB
audio/assets/sfx/whoosh-cinematic.mp3binary173.3 KB
audio/assets/sfx/whoosh-short.mp3binary18.0 KB
audio/assets/sfx/whoosh.mp3binary18.0 KB
audio/references/bgm.mdmarkdown7.8 KB
audio/references/captions/authoring.mdmarkdown8.8 KB
audio/references/captions/motion.mdmarkdown5.5 KB
audio/references/captions/transcript-handling.mdmarkdown5.3 KB
audio/references/remove-background.mdmarkdown8.3 KB
audio/references/requirements.mdmarkdown4.2 KB
audio/references/sfx.mdmarkdown3.4 KB
audio/references/transcribe.mdmarkdown2.7 KB
audio/references/tts-to-captions.mdmarkdown1.2 KB
audio/references/tts.mdmarkdown8.6 KB
audio/scripts/audio.mjsscript12.5 KB
audio/scripts/audio.test.mjsscript1.9 KB
audio/scripts/heygen-tts.mjsexecutablescript4.2 KB
audio/scripts/lib/bgm.mjsscript11.3 KB
audio/scripts/lib/bgm.test.mjsscript1.2 KB
audio/scripts/lib/concurrency.mjsscript527 B
audio/scripts/lib/concurrency.test.mjsscript1.6 KB
audio/scripts/lib/heygen.mjsscript6.9 KB
audio/scripts/lib/heygen.test.mjsscript3.5 KB
audio/scripts/lib/python.mjsscript2.5 KB
audio/scripts/lib/python.test.mjsscript2.6 KB
audio/scripts/lib/sfx.mjsscript5.3 KB
audio/scripts/lib/sfx.test.mjsscript2.8 KB
audio/scripts/lib/tts.mjsscript16.1 KB
audio/scripts/lib/tts.spawn.test.mjsscript5.1 KB
audio/scripts/lib/tts.test.mjsscript6.0 KB
audio/scripts/lyria-recipe.pyscript4.4 KB
audio/scripts/wait-bgm.mjsscript5.4 KB
luts/README.mdmarkdown1.1 KB
luts/index.jsonconfig2.0 KB
references/audio.mdmarkdown1.7 KB
references/grading.mdmarkdown6.6 KB
references/media-treatment-recipes.mdmarkdown35.0 KB
references/media-treatments.mdmarkdown13.6 KB
references/memory.mdmarkdown3.3 KB
references/meta.mdmarkdown6.6 KB
references/operations.mdmarkdown14.1 KB
references/resolve.mdmarkdown8.8 KB
references/setup-providers.mdmarkdown7.1 KB
references/telemetry-dashboard.mdmarkdown4.1 KB
scripts/audio-duck.mjsscript3.9 KB
scripts/dither.mjsscript9.7 KB
scripts/dither.test.mjsscript4.6 KB
scripts/eval.mjsscript14.5 KB
scripts/lib/adopt.mjsscript4.3 KB
scripts/lib/adopt.test.mjsscript2.7 KB
scripts/lib/bgm-provider.mjsscript650 B
scripts/lib/brand-provider.mjsscript1.6 KB
scripts/lib/bundled-sfx-provider.mjsscript3.5 KB
scripts/lib/bundled-sfx-provider.test.mjsscript3.1 KB
scripts/lib/cache.mjsscript4.8 KB
scripts/lib/candidates.mjsscript4.0 KB
scripts/lib/candidates.test.mjsscript5.9 KB
scripts/lib/codex-provider.mjsscript6.5 KB
scripts/lib/codex-provider.test.mjsscript1.3 KB
scripts/lib/coverage.test.mjsscript5.6 KB
scripts/lib/cube-build.mjsscript5.7 KB
scripts/lib/cube-build.test.mjsscript2.7 KB
scripts/lib/cube-validate.mjsscript5.9 KB
scripts/lib/cube-validate.test.mjsscript3.2 KB
scripts/lib/cutlist.mjsscript6.0 KB
scripts/lib/cutlist.test.mjsscript4.0 KB
scripts/lib/duck.mjsscript3.0 KB
scripts/lib/duck.test.mjsscript3.4 KB
scripts/lib/error-diffusion.mjsscript6.5 KB
scripts/lib/error-diffusion.test.mjsscript3.4 KB
scripts/lib/freeze.mjsscript3.0 KB
scripts/lib/freeze.test.mjsscript1.9 KB
scripts/lib/grade-analyzer.mjsscript8.6 KB
scripts/lib/grade-analyzer.test.mjsscript7.7 KB
scripts/lib/heygen-cli.mjsscript7.3 KB
scripts/lib/heygen-cli.test.mjsscript10.0 KB
scripts/lib/heygen-search.mjsscript1.8 KB
scripts/lib/heygen-search.test.mjsscript1.8 KB
scripts/lib/heygen-video-provider.mjsscript3.6 KB
scripts/lib/heygen-video-provider.test.mjsscript12.0 KB
scripts/lib/image-provider.mjsscript1.4 KB
scripts/lib/index-gen.mjsscript1.9 KB
scripts/lib/local-models.mjsscript13.3 KB
scripts/lib/local-models.test.mjsscript5.6 KB
scripts/lib/local-run.mjsscript2.2 KB
scripts/lib/local-run.test.mjsscript1.7 KB
scripts/lib/logo-provider.mjsscript7.0 KB
scripts/lib/logo-provider.test.mjsscript5.7 KB
scripts/lib/ltx-video-provider.mjsscript1.9 KB
scripts/lib/ltx-video-provider.test.mjsscript3.8 KB
scripts/lib/lut-preset-provider.mjsscript8.5 KB
scripts/lib/lut-preset-provider.test.mjsscript6.3 KB
scripts/lib/manifest.mjsscript7.0 KB
scripts/lib/manifest.test.mjsscript11.6 KB
scripts/lib/match.mjsscript1.4 KB
scripts/lib/mflux-provider.mjsscript3.2 KB
scripts/lib/misses.mjsscript1.2 KB
scripts/lib/misses.test.mjsscript2.5 KB
scripts/lib/npx-sync.mjsscript1.3 KB
scripts/lib/npx-sync.test.mjsscript2.0 KB
scripts/lib/parakeet-words.mjsscript1.1 KB
scripts/lib/parakeet-words.test.mjsscript1.4 KB
scripts/lib/prefs-store.mjsscript6.4 KB
scripts/lib/prefs-store.test.mjsscript7.2 KB
scripts/lib/probe.mjsscript1.3 KB
scripts/lib/probe.test.mjsscript1.6 KB
scripts/lib/providers.mjsscript327 B
scripts/lib/recipe-store.mjsscript12.7 KB
scripts/lib/recipe-store.test.mjsscript12.5 KB
scripts/lib/registry.mjsscript9.5 KB
scripts/lib/registry.test.mjsscript9.6 KB
scripts/lib/search.mjsscript906 B
scripts/lib/search.test.mjsscript1.2 KB
scripts/lib/sfx-provider.mjsscript700 B
scripts/lib/specs.mjsscript3.2 KB
scripts/lib/specs.test.mjsscript2.7 KB
scripts/lib/stats.mjsscript3.7 KB
scripts/lib/stats.test.mjsscript5.4 KB
scripts/lib/telemetry.mjsscript8.3 KB
scripts/lib/telemetry.test.mjsscript13.1 KB
scripts/lib/tts-local-provider.mjsscript3.7 KB
scripts/lib/tts-local-provider.test.mjsscript2.6 KB
scripts/lib/usage.mjsscript1.8 KB
scripts/lib/usage.test.mjsscript1.7 KB
scripts/lib/voice-provider.mjsscript1.7 KB
scripts/lib/voice-provider.test.mjsscript1.9 KB
scripts/lib/words.mjsscript764 B
scripts/prefs.mjsscript2.4 KB
scripts/recipe.mjsscript4.0 KB
scripts/resolve.mjsscript43.2 KB
scripts/resolve.test.mjsscript35.8 KB
scripts/transcribe.mjsscript6.5 KB
scripts/transcript-cut.mjsscript7.1 KB

scripts/lib/registry.mjs

View on GitHub
// Provider registry — the v2 contract.
//
// Each media type maps to an ORDERED list of provider entries. Providers are
// tried in order; the first to return a non-null result wins, which keeps
// resolution deterministic (same request -> same provider -> same file ->
// reproducible renders). heygen-CLI is always first for the types it serves.
//
// An entry exposes any of three capability methods — search / generate /
// process — plus { name }. media-use holds no keys; each external tool owns its
// own auth. Providers, by type:
//   - heygen CLI: catalog + TTS, first for every type it serves (OAuth free
//     allowance first, then the user's HeyGen billing path)
//   - mflux: local FLUX-class image gen, spec-selected to the machine's RAM
//     (free, private, offline once cached)
//   - codex CLI: image gen on the user's ChatGPT sub — the better-quality upsell
//     and the fallback when no local model fits
//   - Kokoro (via the hyperframes CLI): local voiceover, free/private fallback
//     when HeyGen credentials are absent or --local-only is requested
//
// Generation is local-first, cloud-upsell. `ctx.provider` forces one provider
// (e.g. "make an image with codex").

import { bgmProvider } from "./bgm-provider.mjs";
import { sfxProvider } from "./sfx-provider.mjs";
import { bundledSfxProvider } from "./bundled-sfx-provider.mjs";
import { imageProvider, iconProvider } from "./image-provider.mjs";
import { brandProvider } from "./brand-provider.mjs";
import {
  svglSearch,
  simpleIconsSearch,
  githubAvatarSearch,
  faviconSearch,
} from "./logo-provider.mjs";
import { heygenTtsGenerate } from "./voice-provider.mjs";
import { heygenVideoGenerate } from "./heygen-video-provider.mjs";
import { ltxVideoGenerate } from "./ltx-video-provider.mjs";
import { localTtsGenerate } from "./tts-local-provider.mjs";
import { codexImageGenerate } from "./codex-provider.mjs";
import { mfluxImageGenerate } from "./mflux-provider.mjs";

// Provider markers: `network` = hits a remote service (skipped by --local-only).
// `paid` = may cost wallet credits after any OAuth/web-plan free allowance
// (documentation for the agent's cost judgment, X4: agent-initiated paid should
// confirm). HeyGen catalog SEARCH is free; HeyGen TTS is free for eligible
// OAuth CLI users up to the monthly allowance, then follows the user's billing.
const A = (name, caps) => ({ name, ...caps }); // local, free
const N = (name, caps) => ({ name, network: true, ...caps }); // remote, free
const P = (name, caps) => ({ name, network: true, paid: true, ...caps }); // remote, paid

// heygen-CLI first. All remote providers are skipped by --local-only.
const REGISTRY = {
  bgm: [N("heygen.audio.sounds", { search: bgmProvider.search })],
  sfx: [
    N("heygen.audio.sounds", { search: sfxProvider.search }),
    A("bundled.sfx", { search: bundledSfxProvider.search }),
  ],
  image: [
    N("heygen.asset.search", { search: imageProvider.search }),
    // Catalog miss -> generate. Local first (best FLUX-class model the machine's
    // RAM can run, spec-selected; free, private, kept under --local-only), then
    // the codex CLI on the user's ChatGPT sub as the better-quality upsell and
    // the fallback when no local model fits.
    A("mflux.local", { generate: mfluxImageGenerate }),
    N("codex.image_gen", { generate: codexImageGenerate }),
  ],
  icon: [N("heygen.asset.search", { search: iconProvider.search })],
  logo: [
    // Official brand marks. Tiers verified by a 54-brand stress test (100%
    // cascade hit); HeyGen asset search is deliberately absent — it returns
    // generic look-alike icons for brand queries. All free, all network →
    // --local-only leaves only the cache rungs.
    N("svgl", { search: svglSearch }),
    N("simple-icons", { search: simpleIconsSearch }),
    N("github.avatar", { search: githubAvatarSearch }),
    N("favicon.ddg", { search: faviconSearch }),
  ],
  voice: [
    // HeyGen TTS first when credentialed so CLI/OAuth users consume the free
    // web-plan allowance (10 min/month) before any paid path. --local-only skips
    // it and keeps Kokoro as the private/offline fallback.
    // Deliberately kept `paid` (X4 confirm-before-call) even though the first
    // 10 min/month are free: the client can't know the remaining allowance, so
    // confirming is safer than risking a silent charge once it's spent. (A
    // tri-state "quota-first, paid after" would need backend quota state.)
    P("heygen.tts", { generate: heygenTtsGenerate }),
    A("kokoro.local", { generate: localTtsGenerate }),
  ],
  video: [
    // HeyGen avatar video first when credentialed; --local-only skips it and
    // keeps LTX as the local fallback.
    P("heygen.video", { generate: heygenVideoGenerate }),
    A("ltx.local", { generate: ltxVideoGenerate }),
  ],
  brand: [
    // Local design spec, not heygen — reads frame.md / design.md tokens.
    A("design_spec", { search: brandProvider.search }),
  ],
  grade: [
    // Local deterministic cascade handled by resolve.mjs so grade records can
    // carry an inline block as well as an optional frozen .cube file.
    A("color_grade.local", { search: async () => null, generate: async () => null }),
  ],
  lut: [
    // Lower-level local LUT generation/freezing path handled by resolve.mjs.
    A("cube_lut.local", { search: async () => null, generate: async () => null }),
  ],
};

function listFor(type) {
  const list = REGISTRY[type];
  if (!list) throw new Error(`unknown media type: ${type}`);
  return list;
}

/** Ordered providers for a type. */
export function getProviders(type) {
  return listFor(type);
}

/** All declared media types. */
export function listTypes() {
  return Object.keys(REGISTRY);
}

/** Provider names available for a type, in cascade order (for --provider validation). */
export function providerNamesFor(type) {
  return listFor(type).map((p) => p.name);
}

/**
 * name -> cost tier ("local" | "network_free" | "network_paid") over a collection
 * of ordered provider lists, i.e. the A / N / P distinction the constructors above
 * already declare. Exported so the conflict rule below is testable against a
 * fixture; production reads the REGISTRY-wide index built from it.
 *
 * A name declared under two media types must carry the same tier in both. If it
 * didn't, "did this resolve cost credit" would depend on which type happened to
 * serve it, and the telemetry property would mean nothing — so this throws at
 * import rather than silently picking one.
 */
export function buildProviderTierIndex(providerLists) {
  const tiers = new Map();
  for (const list of providerLists) {
    for (const p of list) {
      const tier = p.paid ? "network_paid" : p.network ? "network_free" : "local";
      const prior = tiers.get(p.name);
      if (prior && prior !== tier)
        throw new Error(
          `provider "${p.name}" is declared ${prior} under one media type and ${tier} under another`,
        );
      tiers.set(p.name, tier);
    }
  }
  return tiers;
}

const PROVIDER_TIERS = buildProviderTierIndex(Object.values(REGISTRY));

/**
 * Cost tier of a provider by name, or undefined for a name the registry doesn't
 * declare. The registry stays the single owner of "does this cost credit", so
 * dashboards and callers never re-derive it from provider-name string matching.
 */
export function providerTierFor(name) {
  return PROVIDER_TIERS.get(name);
}

/**
 * Does an override token (full name like "codex.image_gen" or a prefix like
 * "codex") match any provider declared for the type? Same match rule as
 * runProviders, so validation and dispatch never disagree.
 */
export function providerMatches(type, want) {
  return providerNamesFor(type).some((n) => n === want || n.startsWith(`${want}.`));
}

/**
 * Back-compat shim for the v1 single-provider API. Returns the first declared
 * provider for the type (tagged with `type`); throws for an unknown type.
 * Kept for v1 callers only — new code should use getProviders/runCapability.
 */
export function getProvider(type) {
  const first = listFor(type)[0] || {};
  return { ...first, type };
}

/**
 * Run a capability across an explicit ordered provider list. Tries each in
 * order, returns the first non-null result, skips providers that don't expose
 * the capability. Pure over its input — the unit-testable core of the cascade.
 *
 * Offline guard: a `network` provider is skipped when `ctx.localOnly` is set —
 * unconditionally, even under a `ctx.provider` override. --local-only is a hard
 * safety flag: it must never make a network call. Forcing a network provider
 * while offline yields a clean miss (the caller explains the conflict), never a
 * silent network request.
 * Provider override: `ctx.provider` (a full name like "codex.image_gen" or a
 * prefix like "codex") pins resolution to matching providers only — this is how
 * a user "make an image WITH codex" forces the upsell instead of taking the
 * free-first default.
 */
export async function runProviders(providers, capability, intent, ctx) {
  const want = ctx?.provider;
  for (const p of providers) {
    if (want && p.name !== want && !p.name.startsWith(`${want}.`)) continue;
    if (p.network && ctx?.localOnly) continue; // --local-only wins, even over --provider
    const fn = p[capability];
    if (typeof fn !== "function") continue;
    const res = await fn(intent, ctx);
    if (res) return res;
  }
  return null;
}

/** Run a capability over the providers for a type (deterministic, heygen-first). */
export async function runCapability(type, capability, intent, ctx) {
  return runProviders(getProviders(type), capability, intent, ctx);
}
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall