Skip to content

ericrisco/rsc-harness / skills/replicate

replicate

Unverified

Use when running, packaging, deploying or scaling a model on the Replicate platform from code — blocking run versus async predictions, handling FileOutput, deployments with warm private endpoints and autoscaling, packaging with Cog, verifying webhook signatures, and cutting GPU spend. NOT crafting image prompts and parameters or picking image model families (that is `replicate-images`).

Replicate platform operations

This skill is about how a model runs in production on Replicate — clients, async, deployments, Cog packaging, webhooks, scaling, and spend. It is the platform-engineering counterpart to image prompt craft. If the question is what prompt, aspect ratio, or model family produces a good image, that is replicate-images, not this skill. Here the mental model is: a prediction is a job. You either wait for it, poll it, or get pinged about it — and where it runs (shared cold pool vs a private warm deployment) is a cost-and-latency dial you set deliberately.

Pinned facts (verified 2026-06-02): Python client replicate 1.x (latest 1.0.7), Python 3.8+; JS client replicate on npm; auth via REPLICATE_API_TOKEN. A 2.0.0aN alpha exists on PyPI but is NOT the default — pin replicate>=1,<2 so a fresh install never silently pulls it.

Decision: how should this model run?

Pick the row by latency tolerance and whether your process can block. Do not default to run() for everything — a 10-minute job inside a web request will time out and burn a worker.

PatternLatencyBlocks your process?Cost shapeUse when
replicate.run(...)secondsYes — waits to completionper-prediction, shared poolinteractive/quick calls, scripts, CLIs
predictions.create() + pollminutesYes, but you control the loopper-prediction, shared poollong job, a worker can babysit it
predictions.create(webhook=...)minutes+No — fire and forgetper-prediction, shared poollong job, the request must return now
Deployment (private endpoint)low + steadydepends on call style abovewarm floor + per-predictionsustained traffic, need warm/private/autoscale cap

Rule: if a human or HTTP request is waiting longer than a few seconds, do not block on run() — switch to predictions + webhook. Why: synchronous timeouts kill the request but the GPU job keeps running and billing.

Auth & install

export REPLICATE_API_TOKEN=r8_...           # clients read this env var automatically
pip install 'replicate>=1,<2'               # pin: 2.0.0aN is alpha; unpinned can pull it
npm install replicate                        # Node client

Python 3.8+ is required. Never pip install replicate unpinned in a Dockerfile or requirements file — a rebuild months later can resolve to the 2.0 alpha and break your imports. Why: the alpha is a full Stainless/httpx rewrite with a different surface.

Run a model

import replicate

output = replicate.run(
    "black-forest-labs/flux-schnell",
    input={"prompt": "a red bicycle", "num_outputs": 1},
)

Since client 1.0.0, file outputs come back as FileOutput objects, not URL strings. Treating one as a string is the single most common bug.

# Bad — output[0] is a FileOutput, not a str; this writes the repr, not the bytes
open("out.png", "w").write(output[0])

# Good — read the bytes, or take the URL explicitly
with open("out.png", "wb") as f:
    f.write(output[0].read())          # bytes
print(output[0].url)                    # hosted URL if you'd rather link

Rule: call .read() for bytes or .url for the link. Why: silently coercing a FileOutput to a string corrupts the file and the error surfaces far from the cause.

Long-running & async

For jobs over a few seconds, create a prediction instead of blocking:

client = replicate.Client()
prediction = client.predictions.create(
    model="owner/model",
    input={"prompt": "..."},
)
prediction.reload()                      # refresh status from the API
while prediction.status not in ("succeeded", "failed", "canceled"):
    time.sleep(2)
    prediction.reload()

Set a deadline so a stuck job auto-cancels instead of billing forever, and cancel() on cleanup paths. Why: a hung prediction with no deadline is silent, open-ended GPU spend. Poll with a small backoff, not a tight loop — you are charged for the prediction, not the polling, but a tight loop wastes your own process and rate budget. Full polling loop with backoff and 5xx handling is in references/webhooks-and-async.md.

Webhooks

For fire-and-forget, hand Replicate a URL and filter to the events you care about:

client.predictions.create(
    model="owner/model",
    input={"prompt": "..."},
    webhook="https://your.app/hooks/replicate",
    webhook_events_filter=["completed"],   # not every intermediate "logs" event
)

Rule: ALWAYS verify the signature before trusting a webhook body. Why: the URL is public — anyone can POST forged completions to it. Replicate signs each delivery; the secret has a whsec_ prefix. Reconstruct signed content as {webhook-id}.{webhook-timestamp}.{body}, HMAC-SHA256 with the base64-decoded secret, base64-encode, and constant-time compare against the webhook-signature header (a space-separated v1,<sig> list). The clients expose a verification helper; the full Python and Node recipe (including idempotency via webhook-id) is in references/webhooks-and-async.md.

Deployments

A deployment is a private, dedicated API endpoint for one model version that autoscales from zero to hundreds of instances. Reach for it when you need warm instances, a private endpoint, or a hard spend cap — not for one-off runs.

  • min_instances — the warm floor. Set >0 to kill cold starts for latency-sensitive traffic; every warm instance bills whether or not it serves a request.
  • max_instances — the spend cap. The ceiling on concurrent instances; protects you from a traffic spike turning into a surprise bill.
  • Hardware (NVIDIA T4, A100, H100, ...) is switchable without touching code — change the deployment config, not predict.py.
  • Rolling updates ship a new version with no downtime; built-in monitoring covers latency, throughput, error rate, and GPU memory.

You still call a deployment with run() / predictions.create() — it just routes to your private instances. Create/update via HTTP API, the clients, or CLI; fields and the rolling-update flow are in references/deployments-api.md.

Package a custom model with Cog

Cog packages a model into a production container. You need two files; Replicate builds the API server for you on push.

# cog.yaml
build:
  gpu: true
  python_version: "3.11"
  python_packages:
    - "torch==2.4.0"
predict: "predict.py:Predictor"
# predict.py
from cog import BasePredictor, Input, Path

class Predictor(BasePredictor):
    def setup(self):
        # load weights ONCE here, not per request
        self.model = load_model("weights.pth")

    def predict(self, prompt: str = Input(description="text prompt")) -> Path:
        result = self.model(prompt)
        return Path(result)            # Cog uploads the file
cog predict -i prompt="hello"          # run locally (needs Docker)
cog push r8.im/owner/model             # build + push; Replicate hosts it

Rule: load weights in setup(), never in predict(). Why: setup() runs once per instance; predict() runs every request — loading weights per request makes every call pay the model-load cost. Full cog.yaml (system packages, run steps), typed Input(...), GPU config, version pinning, and common build failures are in references/cog-packaging.md. Building requires Docker.

Cost & scaling levers

  • Scale to zero is the default — idle deployments stop billing. Only raise min_instances above zero when cold-start latency actually hurts users, and treat the warm floor as a line item.
  • Deadlines on predictions cap the worst case so a stuck job cannot bill open-ended.
  • Right-size the GPU: do not run a 1B model on an H100. Hardware is a deployment-config change.
  • Cache weights in setup() so per-request work is just inference.

This skill covers Replicate-specific levers only. Tracking total AI spend across many providers as a discipline is cost-tracking.

Anti-patterns

Anti-patternWhy it bitesDo instead
Treating a FileOutput as a URL stringCorrupts files / writes a repr; error surfaces far away.read() for bytes, .url for the link
Blocking run() for a 10-min job in a web requestRequest times out; the GPU job keeps running and billingpredictions.create(webhook=...), return now
Webhook handler with no signature checkThe URL is public; anyone can forge completionsVerify HMAC-SHA256 against whsec_ secret
High min_instances "just in case"Every warm instance bills 24/7 idleScale to zero; raise floor only when cold starts hurt
Loading weights inside predict()Every request pays the model-load costLoad once in setup()
pip install replicate unpinnedA rebuild can pull the 2.0 alpha and break importsPin replicate>=1,<2
A deployment for a one-off runPays for a private endpoint you call onceUse the shared pool via run()
No deadline on a long predictionA hung job bills open-ended, silentlySet a deadline; cancel() on cleanup

References

  • references/cog-packaging.md — full cog.yaml + predict.py, GPU config, build/push, version pinning, build failures.
  • references/webhooks-and-async.md — signature verification (Python + Node), event filters, idempotency, polling with backoff, 5xx retries.
  • references/deployments-api.md — create/update/get deployment via HTTP + clients, autoscaling fields, rolling updates, monitoring metrics, CLI.

scripts/verify.sh statically checks an emitted artifact dir: a cog.yaml declaring build: and predict:, a predict.py Predictor with setup + predict, and warns on an unpinned replicate dependency or a FileOutput written as a string. Presence + key checks only — it does not run Docker.

writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 319kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 196kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 118kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 117kUpdated Claude CodeCodexVerifiedInstall