ericrisco/rsc-harness / skills/huggingface
huggingface
VerifiedUse when running open models or working on the Hugging Face platform — the Inference Providers router or InferenceClient, Hub repos via the hf CLI, a dedicated Inference Endpoint with scale-to-zero, a Gradio Space with ZeroGPU, picking an open model by task/license/size, or loading one locally with transformers. NOT serving locally on your own machine (that is `ollama`), NOT renting your own GPU box (that is `runpod`), NOT hosted creative image APIs (that is `replicate-images`), NOT fine-tuning with trl/peft (that is `finetuning`).
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 10.0 KB |
| evals/README.md | markdown | 862 B |
| evals/cases.yaml | config | 2.9 KB |
| references/endpoints-and-spaces.md | markdown | 3.0 KB |
| references/hub-and-cli.md | markdown | 2.6 KB |
| references/inference-providers.md | markdown | 3.8 KB |
| scripts/verify.shexecutable | script | 4.2 KB |
references/inference-providers.md
View on GitHub# Inference Providers — client recipes & routing
The router (`https://router.huggingface.co/v1`, or `InferenceClient`) is the unified, serverless
entry point — formerly "Inference API (serverless)". One token reaches 200+ models across partner
providers (Cerebras, Together, Fireworks, Replicate, DeepInfra, Public AI, …) plus `hf-inference`.
HF passes the provider's price through with **no markup**.
All snippets read the token from `os.environ["HF_TOKEN"]` — never hardcode `hf_...`.
## Chat / LLM
```python
import os
from huggingface_hub import InferenceClient
client = InferenceClient(api_key=os.environ["HF_TOKEN"])
resp = client.chat.completions.create(
model="meta-llama/Llama-3.1-8B-Instruct",
messages=[{"role": "user", "content": "Explain MoE in one line."}],
provider="together", # omit to let HF auto-route to an available provider
max_tokens=256,
)
print(resp.choices[0].message.content)
```
Streaming:
```python
for chunk in client.chat.completions.create(
model="meta-llama/Llama-3.1-8B-Instruct",
messages=[{"role": "user", "content": "Count to five."}],
stream=True,
):
delta = chunk.choices[0].delta.content
if delta:
print(delta, end="", flush=True)
```
## Embeddings (feature_extraction)
```python
vecs = client.feature_extraction(
"Retrieval-augmented generation grounds answers in sources.",
model="BAAI/bge-small-en-v1.5",
provider="hf-inference", # embeddings are a CPU task — hf-inference is correct here
)
# vecs is a numpy array of shape (dim,) or (n_tokens, dim) depending on the model
```
## Text-to-image
```python
image = client.text_to_image(
"a minimalist line drawing of a fox",
model="black-forest-labs/FLUX.1-dev",
provider="replicate",
)
image.save("fox.png") # returns a PIL.Image
```
## Speech-to-text (ASR)
```python
text = client.automatic_speech_recognition(
"sample.flac",
model="openai/whisper-large-v3",
)
print(text.text)
```
## OpenAI-compatible drop-in
```python
import os
from openai import OpenAI
client = OpenAI(
base_url="https://router.huggingface.co/v1", # exact host; anything else won't route
api_key=os.environ["HF_TOKEN"],
)
resp = client.chat.completions.create(
model="meta-llama/Llama-3.1-8B-Instruct:together", # optional :provider suffix
messages=[{"role": "user", "content": "hi"}],
)
```
## Provider selection & the hf-inference niche
- `provider="auto"` (default when omitted) routes to an available partner for the model.
- Name a partner explicitly (`together`, `fireworks-ai`, `cerebras`, `replicate`,
`deepinfra`, …) when you need a specific backend's pricing or features.
- **`hf-inference`** (the old serverless API) as of July 2025 focuses on CPU tasks: embeddings,
text-ranking, text-classification, and small historic LLMs (BERT, GPT-2). Do **not** route a
big LLM there — it 404s or stalls. Send 8B/70B/405B chat to a partner provider.
## Billing & org headers
- Monthly Inference-Providers credits: **Free $0.10**, **PRO $2.00**, **Team/Enterprise $2.00 per
seat** (shared across the org). Past credits → pay-as-you-go (you must purchase credits).
- A **Custom Provider Key** (your own key for a partner) bypasses HF billing — the provider bills
you directly and HF credits do not apply.
- Bill an organization instead of your personal account:
```python
client = InferenceClient(api_key=os.environ["HF_TOKEN"], bill_to="my-org")
# or, with the OpenAI client, set the header: X-HF-Bill-To: my-org
```
## Errors & rate limits
- `402` / out-of-credits → buy credits or switch to a Custom Provider Key.
- `404` on a big LLM via `hf-inference` → wrong provider; pick a partner.
- `429` rate limit → back off and retry with jitter; the router does not retry for you.
- Always read the token from env and catch `huggingface_hub.errors.HfHubHTTPError` to surface the
provider's message.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| receiving-code-reviewobra/superpowersUse when receiving code review feedback, before implementing suggestions, especially if feedback seems unclear or technically questionable - requires technical rigor and verification, not performative agreement or blind implementation | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| requesting-code-reviewobra/superpowersUse when completing tasks, implementing major features, or before merging to verify work meets requirements | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| using-git-worktreesobra/superpowersUse when starting feature work that needs isolation from current workspace or before executing implementation plans - ensures an isolated workspace exists via native tools or git worktree fallback | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| verification-before-completionobra/superpowersUse when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |