Skip to content

aipoch/open-science / resources/skills/esmfold2

esmfold2

Verified

Biohub ESMFold2 / ESMFold2-Fast all-atom co-folding (Candido et al. 2026, github.com/Biohub/esm). Single-sequence and MSA modes; protein, DNA, RNA, ligand (CCD/SMILES), modified residues. FoldBench Ab-Ag 50-55%, PPI 70-77% DockQ-pass. Also covers the ESMC-{300M,600M,6B} protein language models from the same release: masked-LM logits, hidden states, mutation scoring, contact prediction, and the SAE interpretability head. MIT-licensed weights on HuggingFace org `biohub`. Use this skill when: (1) Predicting complex structures with single-sequence input, (2) Validating designed binders with ESMFold2-Fast, (3) Running ESMFold2 with MSA input, (4) Getting ESMC embeddings or per-residue mutation scores, (5) Choosing kernel backend and sampling-step settings for paper-faithful throughput.

FileKindSize
.catalog_stampother13 B
SKILL.mdskill md10.7 KB
references/design-hook.mdmarkdown1.4 KB
references/esmc.mdmarkdown3.0 KB

references/esmc.md

View on GitHub
# ESMC language model (Biohub)

ESMC is the Biohub successor to ESM-2: pre-norm transformer + RoPE + SwiGLU +
QK-LayerNorm + bias-free, trained on 2.8 B sequences (UniRef+MGnify+JGI) with
15% MLM. Three sizes: 300M (30L, d=960), 600M (36L, d=1152), 6B (80L, d=2560).
Contact P@L-LR: ESMC-6B 0.725 (+/- 0.002) vs ESM2-15B 0.593. ESMC-6B weights
= 6 safetensors shards, 25.41 GB total.

## MLM logits + hidden states

```python
import torch
from transformers import AutoModelForMaskedLM, AutoTokenizer

model = AutoModelForMaskedLM.from_pretrained(
    "biohub/ESMC-6B", dtype=torch.bfloat16, device_map="auto"
).eval()
tok = AutoTokenizer.from_pretrained("biohub/ESMC-6B")

inp = tok([seq], return_tensors="pt", padding=True)
inp = {k: v.to(model.device) for k, v in inp.items()}
with torch.inference_mode():
    out = model(**inp, output_hidden_states=True)
# out.logits: [B, L+2, 64]  (vocab=64; BOS/EOS bracket)
# out.hidden_states: tuple of 81 tensors [B, L+2, 2560]  (6B has 80 layers + input emb)
```

**Mask token is `<mask>`** (id 32) — use `tok.mask_token`. The native-SDK `_`
convention does NOT apply to the HF tokenizer: `_` is not in the vocab and
encodes to `<unk>`, silently corrupting mutation scores.
Best layers for downstream tasks: 50-60 for function
(paper section A.1.4.2-3); contact head reads final layer.

## Mutation scoring (zero-shot LLR / pseudo-PPL)

Paper Alg 14 (used as the LM regularizer in binder design): for M passes,
mask 15% of mutable positions, compute CE of true tokens against ESMC logits,
average. Zero-shot single-mutant LLR = `logp(mut) - logp(wt)` at the masked
position (Biohub cookbook `esmc_mutation_scoring.ipynb`).

## SAE features (layer 60, k=64, codebook 16384)

```python
from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained("biohub/ESMC-6B", device_map="auto").eval()
tok   = AutoTokenizer.from_pretrained("biohub/ESMC-6B")
sae   = AutoModel.from_pretrained(
    "biohub/ESMC-6B-sae-k64-codebook16384",
    allow_patterns=["config.json","layer_60.safetensors"],
    device=model.device,
)
sae.initialize_layers([60])
model.add_sae_models([sae.layers["60"]])

inp = tok(seq, return_tensors="pt"); inp = {k:v.to(model.device) for k,v in inp.items()}
with torch.inference_mode():
    out = model(**inp)
feat = out["sae_outputs"]["layer60"].to_dense()[0, 1:-1]   # [L, 16384], strip BOS/EOS
```

Example output (Src, 536 aa): shape [536,16384], nnz=34432 (~64/residue).
Paper's atlas SAE = layer 60. Feature metadata API (alpha — endpoint may
change; check biohub.ai/esm docs):
`https://biohub.ai/esm/protein/api/v1alpha1/features/{idx}` returns
`{description, top_100_uniref_ids, ...}`. 5,382 features classified as
"functional_site" in the paper.

## Contact prediction

Paper section A.1.4.1: 20,775 chains at 40% mmseqs cluster, sep>=24, Cbeta<8A.
P@L-LR (95% CI bootstrap n=5000): ESMC-6B 0.725 (+/- 0.002), ESMC-600M 0.589,
ESM2-15B 0.593. The contact head is read from the final attention/logits —
see the `esm.models.esmc` module for the regression head.
brainstormingobra/superpowersYou MUST use this before any creative work - creating features, building components, adding functionality, modifying behavior, or planning anything new, in software or out of it (a talk, a business, a renovation).Stars: 322kUpdated Claude CodeCodexVerifiedInstall
brand-guidelinesanthropics/skillsApplies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. Use it when brand colors or style guidelines, visual formatting, or company design standards apply.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
frontend-designanthropics/skillsGuidance for distinctive, intentional visual design when building new UI or reshaping an existing one. Helps with aesthetic direction, typography, and making choices that don't read as templated defaults.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
mcp-builderanthropics/skillsGuide for creating high-quality MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools. Use when building MCP servers to integrate external APIs or services, whether in Python (FastMCP) or Node/TypeScript (MCP SDK).Stars: 198kUpdated Claude CodeCodexVerifiedInstall