kali20gakki/msAgent / skills/accuracy/uwc-verify
uwc-verify
Verified自动验证 veRL + Megatron-core + vLLM 训推权重转换一致性(UWC)的 5 项检查:Bridge P0/P2、传输 P2_sender/P5、data_ptr 覆盖、G0/G1 前向、dummy vs safetensors 差分。注入采集钩子、采集权重/指针/日志,生成 report.md / report.html / report.json。当用户要验证 RL 训练与推理权重同步、排查 rollout 输出乱码/漏同步、或复现《RL 权重一致性验证》文档时使用。
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 7.4 KB |
| references/reference.md | markdown | 10.9 KB |
| runtime/uwc/__init__.py | script | 373 B |
| runtime/uwc/collectors/__init__.py | script | 245 B |
| runtime/uwc/collectors/stage1_bridge_collector.py | script | 9.8 KB |
| runtime/uwc/collectors/stage2_transfer_collector.py | script | 2.9 KB |
| runtime/uwc/collectors/stage3_vllm_collector.py | script | 12.8 KB |
| runtime/uwc/common.py | script | 4.1 KB |
| runtime/uwc/comparator.py | script | 7.4 KB |
| runtime/uwc/fault.py | script | 8.4 KB |
| runtime/uwc/hooks.py | script | 26.2 KB |
| runtime/uwc/logits_comparator.py | script | 3.7 KB |
| runtime/uwc/name_normalizer.py | script | 7.3 KB |
| runtime/uwc/offline/__init__.py | script | 139 B |
| runtime/uwc/offline/forward_verify.py | script | 11.1 KB |
| runtime/uwc/offline/stage3_diff_test.py | script | 11.4 KB |
| runtime/uwc/offline/uwc_compare.py | script | 14.0 KB |
| runtime/uwc/offline/uwc_summary.py | script | 20.5 KB |
| scripts/uwc_verify.py | script | 44.4 KB |
| uwc.example.yaml | config | 3.4 KB |
runtime/uwc/fault.py
View on GitHub"""Opt-in runtime fault injection used to validate the uwc-verify hooks.
Every fault is disabled unless its ``UWC_FAULT_*`` environment variable is set,
so normal runs are unaffected. The faults implement the R1-R5
negative-validation experiments of the uwc-verify design notes:
=================== ======================================================
Variable Effect
=================== ======================================================
UWC_FAULT_STAGE1 After the Bridge load: perturb the first mcore parameter
whose name contains ``<substr>``. P2 then disagrees
with P0 (conversion / load defect).
UWC_FAULT_STAGE2 After P2_sender is dumped: perturb the first tensor
handed to the transport. P5 then disagrees with
P2_sender (transport defect).
UWC_FAULT_STAGE3_SKIP Drop matching tensors before vLLM ``_update_weights``,
after P5 was recorded: the receiver never applies them
(missed-sync defect).
UWC_FAULT_STAGE4 Before the mcore forward: shift every element of the
first parameter whose name contains ``<substr>``
(forward defect; a single element is too small to move
the G0/G1 cosine).
UWC_FAULT_STAGE5_SAME_FORMAT
Force both stage-5 runs to record ``load_format=dummy``
(A/B override defect).
=================== ======================================================
Spec syntax is ``<substr>:<delta>[,<substr>:<delta>]``; ``delta`` defaults to
0.01. All helpers are best-effort: a fault that cannot be applied only logs a
``[UWC] fault`` line.
"""
from __future__ import annotations
import os
from .common import env_bool, log
def _spec(name: str):
raw = os.environ.get(name)
if not raw:
return []
specs = []
for chunk in raw.split(","):
chunk = chunk.strip()
if not chunk:
continue
if ":" in chunk:
substr, delta = chunk.rsplit(":", 1)
try:
delta = float(delta)
except ValueError:
substr, delta = chunk, 0.01
else:
substr, delta = chunk, 0.01
if substr:
specs.append((substr, delta))
return specs
def stage1_spec():
return _spec("UWC_FAULT_STAGE1")
def stage2_spec():
return _spec("UWC_FAULT_STAGE2")
def stage3_skip_patterns():
raw = os.environ.get("UWC_FAULT_STAGE3_SKIP")
if not raw:
return []
return [item.strip() for item in raw.split(",") if item.strip()]
def stage4_spec():
return _spec("UWC_FAULT_STAGE4")
def stage5_same_format() -> bool:
return env_bool("UWC_FAULT_STAGE5_SAME_FORMAT")
def _bump(tensor, delta) -> bool:
"""Change one element in place; return True when the value really changed."""
import torch
# Model parameters are leaves that require grad; in-place writes to a view
# of such a leaf are rejected unless autograd tracking is disabled.
with torch.no_grad():
flat = tensor.reshape(-1) if hasattr(tensor, "reshape") else tensor.view(-1)
if flat.numel() == 0:
return False
old = flat[0].item()
for candidate in (old + delta, old * 1.01 + delta, old + 1.0):
try:
flat[0] = candidate
except Exception:
continue
if flat[0].item() != old:
return True
flat[0] = old
return False
def _bump_full(tensor, delta) -> bool:
"""Shift every element in place (stage 4).
A single element is below the cosine threshold on a 4B model: the 2026-11-17
single-chip R4 attempt changed one ``linear_proj`` weight by 0.5 and the
G0/G1 cosine stayed at 0.99988, so the forward check still passed. Shifting
the whole tensor models a wrong conversion / wrong tensor, which is what the
forward check is supposed to catch.
"""
import torch
with torch.no_grad():
try:
before = tensor.detach().clone()
tensor.add_(delta)
except Exception:
return False
try:
return not bool(torch.equal(tensor.detach(), before))
except Exception:
return True
def perturb_named(named, specs, stage, full: bool = False) -> list[str]:
"""Perturb the first parameter per spec; ``named`` is a callable/list."""
if not specs:
return []
bump = _bump_full if full else _bump
items = list(named() if callable(named) else named)
changed = []
for substr, delta in specs:
for name, tensor in items:
if substr in str(name):
try:
if bump(tensor, delta):
changed.append(str(name))
log(f"fault stage{stage}: perturbed {name} (+{delta})")
else:
log(f"fault stage{stage}: could not change {name}")
except Exception as exc:
log(f"fault stage{stage}: perturb failed for {name}: {exc}")
break
else:
log(f"fault stage{stage}: no parameter matched {substr!r}")
return changed
def perturb_models(models, specs, stage, full: bool = False) -> list[str]:
"""Perturb parameters of one model or a list of pipeline chunks."""
if not specs or models is None:
return []
chunks = models if isinstance(models, (list, tuple)) else [models]
changed = []
for chunk in chunks:
named = getattr(chunk, "named_parameters", None)
if callable(named):
changed.extend(perturb_named(named, specs, stage, full=full))
return changed
def perturb_weights(weights, specs, stage) -> list[str]:
"""Corrupt the tensors handed to the transport (``[(name, tensor), ...]``).
The matched entry is replaced by a perturbed copy so the fault hits what is
sent without writing through to the training model (stage-2 tensors may
alias model parameters). Only matched tensors are copied, so a one-layer
fault costs one tensor instead of duplicating the whole export set.
"""
if not specs or weights is None:
return []
is_dict = hasattr(weights, "items")
items = list(weights.items()) if is_dict else list(weights)
changed = []
for substr, delta in specs:
for index, (name, tensor) in enumerate(items):
if substr in str(name):
try:
target = tensor.detach().clone()
if _bump(target, delta):
if is_dict:
weights[name] = target
else:
weights[index] = (items[index][0], target)
items[index] = (items[index][0], target)
changed.append(str(name))
log(f"fault stage{stage}: perturbed {name} (+{delta})")
else:
log(f"fault stage{stage}: could not change {name}")
except Exception as exc:
log(f"fault stage{stage}: perturb failed for {name}: {exc}")
break
else:
log(f"fault stage{stage}: no tensor matched {substr!r}")
return changed
def drop_weights(weights, patterns, stage=3) -> list[str]:
"""Remove matching entries from a list/dict in place, after P5 is recorded."""
if not patterns or weights is None:
return []
dropped = []
def matches(name) -> bool:
return name is not None and any(pattern in str(name) for pattern in patterns)
if isinstance(weights, dict):
for name in list(weights):
if matches(name):
weights.pop(name)
dropped.append(str(name))
elif isinstance(weights, list):
keep = []
for item in weights:
name = item[0] if isinstance(item, (list, tuple)) and item else None
if matches(name):
dropped.append(str(name))
else:
keep.append(item)
if dropped:
weights[:] = keep
else:
log(
f"fault stage{stage}: weights is {type(weights).__name__}, "
"not a list/dict; skip fault not applied"
)
return dropped
if dropped:
log(f"fault stage{stage}: dropped {len(dropped)} tensor(s), e.g. {dropped[0]}")
else:
log(f"fault stage{stage}: no tensor matched the skip patterns")
return dropped
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |