kali20gakki/msAgent / skills/accuracy/uwc-verify
uwc-verify
Verified自动验证 veRL + Megatron-core + vLLM 训推权重转换一致性(UWC)的 5 项检查:Bridge P0/P2、传输 P2_sender/P5、data_ptr 覆盖、G0/G1 前向、dummy vs safetensors 差分。注入采集钩子、采集权重/指针/日志,生成 report.md / report.html / report.json。当用户要验证 RL 训练与推理权重同步、排查 rollout 输出乱码/漏同步、或复现《RL 权重一致性验证》文档时使用。
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 7.4 KB |
| references/reference.md | markdown | 10.9 KB |
| runtime/uwc/__init__.py | script | 373 B |
| runtime/uwc/collectors/__init__.py | script | 245 B |
| runtime/uwc/collectors/stage1_bridge_collector.py | script | 9.8 KB |
| runtime/uwc/collectors/stage2_transfer_collector.py | script | 2.9 KB |
| runtime/uwc/collectors/stage3_vllm_collector.py | script | 12.8 KB |
| runtime/uwc/common.py | script | 4.1 KB |
| runtime/uwc/comparator.py | script | 7.4 KB |
| runtime/uwc/fault.py | script | 8.4 KB |
| runtime/uwc/hooks.py | script | 26.2 KB |
| runtime/uwc/logits_comparator.py | script | 3.7 KB |
| runtime/uwc/name_normalizer.py | script | 7.3 KB |
| runtime/uwc/offline/__init__.py | script | 139 B |
| runtime/uwc/offline/forward_verify.py | script | 11.1 KB |
| runtime/uwc/offline/stage3_diff_test.py | script | 11.4 KB |
| runtime/uwc/offline/uwc_compare.py | script | 14.0 KB |
| runtime/uwc/offline/uwc_summary.py | script | 20.5 KB |
| scripts/uwc_verify.py | script | 44.4 KB |
| uwc.example.yaml | config | 3.4 KB |
runtime/uwc/common.py
View on GitHub"""Shared helpers for UWC collectors and offline tools."""
from __future__ import annotations
import json
import os
import sys
import time
from pathlib import Path
LOG_PREFIX = "[UWC]"
def env_bool(name: str, default: bool = False) -> bool:
value = os.environ.get(name)
if value is None:
return default
return value.strip().lower() in {"1", "true", "yes", "on"}
def get_dump_dir() -> Path:
return Path(os.environ.get("UWC_DUMP_DIR", "/tmp/uwc_dump"))
def get_step() -> int:
try:
return int(os.environ.get("UWC_STEP", "0"))
except ValueError:
return 0
def get_rank() -> int:
"""Return global rank, preferring torch.distributed when initialised."""
try:
import torch.distributed as dist
if dist.is_available() and dist.is_initialized():
return int(dist.get_rank())
except Exception:
pass
try:
return int(os.environ.get("RANK", os.environ.get("LOCAL_RANK", "0")))
except ValueError:
return 0
def is_main_rank() -> bool:
return get_rank() == 0
def stage_enabled(stage: int) -> bool:
"""UWC_ENABLE is the master switch; UWC_STAGEn defaults to on when enabled."""
if not env_bool("UWC_ENABLE"):
return False
return env_bool(f"UWC_STAGE{stage}", True)
def log(message: str) -> None:
print(f"{LOG_PREFIX} {message}", flush=True)
def log_warning(message: str) -> None:
print(f"{LOG_PREFIX} WARNING: {message}", flush=True)
def sync_device() -> None:
"""Synchronise the active accelerator before copying device memory.
NPU IPC / shared-memory transfers are asynchronous; copying without a
synchronise may read a partially written buffer.
"""
try:
import torch
npu = getattr(torch, "npu", None)
if npu is not None and hasattr(npu, "synchronize"):
try:
if npu.is_available():
npu.synchronize()
return
except Exception:
pass
if torch.cuda.is_available():
torch.cuda.synchronize()
except Exception:
pass
def to_cpu(tensor):
"""Detach + copy a tensor to CPU, materialising lazy / GPU-backed views."""
try:
return tensor.detach().to("cpu", copy=True).contiguous()
except Exception:
return tensor.cpu().contiguous()
def ensure_dir(path: Path) -> Path:
path.mkdir(parents=True, exist_ok=True)
return path
def save_pt(path: Path, payload, torch_module=None) -> str:
"""Atomically save a torch payload.
``run`` treats the appearance of a dump file as "stage finished" and kills
the process group, so a half-written ``.pt`` would look complete and then
fail to load. Write a sibling ``.tmp`` file first and rename into place.
"""
if torch_module is None:
import torch as torch_module
path = Path(path)
ensure_dir(path.parent)
tmp = Path(str(path) + ".tmp")
with open(tmp, "wb") as handle:
torch_module.save(payload, handle)
handle.flush()
os.fsync(handle.fileno())
os.replace(tmp, path)
return str(path)
def write_json(path: Path, payload) -> str:
ensure_dir(Path(path).parent)
tmp = Path(str(path) + ".tmp")
with open(tmp, "w", encoding="utf-8") as handle:
json.dump(payload, handle, indent=2, ensure_ascii=False, default=_json_default)
os.replace(tmp, path)
return str(path)
def read_json(path: Path):
with open(path, "r", encoding="utf-8") as handle:
return json.load(handle)
def _json_default(value):
"""Best-effort JSON conversion for torch tensors / numpy types."""
if hasattr(value, "detach"):
try:
return value.detach().cpu().tolist()
except Exception:
return str(value)
if hasattr(value, "item"):
try:
return value.item()
except Exception:
return str(value)
if isinstance(value, (set, tuple)):
return list(value)
return str(value)
def now_iso() -> str:
return time.strftime("%Y-%m-%dT%H:%M:%S%z", time.localtime())
def python_version() -> str:
return sys.version.split()[0]
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |