Skip to content

Ascend/msagent / skills/accuracy/rl-consistency-analysis

rl-consistency-analysis

Verified

训练与推理数据不一致的端到端根因分析。当模块映射/值比较不足以定位问题时使用,可追踪首个可信的分歧边界,过滤融合或结构性误报,遵循生产者-消费者链,并生成包含具体假设和证据的根因报告。

scripts/common.py

View on GitHub
import json
import re
from pathlib import Path
from typing import Any

EPS = 1e-12
THRESHOLDS_OK = 1e-4
THRESHOLDS_WARN = 1e-2

STRUCTURAL_PATTERNS = {
    "linear_q_down_proj": "fused_qkv_a_proj",
    "linear_kv_down_proj": "fused_qkv_a_proj",
}


def load_json(path: Path) -> dict[str, Any]:
    return json.loads(path.read_text(encoding="utf-8"))


def get_module_keys(dump_data: dict[str, Any]) -> list[str]:
    data = dump_data.get("data", {})
    if not isinstance(data, dict):
        return []
    return [key for key in data if isinstance(key, str) and key.startswith("Module.")]


def get_module_items(dump_data: dict[str, Any]) -> list[tuple[str, dict[str, Any]]]:
    data = dump_data.get("data", {})
    if not isinstance(data, dict):
        return []
    return [(k, v) for k, v in data.items() if isinstance(k, str) and k.startswith("Module.")]


def parse_layer_idx(key: str) -> int | None:
    match = re.search(r"\.layers\.(\d+)\.", key)
    return int(match.group(1)) if match else None


def infer_block(key: str) -> str:
    lowered = key.lower()
    if ".mlp." in lowered:
        return "mlp"
    if "self_attn" in lowered or "self_attention" in lowered or "attention" in lowered:
        return "attn"
    if "norm" in lowered:
        return "norm"
    if "embed" in lowered:
        return "embed"
    return "other"


def structural_false_positive(train_key: str, rollout_key: str) -> str:
    lowered_train = train_key.lower()
    lowered_rollout = rollout_key.lower()
    for key, value in STRUCTURAL_PATTERNS.items():
        if key in lowered_train and value in lowered_rollout:
            return f"structural fusion pattern: {key} -> {value}"
    return ""


def is_fused_qkv_pair(train_key: str, rollout_key: str) -> bool:
    return bool(structural_false_positive(train_key, rollout_key))
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 197kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall