Skip to content

hashgraph-online/awesome-codex-plugins / plugins/aklofas/kicad-happy/skills/datasheets

datasheets

Verified

Extract structured specifications from electronic component datasheet PDFs — pinouts, electrical characteristics, peripherals, topology, and features. Cache extractions per project for consumption by schematic and PCB analyzers. Primary consumer infrastructure for `kicad`, `emc`, `spice`, and `thermal` analyzers. Use this skill whenever the user asks to extract, verify, or read specs from a component datasheet; when analyzers need verified IC knowledge (EN pin thresholds, PG presence, USB peripheral speed); or when a review mentions datasheet coverage, extraction quality, or per-MPN specifications. Also triggers on "extract this datasheet", "what are the specs for MPN X", "verify datasheet extraction", or "check pin functions for part Y".

FileKindSize
SKILL.mdskill md11.2 KB
datasheet_types/__init__.pyscript1.9 KB
datasheet_types/base_block.pyscript6.5 KB
datasheet_types/cache_reader.pyscript12.3 KB
datasheet_types/codec.pyscript6.3 KB
datasheet_types/extraction.pyscript8.0 KB
datasheet_types/pinout.pyscript5.2 KB
datasheet_types/regulator.pyscript5.3 KB
datasheet_types/spec_value.pyscript2.4 KB
datasheet_types/trust_gating.pyscript5.0 KB
distributor_core/__init__.pyscript0 B
distributor_core/cli.pyscript2.9 KB
distributor_core/download.pyscript36.7 KB
distributor_core/inputs.pyscript9.0 KB
distributor_core/manifest.pyscript5.7 KB
distributor_core/sync.pyscript21.0 KB
distributor_core/verify.pyscript12.1 KB
examples/abm8g-106-12.000mhz-t.jsonconfig9.4 KB
examples/irlml6344.jsonconfig16.0 KB
examples/lm2596-adj.jsonconfig14.3 KB
examples/lm358.jsonconfig15.6 KB
examples/mbrs540t3g.jsonconfig12.3 KB
examples/stm32f103c8t6.jsonconfig51.1 KB
prompts/base.mdmarkdown4.5 KB
prompts/crystal.mdmarkdown8.8 KB
prompts/diode.mdmarkdown8.3 KB
prompts/mcu.mdmarkdown9.6 KB
prompts/opamp.mdmarkdown10.7 KB
prompts/pinout.mdmarkdown4.7 KB
prompts/regulator.mdmarkdown4.4 KB
prompts/scout.mdmarkdown3.5 KB
prompts/transistor.mdmarkdown12.5 KB
references/cache-layout.mdmarkdown8.4 KB
references/consumer-api.mdmarkdown10.8 KB
references/dispatch-claude-code.mdmarkdown9.4 KB
references/dispatcher-contract.mdmarkdown8.2 KB
references/extraction-pipeline.mdmarkdown1.7 KB
references/extraction-schema.mdmarkdown18.0 KB
references/field-extraction-guide.mdmarkdown14.9 KB
references/quality-scoring.mdmarkdown8.4 KB
schemas/CHANGELOG.mdmarkdown16.0 KB
schemas/base.schema.jsonconfig9.3 KB
schemas/crystal.schema.jsonconfig7.2 KB
schemas/diode.schema.jsonconfig7.9 KB
schemas/extraction.schema.jsonconfig3.7 KB
schemas/fixtures/lm2596-adj.example.jsonconfig14.3 KB
schemas/fixtures/manifest.example.jsonconfig1000 B
schemas/fixtures/minimal.example.jsonconfig2.1 KB
schemas/manifest.schema.jsonconfig3.0 KB
schemas/mcu.schema.jsonconfig12.4 KB
schemas/opamp.schema.jsonconfig9.2 KB
schemas/pinout.schema.jsonconfig5.9 KB
schemas/plan.schema.jsonconfig4.6 KB
schemas/regulator.schema.jsonconfig8.4 KB
schemas/scout.schema.jsonconfig2.9 KB
schemas/spec_value.schema.jsonconfig3.0 KB
schemas/transistor.schema.jsonconfig12.0 KB
scripts/_smoke_v14_roundtrip.pyscript8.9 KB
scripts/datasheet_extract_cache.pyscript21.2 KB
scripts/datasheet_features.pyscript17.1 KB
scripts/datasheet_page_selector.pyscript16.7 KB
scripts/datasheet_score.pyscript15.5 KB
scripts/datasheet_verify.pyscript45.6 KB
scripts/extraction_schema.pyscript1.8 KB
scripts/merge_results.pyscript13.0 KB
scripts/plan_extraction.pyscript9.5 KB
scripts/validate_extraction_result.pyscript4.0 KB
scripts/validate_sanity_vector.pyscript6.4 KB

scripts/merge_results.py

View on GitHub
#!/usr/bin/env python3
"""Merge per-task result files into <mpn>.json.

Reads <mpn>.plan.json + per-task wrapped result files, validates each `data`
against the declared schema (using referencing.Registry for local $ref
resolution), splices into the canonical extraction shape, and writes <mpn>.json.

Failure semantics:
- status:"complete" + schema-valid  → splice into top-level field.
- status:"failed" OR schema-invalid → on first failure, mark task for retry
  (no auto re-dispatch — the dispatcher recipe handles re-run); on second
  failure, partial-merge with {"_extraction_failed": true, "reason": ...}.

Stdlib + jsonschema + referencing (already dev deps). Requires jsonschema and
referencing — extraction_schema imports them when validation runs and raises
RuntimeError when they are missing; no fallback. The script's only purpose is
schema validation; silent-pass on missing deps is dangerous.
"""

from __future__ import annotations

import argparse
import json
import re
import sys
from datetime import datetime, timezone
from pathlib import Path

from extraction_schema import validate_against_schema

_SKILL_PARENT = str(Path(__file__).resolve().parents[1])
if _SKILL_PARENT not in sys.path:
    sys.path.insert(0, _SKILL_PARENT)

from datasheet_types.cache_reader import read_json_object, sanitize_mpn  # noqa: E402
from distributor_core.manifest import write_json_atomic  # noqa: E402


SKILL_ROOT = Path(__file__).resolve().parents[1]          # skills/datasheets
SCHEMA_DIR = SKILL_ROOT / "schemas"
_LEGACY_PREFIX = "skills/datasheets/"


def resolve_skill_resource(rel: str) -> Path:
    """Map a resource reference to a path inside this skill.

    Extraction plans persisted before v3 carry repo-relative strings such as
    'skills/datasheets/schemas/base.schema.json'; a copied skill still owns
    that file at SKILL_ROOT / 'schemas/base.schema.json'. Absolute paths pass
    through unchanged (custom schema locations)."""
    p = Path(rel)
    if p.is_absolute():
        return p
    s = p.as_posix()
    if s.startswith(_LEGACY_PREFIX):
        s = s[len(_LEGACY_PREFIX):]
    return SKILL_ROOT / s


EXTRACTOR_SCHEMA_VERSION = "1.0"


# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------


def _now_iso() -> str:
    return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")


_SAFE_TASK_ID = re.compile(r"[A-Za-z0-9_.\-]+")


def _read_result(cache: Path, mpn: str, task_id: str) -> "dict | None":
    if not _SAFE_TASK_ID.fullmatch(task_id) or task_id in (".", ".."):
        return {
            "task_id": task_id,
            "status": "failed",
            "error": "task id is not a safe file name component",
            "data": None,
        }
    p = cache / f"{sanitize_mpn(mpn)}.{task_id}.result.json"
    if not p.exists():
        return None
    result = read_json_object(p)
    if result is None:
        return {
            "task_id": task_id,
            "status": "failed",
            "error": "result file is not a readable JSON object",
            "data": None,
        }
    return result


def _classify(result: "dict | None", schema: dict,
              pdf_sha256: "str | None" = None) -> "tuple[str, str | None]":
    """Return ('complete'|'failed', error_message). A result must name the
    PDF the plan was built from, so facts from another revision never merge."""
    if result is None:
        return "failed", "no result file found"
    if result.get("status") != "complete":
        return "failed", result.get("error") or "result file status != 'complete'"
    err = validate_against_schema(result.get("data"), schema)
    if err:
        return "failed", f"schema validation: {err}"
    bound = result.get("pdf_sha256")
    if not bound:
        return "failed", ("result has no pdf_sha256, so it cannot be tied to the planned PDF; "
                          "re-dispatch this task and copy pdf_sha256 from the plan into the wrapper")
    if bound != pdf_sha256:
        return "failed", (f"result is bound to PDF {bound}, not the planned PDF "
                          f"{pdf_sha256}; re-dispatch this task")
    return "complete", None


def _splice_field(extraction: dict, task_id: str, role: str, data) -> None:
    """Place `data` at the right top-level field per spec.

    - base task data  → extraction["base"] (object).
    - pinout task data → extraction["base"]["pinout"] (list[Pin]).
      If base was already merged, the pinout task's data overwrites base.pinout.
    - category tasks  → extraction[role] (e.g. extraction["regulator"]).
    """
    if role == "base":
        # Preserve pinout if it was spliced before base (shouldn't happen, but
        # defend against task ordering variations).
        existing_pinout = (
            extraction.get("base", {}).get("pinout")
            if extraction.get("base")
            else None
        )
        extraction["base"] = data
        if existing_pinout is not None:
            extraction["base"]["pinout"] = existing_pinout
    elif role == "pinout":
        # Pinout task data is list[Pin]; nest under base.pinout.
        extraction.setdefault("base", {})["pinout"] = data
    else:
        # Category extension keyed by role (regulator, mcu, ...).
        extraction[role] = data
        cats = extraction.setdefault("categories", [])
        if role not in cats:
            cats.append(role)


def _local_path(pdf_path: str, cache: Path) -> str:
    """Return pdf_path relative to datasheets/ (the cache's parent) when the
    PDF is inside it, so the extraction survives a copied or moved project;
    otherwise pdf_path as given."""
    try:
        return Path(pdf_path).resolve().relative_to(cache.resolve().parent).as_posix()
    except (OSError, RuntimeError, ValueError):
        return pdf_path


def _initial_extraction(plan: dict, cache: Path) -> dict:
    return {
        "schema_version": {"base": "1.0", "categories": {}},
        "source": {
            "manufacturer": "",
            "mpn": plan["mpn"],
            "datasheet_revision": None,
            "datasheet_date": None,
            "source_url": None,
            "local_path": _local_path(plan["pdf_path"], cache),
            "sha256": plan["pdf_sha256"],
            "page_count": None,
            "family_ref": None,
        },
        "extraction": {
            "extracted_at": _now_iso(),
            "extractor_scout": None,
            "extractor_schema_version": EXTRACTOR_SCHEMA_VERSION,
            "quality_score": None,
            "plan_ref": f"{sanitize_mpn(plan['mpn'])}.plan.json",
        },
        "base": {},
        "categories": [],
    }


def _enrich_source_from_scout(extraction: dict, cache: Path, mpn: str) -> None:
    """Overlay scout metadata onto the source block. Silent on missing file."""
    scout = read_json_object(cache / f"{sanitize_mpn(mpn)}.scout.json")
    if scout is None:
        return
    md = scout.get("metadata", {}) or {}
    src = extraction["source"]
    src["manufacturer"] = md.get("manufacturer", src["manufacturer"])
    src["datasheet_revision"] = md.get("datasheet_revision", src["datasheet_revision"])
    src["datasheet_date"] = md.get("datasheet_date", src["datasheet_date"])
    src["source_url"] = md.get("source_url", src["source_url"])
    src["page_count"] = md.get("page_count", src["page_count"])


def _record_outcome(
    plan: dict,
    task_id: str,
    attempts: int,
    status: str,
    err: "str | None",
) -> None:
    outcomes = plan["execution"]["outcomes"]
    for o in outcomes:
        if o["task_id"] == task_id:
            o["attempts"] = attempts
            o["final_status"] = status
            o["last_error"] = err
            return
    outcomes.append(
        {
            "task_id": task_id,
            "attempts": attempts,
            "final_status": status,
            "last_error": err,
        }
    )


# ---------------------------------------------------------------------------
# Core merge logic
# ---------------------------------------------------------------------------


def merge(cache: Path, mpn: str, *, retry_failed: bool = False) -> int:
    plan_path = cache / f"{sanitize_mpn(mpn)}.plan.json"
    if not plan_path.exists():
        print(f"error: plan not found at {plan_path}", file=sys.stderr)
        return 2
    plan = read_json_object(plan_path)
    if plan is None:
        print(f"error: plan at {plan_path} is not a readable JSON object", file=sys.stderr)
        return 2
    if plan["execution"]["started_at"] is None:
        plan["execution"]["started_at"] = _now_iso()

    extraction = _initial_extraction(plan, cache)
    _enrich_source_from_scout(extraction, cache, mpn)

    failed_tasks: list[tuple[dict, str]] = []
    schema_versions: dict[str, str] = {}

    for task in plan["tasks"]:
        tid = task["task_id"]
        role = task["subagent_role"]
        schema_path = resolve_skill_resource(task["schema"])
        schema = json.loads(schema_path.read_text(encoding="utf-8"))
        result = _read_result(cache, mpn, tid)
        status, err = _classify(result, schema, plan.get("pdf_sha256"))

        # Track attempt count — start from any prior recorded attempts.
        prior = next(
            (o for o in plan["execution"]["outcomes"] if o["task_id"] == tid), None
        )
        attempts = (prior["attempts"] if prior else 0) + 1

        if status == "complete":
            _splice_field(extraction, tid, role, result["data"])
            task["status"] = "complete"
            task["result_ref"] = f"{sanitize_mpn(mpn)}.{tid}.result.json"
            _record_outcome(plan, tid, attempts, "complete", None)
            sv = result.get("schema_version")
            if sv and role not in ("base", "pinout"):
                schema_versions[role] = sv
        else:
            failed_tasks.append((task, err or "unknown failure"))
            task["status"] = "failed"
            _record_outcome(plan, tid, attempts, "failed", err)

    if schema_versions:
        extraction["schema_version"]["categories"].update(schema_versions)

    # Failure/retry policy.
    if failed_tasks and not retry_failed:
        # First merge with failures: write plan state, exit nonzero.
        # Caller re-dispatches failed tasks, then re-runs with --retry-failed.
        write_json_atomic(plan_path, plan, sort_keys=True)
        for task, err in failed_tasks:
            print(f"failed: {task['task_id']}: {err}", file=sys.stderr)
        print(
            "re-dispatch failed tasks per dispatcher recipe, "
            "then re-run with --retry-failed",
            file=sys.stderr,
        )
        return 1

    if failed_tasks and retry_failed:
        # Second failure after retry: partial-merge with sentinel.
        for task, err in failed_tasks:
            tid = task["task_id"]
            role = task["subagent_role"]
            sentinel = {"_extraction_failed": True, "reason": err}
            if role == "base":
                extraction["base"] = sentinel
            elif role == "pinout":
                extraction.setdefault("base", {})["pinout"] = sentinel
            else:
                extraction[role] = sentinel
                cats = extraction.setdefault("categories", [])
                if role not in cats:
                    cats.append(role)
            _record_outcome(plan, tid, 2, "partial", err)

    try:
        sys.path.insert(0, str(Path(__file__).resolve().parent))
        from datasheet_score import score_v14_extraction
        extraction["extraction"]["quality_score"] = score_v14_extraction(extraction)["score"]
    except Exception as exc:  # noqa: BLE001
        print(f"warning: quality scoring failed: {exc}", file=sys.stderr)
        extraction["extraction"]["quality_score"] = None
    plan["execution"]["completed_at"] = _now_iso()

    out_path = cache / f"{sanitize_mpn(mpn)}.json"
    write_json_atomic(out_path, extraction, sort_keys=True)
    write_json_atomic(plan_path, plan, sort_keys=True)
    print(f"{mpn}: merged → {out_path}")
    return 0


# ---------------------------------------------------------------------------
# CLI
# ---------------------------------------------------------------------------


def main(argv: "list[str] | None" = None) -> int:
    ap = argparse.ArgumentParser(
        description="Merge per-task result files into <mpn>.json."
    )
    ap.add_argument("mpn")
    ap.add_argument(
        "--cache-dir",
        type=Path,
        default=Path("datasheets/extracted"),
    )
    ap.add_argument(
        "--retry-failed",
        action="store_true",
        help=(
            "Treat this as the post-retry merge — "
            "partial-merge any still-failing tasks."
        ),
    )
    args = ap.parse_args(argv)
    return merge(args.cache_dir, args.mpn, retry_failed=args.retry_failed)


if __name__ == "__main__":
    for _stream in (sys.stdout, sys.stderr):  # UTF-8 on a cp1252 pipe (Windows)
        if hasattr(_stream, "reconfigure"):
            _kw = {"newline": "\n"}
            if (getattr(_stream, "encoding", "") or "").lower().replace("-", "") != "utf8":
                _kw.update(encoding="utf-8", errors="backslashreplace")
            _stream.reconfigure(**_kw)
    sys.exit(main())
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 197kUpdated Claude CodeCodexVerifiedInstall