oberconcejo/VS-CODE / .agents/skills/skill-comply
skill-comply
VerifiedVisualize whether skills, rules, and agent definitions are actually followed — auto-generates scenarios at 3 prompt strictness levels, runs agents, classifies behavioral sequences, and reports compliance rates with full tool call timelines. Use when checking whether agents actually follow the skills, rules, and definitions they were given, rather than assuming they do.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 2.4 KB |
| fixtures/compliant_trace.jsonl | other | 1.0 KB |
| fixtures/noncompliant_trace.jsonl | other | 629 B |
| fixtures/tdd_spec.yaml | config | 1.2 KB |
| prompts/classifier.md | markdown | 1.0 KB |
| prompts/scenario_generator.md | markdown | 1.9 KB |
| prompts/spec_generator.md | markdown | 1.6 KB |
| pyproject.toml | manifest | 360 B |
| scripts/__init__.py | script | 0 B |
| scripts/classifier.py | script | 2.9 KB |
| scripts/grader.py | script | 7.6 KB |
| scripts/parser.py | script | 2.7 KB |
| scripts/report.py | script | 6.3 KB |
| scripts/run.py | script | 4.2 KB |
| scripts/runner.py | script | 9.9 KB |
| scripts/scenario_generator.py | script | 1.8 KB |
| scripts/spec_generator.py | script | 2.2 KB |
| scripts/utils.py | script | 411 B |
| tests/test_classifier.py | script | 3.4 KB |
| tests/test_grader.py | script | 19.5 KB |
| tests/test_parser.py | script | 3.1 KB |
| tests/test_runner.py | script | 21.8 KB |
| tests/test_utf8.py | script | 10.1 KB |
scripts/parser.py
View on GitHub"""Parse observation traces (JSONL) and compliance specs (YAML)."""
from __future__ import annotations
import json
from dataclasses import dataclass
from pathlib import Path
import yaml
@dataclass(frozen=True)
class ObservationEvent:
timestamp: str
event: str
tool: str
session: str
input: str
output: str
@dataclass(frozen=True)
class Detector:
description: str
after_step: str | None = None
before_step: str | None = None
@dataclass(frozen=True)
class Step:
id: str
description: str
required: bool
detector: Detector
@dataclass(frozen=True)
class ComplianceSpec:
id: str
name: str
source_rule: str
version: str
steps: tuple[Step, ...]
threshold_promote_to_hook: float
def parse_trace(path: Path) -> list[ObservationEvent]:
"""Parse a JSONL observation trace file into sorted events."""
if not path.is_file():
raise FileNotFoundError(f"Trace file not found: {path}")
text = path.read_text(encoding="utf-8").strip()
if not text:
return []
events: list[ObservationEvent] = []
for i, line in enumerate(text.splitlines(), 1):
try:
raw = json.loads(line)
except json.JSONDecodeError as e:
raise ValueError(f"Invalid JSON at line {i}: {e}") from e
try:
events.append(ObservationEvent(
timestamp=raw["timestamp"],
event=raw["event"],
tool=raw["tool"],
session=raw["session"],
input=raw.get("input", ""),
output=raw.get("output", ""),
))
except KeyError as e:
raise ValueError(f"Missing required field {e} at line {i}") from e
return sorted(events, key=lambda e: e.timestamp)
def parse_spec(path: Path) -> ComplianceSpec:
"""Parse a YAML compliance spec file."""
if not path.is_file():
raise FileNotFoundError(f"Spec file not found: {path}")
raw = yaml.safe_load(path.read_text(encoding="utf-8"))
steps: list[Step] = []
for s in raw["steps"]:
d = s["detector"]
steps.append(Step(
id=s["id"],
description=s["description"],
required=s["required"],
detector=Detector(
description=d["description"],
after_step=d.get("after_step"),
before_step=d.get("before_step"),
),
))
if "scoring" not in raw:
raise KeyError("Missing 'scoring' section in compliance spec")
return ComplianceSpec(
id=raw["id"],
name=raw["name"],
source_rule=raw["source_rule"],
version=raw["version"],
steps=tuple(steps),
threshold_promote_to_hook=raw["scoring"]["threshold_promote_to_hook"],
)
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 466k | Updated | Claude CodeCodex | Verified | Install |
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 466k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |