oberconcejo/VS-CODE / .agents/skills/skill-comply
skill-comply
VerifiedVisualize whether skills, rules, and agent definitions are actually followed — auto-generates scenarios at 3 prompt strictness levels, runs agents, classifies behavioral sequences, and reports compliance rates with full tool call timelines. Use when checking whether agents actually follow the skills, rules, and definitions they were given, rather than assuming they do.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 2.4 KB |
| fixtures/compliant_trace.jsonl | other | 1.0 KB |
| fixtures/noncompliant_trace.jsonl | other | 629 B |
| fixtures/tdd_spec.yaml | config | 1.2 KB |
| prompts/classifier.md | markdown | 1.0 KB |
| prompts/scenario_generator.md | markdown | 1.9 KB |
| prompts/spec_generator.md | markdown | 1.6 KB |
| pyproject.toml | manifest | 360 B |
| scripts/__init__.py | script | 0 B |
| scripts/classifier.py | script | 2.9 KB |
| scripts/grader.py | script | 7.6 KB |
| scripts/parser.py | script | 2.7 KB |
| scripts/report.py | script | 6.3 KB |
| scripts/run.py | script | 4.2 KB |
| scripts/runner.py | script | 9.9 KB |
| scripts/scenario_generator.py | script | 1.8 KB |
| scripts/spec_generator.py | script | 2.2 KB |
| scripts/utils.py | script | 411 B |
| tests/test_classifier.py | script | 3.4 KB |
| tests/test_grader.py | script | 19.5 KB |
| tests/test_parser.py | script | 3.1 KB |
| tests/test_runner.py | script | 21.8 KB |
| tests/test_utf8.py | script | 10.1 KB |
tests/test_parser.py
View on GitHub"""Tests for parser module — JSONL trace and YAML spec parsing."""
from pathlib import Path
import pytest
from scripts.parser import (
ComplianceSpec,
Detector,
ObservationEvent,
Step,
parse_spec,
parse_trace,
)
FIXTURES = Path(__file__).parent.parent / "fixtures"
class TestParseTrace:
def test_parses_compliant_trace(self) -> None:
events = parse_trace(FIXTURES / "compliant_trace.jsonl")
assert len(events) == 5
assert all(isinstance(e, ObservationEvent) for e in events)
def test_events_sorted_by_timestamp(self) -> None:
events = parse_trace(FIXTURES / "compliant_trace.jsonl")
timestamps = [e.timestamp for e in events]
assert timestamps == sorted(timestamps)
def test_event_fields(self) -> None:
events = parse_trace(FIXTURES / "compliant_trace.jsonl")
first = events[0]
assert first.tool == "Write"
assert first.session == "sess-001"
assert "test_fib.py" in first.input
assert first.output == "File created"
def test_parses_noncompliant_trace(self) -> None:
events = parse_trace(FIXTURES / "noncompliant_trace.jsonl")
assert len(events) == 3
assert "src/fib.py" in events[0].input
def test_empty_file_returns_empty_list(self, tmp_path: Path) -> None:
empty = tmp_path / "empty.jsonl"
empty.write_text("")
events = parse_trace(empty)
assert events == []
def test_nonexistent_file_raises(self) -> None:
with pytest.raises(FileNotFoundError):
parse_trace(Path("/nonexistent/trace.jsonl"))
class TestParseSpec:
def test_parses_tdd_spec(self) -> None:
spec = parse_spec(FIXTURES / "tdd_spec.yaml")
assert isinstance(spec, ComplianceSpec)
assert spec.id == "tdd-workflow"
assert len(spec.steps) == 5
def test_step_fields(self) -> None:
spec = parse_spec(FIXTURES / "tdd_spec.yaml")
first = spec.steps[0]
assert isinstance(first, Step)
assert first.id == "write_test"
assert first.required is True
assert isinstance(first.detector, Detector)
assert "test file" in first.detector.description
assert first.detector.before_step == "write_impl"
def test_optional_detector_fields(self) -> None:
spec = parse_spec(FIXTURES / "tdd_spec.yaml")
write_test = spec.steps[0]
assert write_test.detector.after_step is None
run_test_red = spec.steps[1]
assert run_test_red.detector.after_step == "write_test"
assert run_test_red.detector.before_step == "write_impl"
def test_scoring_threshold(self) -> None:
spec = parse_spec(FIXTURES / "tdd_spec.yaml")
assert spec.threshold_promote_to_hook == 0.6
def test_required_vs_optional_steps(self) -> None:
spec = parse_spec(FIXTURES / "tdd_spec.yaml")
required = [s for s in spec.steps if s.required]
optional = [s for s in spec.steps if not s.required]
assert len(required) == 4
assert len(optional) == 1
assert optional[0].id == "refactor"
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 467k | Updated | Claude CodeCodex | Verified | Install |
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 467k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |