Skip to content

MiaoDX/roboclaws / skills/eval-harness

eval-harness

Verified

Select and run Roboclaws validation, product, eval-suite, and live-agent eval rows from a plan, diff, or explicit capability request.

FileKindSize
SKILL.mdskill md7.4 KB
catalog/rows.jsonconfig36.1 KB
skill.jsonconfig1.5 KB

skill.json

View on GitHub
{
  "schema": "roboclaws_skill_manifest_v1",
  "name": "eval-harness",
  "summary": "Select and execute relevant deterministic, product, eval-suite, live-agent eval, and regression-promotion rows.",
  "abstraction_level": "agent_skill",
  "role": "eval_orchestration",
  "status": "active",
  "mcp": {
    "profiles": [],
    "required_tools": [],
    "optional_tools": [],
    "privileged_tools": [],
    "boundary_notes": "This skill orchestrates existing public product routes, eval suites, and private runner mechanics. It does not expose robot capabilities or private evaluator truth."
  },
  "capability_requirements": {
    "profiles": [],
    "backend_variants": [
      "molmospaces_subprocess",
      "isaaclab_subprocess",
      "agibot_gdk"
    ],
    "notes": "Selected rows may require household world, manipulation, episode, visual-grounding, OpenAI Agents SDK live-agent, eval-suite, or simulator runtime support depending on matched signals."
  },
  "evidence_outputs": [
    "eval_harness.json",
    "eval_harness.md",
    "rows/<row-id>/stdout.log",
    "rows/<row-id>/stderr.log",
    "evals/<suite-id>/<stamp>/eval_results.json",
    "evals/<suite-id>/<stamp>/eval_report.html"
  ],
  "lifecycle": {
    "refactor_signals": [
      "selection rules become hard to audit from deterministic source signals",
      "eval orchestration starts moving task behavior out of task skills",
      "multiple repos need the same eval-harness manifest contract"
    ],
    "promote_to_mcp_when": []
  }
}
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall