modassarislam37/funnel-engineering-system / skills/fes-evaluation
fes-evaluation
VerifiedEvaluate Funnel against standard agent with paired, held-out engineering tasks, reproducible quality evidence and explicit pass/fail metrics; use when assessing reliability or promoting a generated skill.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 1.3 KB |
SKILL.md
View on GitHub---
name: fes-evaluation
description: Evaluate Funnel against standard agent with paired, held-out engineering tasks, reproducible quality evidence and explicit pass/fail metrics; use when assessing reliability or promoting a generated skill.
---
# Funnel Evaluation Laboratory
Build a small balanced evaluation corpus: ordinary feature, bugfix, refactor, auth regression, injection issue, dependency vulnerability, browser UI flow, rollback and a misleading static pattern.
Run **the same starting commit, permissions, test infrastructure and time budget** for baseline agent and Funnel candidate. Keep a held-out set not used in skill construction. Measure executed test success, security issues still open, scope violations and manual interventions. For security tests, use non-production local fixtures and no active exploitation of third-party systems.
Write each paired result in the JSON format under `docs/evaluation.md`. Evaluate with:
`python3 <FES_ROOT>/tools/evaluation.py --input paired-results.json --output comparison.json`
The scorer does **not** call the host agent or verify observations. A pass on the local Python fixture suite is not an agent-performance benchmark. No claim of improvement without actual paired experiments. Use measured outcomes to decide whether a generated skill is worth reusing.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |