modassarislam37/funnel-engineering-system / skills/fes-evaluation
fes-evaluation
VerifiedEvaluate Funnel against standard agent with paired, held-out engineering tasks, reproducible quality evidence and explicit pass/fail metrics; use when assessing reliability or promoting a generated skill.
Funnel Evaluation Laboratory
Build a small balanced evaluation corpus: ordinary feature, bugfix, refactor, auth regression, injection issue, dependency vulnerability, browser UI flow, rollback and a misleading static pattern. Run the same starting commit, permissions, test infrastructure and time budget for baseline agent and Funnel candidate. Keep a held-out set not used in skill construction. Measure executed test success, security issues still open, scope violations and manual interventions. For security tests, use non-production local fixtures and no active exploitation of third-party systems.
Write each paired result in the JSON format under docs/evaluation.md. Evaluate with:
python3 <FES_ROOT>/tools/evaluation.py --input paired-results.json --output comparison.json
The scorer does not call the host agent or verify observations. A pass on the local Python fixture suite is not an agent-performance benchmark. No claim of improvement without actual paired experiments. Use measured outcomes to decide whether a generated skill is worth reusing.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 467k | Updated | Claude CodeCodex | Verified | Install |
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 467k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |