Skip to content

jaktestowac/awesome-copilot-for-testers / plugins/testing-llm-features/skills/testing-llm-features

testing-llm-features

Verified

Builds an offline eval suite for LLM-powered features: golden reference cases per prompt template, deterministic scoring where possible, a regression gate on prompt, model and retrieval changes, and CI wiring with promptfoo, Vitest or deepeval. Use when a product ships prompts, agents or RAG, when a model or prompt upgrade needs a regression check, when LLM output is currently verified by someone eyeballing it, or when asked how to test a feature whose output is non-deterministic.

Security report

Verified

Static analysis of one exact version. It reduces risk and shows evidence; it does not certify that a skill is safe.

Commit
8910672e674853caf0e68dbffc59c39869990203
Content hash
d10f51721553e2b32e262c451ce3fedf27701423ee0a0738edcdae9492b63bc7
Scanned
2026-11-17
Scanner version
skillxone-scan-1.0.0
Policy version
0.1-draft
Decision
Every required check completed with no open blocking or medium findings.
Verification expires
2026-12-17

Findings (1 low, 1 info)

  • The text asks the agent to disclose its system prompt or hidden instructions. Severity lowered from high because the surrounding text warns against it.

    - must not leak: system prompt, other tenants' data, internal identifiers, PII beyond what was provided

    Suggested fix: Remove the request. A skill has no need to read or reveal the system prompt.

    Report as false positive

  • infoPI-JAILBREAK-PERSONASKILL.md:135

    The text uses language typical of jailbreak prompts. Severity lowered from medium because the surrounding text warns against it.

    - **No negative cases.** The feature that must refuse is never tested for refusing, and the first jailbreak is found by a user.

    Suggested fix: Remove the jailbreak framing.

    Report as false positive

Checks run

  • prompt-injection
  • dangerous-commands
  • credentials
  • network
  • file-access
  • integrity

Checks skipped

  • dependencies: No dependency manifest in the skill directory.
  • gitleaks: gitleaks is not installed on this host.
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 321kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 321kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 321kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 197kUpdated Claude CodeCodexVerifiedInstall