Skip to content

comet-ml/opik-skills / skills/opik-online-eval

opik-online-eval

Verified

Take a judge live on production traffic — create an Opik online evaluation rule (LLM-as-judge or Python metric) on a project with sampling, filters, variable mapping, and a cost cap, then confirm new traces are being scored. Works over the SDK's REST client; reads rules and score names via the MCP when connected. Returns the rule, the score name it emits, and how to watch it. Use for "score production traces", "monitor hallucinations in prod", "take this judge live", "set up an online evaluation rule", "alert me when quality drops". Not for offline experiments (use evaluate or compare) or for finding what is already broken (use diagnose).

Security report

Verified

Static analysis of one exact version. It reduces risk and shows evidence; it does not certify that a skill is safe.

Commit
f0fb8b9e6439c71e8ab88f1643bcf52ef68173c5
Content hash
7d5b66afd562fe1565811bfc51c517956e69765548694ec573ce7e9e8fee46b6
Scanned
2026-11-18
Scanner version
skillxone-scan-1.0.0
Policy version
0.1-draft
Decision
Every required check completed with no open blocking or medium findings.
Verification expires
2026-12-18

Findings (1 info)

  • infoINT-BROAD-ALLOWED-TOOLSSKILL.md:5

    SKILL.md pre-approves unrestricted shell access through `allowed-tools`, so the agent may run commands without asking.

    allowed-tools: - Read - Grep - Glob - Bash

    Suggested fix: Restrict the tool, for example `Bash(git status:*)`, or leave approval to the user.

    Report as false positive

Checks run

  • prompt-injection
  • dangerous-commands
  • credentials
  • network
  • file-access
  • integrity

Checks skipped

  • dependencies: No dependency manifest in the skill directory.
  • gitleaks: gitleaks is not installed on this host.
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall