Skip to content

dominodatalab/domino-ai-plugin / skills/domino-trace-setup

domino-trace-setup

Verified

Set up GenAI tracing for an LLM or agent application in Domino by checking the Domino SDK and MLflow requirements (the DSE needs nothing installed), then adding a tracing_setup.py helper, span evaluators, a config.yaml, and @add_tracing / DominoAgentContext wiring. Use when the user asks to add tracing, observability, or evaluation to an agent or LLM app running on Domino.

FileKindSize
SKILL.mdskill md6.9 KB
assets/config.yamlconfig1.7 KB
assets/tracing_setup.pyscript7.4 KB

assets/config.yaml

View on GitHub
# Domino agent configuration
# Used with: DominoAgentContext(agent_config_path="config.yaml")  (or DominoRun)
#
# The SDK flattens this file (dotted keys, for example models.primary) and logs every
# value as an MLflow parameter on the evaluation run, so any run can be reproduced from
# its parameters in the Domino UI. Without agent_config_path the SDK reads the path in
# DOMINO_AGENT_CONFIG_PATH, defaulting to ./agent_config.yaml.
#
# Replace the model names with the ones the agent actually calls (gateway aliases when the
# agent calls LLM Gateway 2.0, the served model name for a Domino-hosted endpoint).

models:
  # Model the agent calls
  primary: gpt-6.1-sol        # example model name; use one your provider serves

  # Model used by the LLM-as-judge evaluator; must match the judge_model passed to
  # create_llm_judge_evaluator() (or the JUDGE_MODEL environment variable)
  judge: gpt-6.1-sol          # example; keep equal to the judge model the evaluator uses

agents:
  # Default settings for all agents
  default:
    temperature: 0.7
    max_tokens: 1000
    timeout_seconds: 30

  classifier:
    temperature: 0.3
    max_tokens: 500
    system_prompt: "You are a classifier. Categorize inputs accurately."

  responder:
    temperature: 0.7
    max_tokens: 1500
    system_prompt: "You are a helpful assistant. Provide clear, accurate responses."

  evaluator:
    temperature: 0
    max_tokens: 10
    system_prompt: "You are an evaluator. Rate responses objectively."

evaluation:
  # Add the LLM-as-judge evaluator to @add_tracing
  enable_llm_judge: true

  # Quality thresholds your own code can read
  thresholds:
    minimum_quality: 0.7
    minimum_confidence: 0.8

settings:
  retry_count: 3
  timeout_seconds: 30
  batch_size: 10
  verbose: false
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall