RandallLiuXin/GodotMaker / skills/core/visual-qa
visual-qa
UnverifiedVisual quality assurance: analyze game screenshots for defects, compare against reference, check motion in frame sequences. Supports runtime-native inspection plus Gemini or OpenAI API-backed VQA.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 9.5 KB |
| scripts/criteria.md | markdown | 4.2 KB |
| scripts/dynamic_prompt.md | markdown | 1.9 KB |
| scripts/question_prompt.md | markdown | 2.4 KB |
| scripts/static_prompt.md | markdown | 1.7 KB |
| scripts/visual_qa.py | script | 8.8 KB |
SKILL.md
View on GitHub---
name: visual-qa
description: |
Visual quality assurance: analyze game screenshots for defects, compare against reference, check motion in frame sequences.
Supports runtime-native inspection plus Gemini or OpenAI API-backed VQA.
---
# Visual QA
$ARGUMENTS
<!-- agent-operations: inspect_images, read_project_config, read_skill_reference, run_shell_command -->
Resolve bundled prompts, criteria, and scripts relative to this skill's
published directory using the active runtime's `read_skill_reference` mapping.
Call that resolved directory `<visual-qa-skill-dir>`; do not assume a
runtime-specific skill-root environment variable.
CRITICAL: Find acceptance-blocking problems. Do not rationalize defects that
block the caller-provided Task Context.
CRITICAL: When Task Context is provided, use its `Verify:` criteria and any
design rules it lists as the gate. A reference image is provenance and
auxiliary context only — never a pixel-perfect or style-matching bar. Do not
fail a check for pure reference mismatch (palette, capitalization, wording,
roundedness, spacing, polish, composition) unless it breaks a stated design
rule or the `Verify:` criteria, blocks operation, destabilizes layout, or makes
the visible state logically false.
CRITICAL: When the caller lists design rules, answer every one of them by
`rule_id` and report verdict, evidence, and confidence. Do not assign severity
and do not decide what blocks acceptance — `tools/design_rules.py grade`
derives that from the rule text. See `scripts/criteria.md` § "Design Rule
Checks".
## Execution Steps
### Step 1 - Parse Arguments
If arguments include `--log <path>`, set `VQA_LOG` to that path and remove
those tokens before mode detection.
Reject unsupported argv shapes such as `--screenshot <file> --requirements "..."`.
### Step 2 - Detect Mode
Pick the mode from caller args by matching the first row whose precondition
holds. If no row matches, STOP and tell the caller their args are malformed.
| Mode | Precondition | Required argv shape |
|---|---|---|
| Static | `references/<ref>.png` path AND exactly 1 screenshot path | `Check references/<ref>.png against <screenshot.png> - Goal: ... Requirements: ... Verify: ...` |
| Dynamic | `references/<ref>.png` path AND 2 or more frame paths | `Check references/<ref>.png against <frame_glob> - Goal: ... Requirements: ... Verify: ...` |
| Question | No `references/` path; caller asks a question about screenshots | `--question "..." <screenshot.png> [...]` |
If a reference path appears in the args but the file does not exist on disk,
STOP. Return `verdict: error` with `reason: "reference file missing: <path>"`.
Screenshot paths for Static/Dynamic mode may come from `e2e/screenshots/`,
`reports/fixgap-visual/`, or `reports/verifier-temp/`.
Each call takes one scene's reference plus that scene's screenshot or frame
paths. Reject a single stitched, montage, or contact-sheet image supplied in
place of per-scene paths.
### Step 3 - Select Backend
Read `.godotmaker/config.yaml` for model selection:
- `vqa_model`: primary VQA backend. Supported values are `native`, `codex`, `gemini:<model>`, and `openai:<model>`.
- `vqa_fallback_model`: fallback when the primary backend is unavailable. Supported values are `native`, `codex`, and `none`.
If `.godotmaker/config.yaml` is missing, `vqa_model` is missing, or
`vqa_model` is empty, STOP with `verdict: error`.
Runtime-native VQA means direct image inspection by the selected runtime
provider. `native` uses the active agent runtime. `codex` uses Codex image
inspection, including from a Claude Code orchestration when Codex is available.
API-backed VQA runs `<visual-qa-skill-dir>/scripts/visual_qa.py` with
`--model <vqa_model>`.
Do not silently switch providers except for the explicit `vqa_fallback_model`
path.
### Step 4A - Runtime-Native Execution
Use this path when `vqa_model` is `native` or `codex`, or when an API-backed
model fails and `vqa_fallback_model` is `native` or `codex`.
1. Confirm the mode detected in Step 2.
2. Read the mode prompt:
- Static: `scripts/static_prompt.md`
- Dynamic: `scripts/dynamic_prompt.md`
- Question: `scripts/question_prompt.md`
3. For Static and Dynamic, read `scripts/criteria.md`.
4. Combine the mode prompt, criteria, and Task Context.
5. Read every image file referenced in the arguments using the active runtime
image-reading path.
6. Analyze only the supplied images and constructed prompt. Never inspect game
code.
7. Produce the required output format.
8. Append the debug log entry.
For `native`, use the active agent runtime's image inspection. For `codex`, use
the active runtime mapping's `inspect_images` invocation path. Non-Codex
runtimes must use the mapped `codex exec` image-input path rather than pretending
their own model inspected the files.
### Step 4B - API-Backed Execution
Use this path when `vqa_model` is `gemini:<model>` or `openai:<model>`.
Parse the arguments to construct the command. The script is at
`<visual-qa-skill-dir>/scripts/visual_qa.py`.
Detect the available Python command: run `python3 --version` and
`python --version`, then use whichever succeeds. Cache the result for the
session.
Model selection:
- If `vqa_model` is API-backed, pass it as `--model <value>`.
- If `vqa_model` is `native` or `codex`, use Step 4A.
- If `vqa_model` is missing or empty, STOP with `verdict: error`.
```bash
VQA_MODEL=$(grep -oP 'vqa_model:\s*\K\S+' .godotmaker/config.yaml 2>/dev/null || echo "")
MODEL_FLAG=""
case "$VQA_MODEL" in
"") echo "Missing vqa_model in .godotmaker/config.yaml" >&2; exit 1 ;;
native|codex) echo "Use Runtime-Native Execution for vqa_model: $VQA_MODEL" >&2; exit 1 ;;
gemini:*|openai:*) MODEL_FLAG="--model $VQA_MODEL" ;;
*) echo "Unsupported vqa_model for API-backed visual_qa.py: $VQA_MODEL" >&2; exit 1 ;;
esac
# Static
PYTHON "<visual-qa-skill-dir>/scripts/visual_qa.py" --log ${VQA_LOG:-.vqa.log} $MODEL_FLAG [--context "Goal: ... Requirements: ... Verify: ..."] reference.png screenshot.png
# Dynamic
PYTHON "<visual-qa-skill-dir>/scripts/visual_qa.py" --log ${VQA_LOG:-.vqa.log} $MODEL_FLAG [--context "..."] reference.png frame1.png frame2.png ...
# Question
PYTHON "<visual-qa-skill-dir>/scripts/visual_qa.py" --log ${VQA_LOG:-.vqa.log} $MODEL_FLAG --question "the question" screenshot.png [frame2.png ...]
```
Always pass `--log`. Use `.vqa.log` unless the caller provides a log path.
Print the script output as your response.
If the configured API-backed model is unavailable and `vqa_fallback_model` is
`native` or `codex`, switch to Step 4A and record the fallback in the log entry's
`model` field. If `vqa_fallback_model` is `none`, stop with an error.
### Step 5 - Log Runtime-Native Output
After runtime-native execution produces output, append a debug log entry:
```bash
printf '{"ts":"%s","mode":"MODE","model":"MODEL","query":"QUERY","files":["FILE1","FILE2"],"output":"FIRST_LINE..."}\n' \
"$(date -u +%Y-%m-%dT%H:%M:%SZ)" >> "${VQA_LOG:-.vqa.log}"
```
Set `MODEL` to the `vqa_model` or `vqa_fallback_model` value used for the call.
## Analysis Criteria
Read `scripts/criteria.md` before choosing the final verdict.
## Output Format
### Static / Dynamic
```text
### Verdict: {pass | fail | warning}
### Reference Provenance
{1-3 sentences: context only — how the capture relates to the reference's visual language. Never a pass/fail criterion on its own.}
### Goal Assessment
{1-3 sentences from Task Context. "No task context provided." if none.}
### Issues
{If none: "No issues detected." Otherwise:}
#### Issue {N}: {short title}
- **Type:** style mismatch | visual bug | logical inconsistency | motion anomaly | placeholder
- **Severity:** major | minor | note
- **Acceptance impact:** blocks acceptance | non-blocking | style-only
- **Frames:** {dynamic only: which frames}
- **Location:** {where in frame}
- **Description:** {1-2 sentences}
### Summary
{One sentence.}
```
Severity: major = must fix. minor = non-blocking, record only. note =
cosmetic/style-only, can ship.
### Question Mode
```text
### Verdict: {pass | fail | warning}
### Answer
{Direct, specific, actionable answer. Reference locations, frames, colors, objects.}
### Content Verdict: {pass | fail | warning | n/a}
{Omit only when the question states no content requirements. Covers the content requirements ALONE — required visible elements, readability, layout, motion. A design rule deviation never moves this line.}
### Design Rule Findings
{Omit when the question lists no design rules. Otherwise one line per rule, every rule answered:}
- rule_id: {id} | verdict: {pass | fail | uncertain | not_applicable} | confidence: {high | medium | low} | evidence_conflict: {yes | no} | capture: {file} | evidence: {what you observe}
### Visual Evidence
{What in the screenshots supports the answer. Reference specific frames and locations.}
```
When a question carries both content requirements and design rules, the two are
reported separately on purpose. The overall `### Verdict` is a combined summary
and is **not** a content-only gate: a caller that maps it straight to a blocking
issue would promote an ordinary design rule deviation into one, which the
downstream grader explicitly classifies as non-blocking. Gate content on
`### Content Verdict`; design rule severity comes from
`tools/design_rules.py grade`.
The `Design Rule Findings` block is what
`tools/design_rules.py grade --findings` consumes. Every `rule_id` the question
listed must appear exactly once; a `fail` or `uncertain` without evidence is
rejected there, not silently accepted. Report `evidence_conflict` as `yes` or
`no` — never leave it blank.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |