chrisgagne/grounded-forge / .claude/skills/ingesting-images
ingesting-images
VerifiedClassifies images extracted from an already-ingested source as substantive (diagrams, charts, frameworks) or decorative (photos, covers, logos), and appends substantive entries to the corpus image index.
Ingesting Images
Visual image classification for a source whose text-axis ingestion is already complete. Produces SUBSTANTIVE entries appended to corpus.commons/{corpus}/sources/converted/IMAGE-INDEX.yaml, deletes DECORATIVE files, updates the source's Pass H verification log count.
Completes the image axis when ingesting-resources ran text-only (or image classification was abandoned mid-run from context budget).
When to use
- The source's IMAGE-INDEX.yaml entries are stub-format or labelled PARTIAL and you want full coverage.
- A source was ingested text-only originally and you want to add the image axis now.
- A source's images were re-extracted (e.g., after a substantive PDF revision) and need re-classification.
When not to use
- Full source ingestion: use
ingesting-resources. - Searching the corpus for an existing source: use
matching-references. - Routine reading of an existing classified source: read
corpus.commons/{corpus}/sources/converted/IMAGE-INDEX.yamldirectly.
Prerequisites
- The source's deep reference exists at
corpus.commons/{corpus}/references/{prefix}-{slug}-deep.md. - Either (a) the source PDF / EPUB exists at
corpus.commons/{corpus}/sources/original/{slug}.{ext}(so images can be re-extracted), or (b) the image directory atcorpus.commons/{corpus}/sources/converted/{md-slug}-images/already contains extracted files.
Procedure
- Resolve scope. If the argument is a single slug, classify that source. If
all, scan IMAGE-INDEX.yaml for entries whose comments includePARTIALorDescription not yet authored, and process each affected source in turn. - Ensure images extracted. If
{md-slug}-images/is empty or missing, runpython3 scripts/extract-images.py corpus.commons/{corpus}/sources/original/{slug}.{ext}to populate. - Read existing index entries for the source from the corpus's
IMAGE-INDEX.yamlto know what's already classified. Do not duplicate entries already present. - Split into waves. For sources over ~100 images, group the file list into waves of 64 (8 batches × 8 files conceptually). One wave = one read-only Sonnet dispatch + one splice + one commit. This bounds context per dispatch, gives a clean recovery point if a session is interrupted, and lets the operator pause between waves without losing state.
- Dispatch one read-only Sonnet agent per wave. The agent gets the Read tool only: no Bash, no Edit, no Write. Brief it with: (a) the full list of file paths for this wave, (b) the SUBSTANTIVE / DECORATIVE criteria, (c) the exact YAML-line format for substantive entries, (d) the hard rule that data tables / exercise data sets / source documents are substantive (do not flag as decorative just because they lack diagrams), (e) instruction to read each file and report, never fabricate from filename. The single-agent-of-64 pattern is materially more context-efficient than fanning out 8 parallel agents of 8.
- Stage the wave's YAML in
/tmp. Write the agent's substantive entries to/tmp/{slug}-wave{N}.yaml. This keeps the in-progress wave out of git and recoverable if validation fails. - Delete decoratives file-by-file. The auto-mode classifier blocks bulk-glob
rmof pre-existing files. Chain individualrmcommands with&&(one per file). Then verify the new disk count. - Splice the staging YAML into IMAGE-INDEX.yaml. Use
sed -i.bakto update the PARTIAL header (entries / decoratives deleted / unclassified remaining), thencat /tmp/{slug}-wave{N}.yaml >> IMAGE-INDEX.yaml. Validate withpython3 -c "import yaml; yaml.safe_load(open(...))": fails fast on any malformed entry. If the source has a staging file at_planning/staging/{corpus}/images/_ingest_image_index_{slug}.yaml, append the wave there too: the nextbuild_image_index.py --corpus {corpus}replaces the source's index entries with that file's, and would drop a wave it lacks. - Commit each wave. Convention:
image-ingestion: {slug} wave {N} — N kept, M deleted [corpus-data]with a body summarising the chapter / topic coverage. Per-wave commits are the recovery point; if a later wave goes sideways, the wave-N commit is intact. - On the final wave, switch PARTIAL → FINAL in the header, with the reconciliation line
indexed + deleted = original-extraction-count. Update the planning log entry from PARTIAL to the consolidated final summary. - Append one consolidated line per source to
_planning/image-ingestion-log.md(or whichever the operator names). Per-wave entries belong in commit messages, not the log. Drift between log header and actual index entries is a known drift hazard: verify the index count withpython3 -c "import yaml; d=yaml.safe_load(open(...)); print(len([e for e in d if e.get('file','').startswith('{slug}-images/')]))"before writing FINAL counts. - Update Pass H at
corpus.commons/{corpus}/references/_audit/_ingest_pass_H_{prefix}-{slug}_verification.mdwith the refreshed image-classification counts (substantive / decorative). - Verify and report: total images processed, substantive count, decorative count, decorative rate, anomalies, recovery actions needed (e.g., orphaned files on disk without index entries).
Hard rule (Source Integrity)
Every entry shipped in IMAGE-INDEX.yaml must have been visually inspected. Do not bulk-classify by filename, file extension, file size, or any metadata-only signal. Those heuristics are unreliable across publishers: the same format carries diagrams in one source and photographs in another. Visual inspection is non-negotiable.
If a source has hundreds of images and one session won't finish them, stop and report progress; the operator re-invokes in a fresh session. Do not silently truncate. Label partial IMAGE-INDEX entries as such (header comment naming what's classified and what isn't).
Hard rule (Staging file integrity)
The per-wave staging file at /tmp/{slug}-wave{N}.yaml is the agent's actual classification output. If a session is summarized and resumes later, do NOT overwrite that file from memory. Either (a) re-dispatch the agent for that wave, or (b) confirm the staging file's contents match real reads by spot-checking 3-5 images against the description. The Write tool happily overwrites without warning: losing the agent's verified output and replacing it with reconstruction from page-number inference is the most common source-integrity failure mode in this skill.
Sonnet failure modes to override
Read-only Sonnet dispatches reliably classify images but consistently make the same three errors. The operator (Opus) must override these:
- Exercise data tables flagged as decorative because they lack diagrams or annotation. Override: any data table the student computes from is substantive.
- Variance / source-document scans (invoices, checks, payslips) flagged as decorative. Override: documents that carry pedagogical data are substantive.
- Cumulative count drift: Sonnet's summary totals can be off-by-one or off-by-two across a wave's classification. Override: recount decoratives yourself from the actual flagged entries, don't trust the summary number.
Related skills
ingesting-resources, full ingestion of a new source (text axis plus image axis at pre-flight scope).matching-references, search the corpus for an existing source by topic.creating-distillations, project one existing verified deep reference onto one existing task axis (per-distillation projection).creating-applications, assemble a new application across multiple distillations.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 197k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |