Skip to content

chrisgagne/grounded-forge / .claude/skills/ingesting-images

ingesting-images

Verified

Classifies images extracted from an already-ingested source as substantive (diagrams, charts, frameworks) or decorative (photos, covers, logos), and appends substantive entries to the corpus image index.

Ingesting Images

Visual image classification for a source whose text-axis ingestion is already complete. Produces SUBSTANTIVE entries appended to corpus.commons/{corpus}/sources/converted/IMAGE-INDEX.yaml, deletes DECORATIVE files, updates the source's Pass H verification log count.

Completes the image axis when ingesting-resources ran text-only (or image classification was abandoned mid-run from context budget).

When to use

  • The source's IMAGE-INDEX.yaml entries are stub-format or labelled PARTIAL and you want full coverage.
  • A source was ingested text-only originally and you want to add the image axis now.
  • A source's images were re-extracted (e.g., after a substantive PDF revision) and need re-classification.

When not to use

  • Full source ingestion: use ingesting-resources.
  • Searching the corpus for an existing source: use matching-references.
  • Routine reading of an existing classified source: read corpus.commons/{corpus}/sources/converted/IMAGE-INDEX.yaml directly.

Prerequisites

  • The source's deep reference exists at corpus.commons/{corpus}/references/{prefix}-{slug}-deep.md.
  • Either (a) the source PDF / EPUB exists at corpus.commons/{corpus}/sources/original/{slug}.{ext} (so images can be re-extracted), or (b) the image directory at corpus.commons/{corpus}/sources/converted/{md-slug}-images/ already contains extracted files.

Procedure

  1. Resolve scope. If the argument is a single slug, classify that source. If all, scan IMAGE-INDEX.yaml for entries whose comments include PARTIAL or Description not yet authored, and process each affected source in turn.
  2. Ensure images extracted. If {md-slug}-images/ is empty or missing, run python3 scripts/extract-images.py corpus.commons/{corpus}/sources/original/{slug}.{ext} to populate.
  3. Read existing index entries for the source from the corpus's IMAGE-INDEX.yaml to know what's already classified. Do not duplicate entries already present.
  4. Split into waves. For sources over ~100 images, group the file list into waves of 64 (8 batches × 8 files conceptually). One wave = one read-only Sonnet dispatch + one splice + one commit. This bounds context per dispatch, gives a clean recovery point if a session is interrupted, and lets the operator pause between waves without losing state.
  5. Dispatch one read-only Sonnet agent per wave. The agent gets the Read tool only: no Bash, no Edit, no Write. Brief it with: (a) the full list of file paths for this wave, (b) the SUBSTANTIVE / DECORATIVE criteria, (c) the exact YAML-line format for substantive entries, (d) the hard rule that data tables / exercise data sets / source documents are substantive (do not flag as decorative just because they lack diagrams), (e) instruction to read each file and report, never fabricate from filename. The single-agent-of-64 pattern is materially more context-efficient than fanning out 8 parallel agents of 8.
  6. Stage the wave's YAML in /tmp. Write the agent's substantive entries to /tmp/{slug}-wave{N}.yaml. This keeps the in-progress wave out of git and recoverable if validation fails.
  7. Delete decoratives file-by-file. The auto-mode classifier blocks bulk-glob rm of pre-existing files. Chain individual rm commands with && (one per file). Then verify the new disk count.
  8. Splice the staging YAML into IMAGE-INDEX.yaml. Use sed -i.bak to update the PARTIAL header (entries / decoratives deleted / unclassified remaining), then cat /tmp/{slug}-wave{N}.yaml >> IMAGE-INDEX.yaml. Validate with python3 -c "import yaml; yaml.safe_load(open(...))": fails fast on any malformed entry. If the source has a staging file at _planning/staging/{corpus}/images/_ingest_image_index_{slug}.yaml, append the wave there too: the next build_image_index.py --corpus {corpus} replaces the source's index entries with that file's, and would drop a wave it lacks.
  9. Commit each wave. Convention: image-ingestion: {slug} wave {N} — N kept, M deleted [corpus-data] with a body summarising the chapter / topic coverage. Per-wave commits are the recovery point; if a later wave goes sideways, the wave-N commit is intact.
  10. On the final wave, switch PARTIAL → FINAL in the header, with the reconciliation line indexed + deleted = original-extraction-count. Update the planning log entry from PARTIAL to the consolidated final summary.
  11. Append one consolidated line per source to _planning/image-ingestion-log.md (or whichever the operator names). Per-wave entries belong in commit messages, not the log. Drift between log header and actual index entries is a known drift hazard: verify the index count with python3 -c "import yaml; d=yaml.safe_load(open(...)); print(len([e for e in d if e.get('file','').startswith('{slug}-images/')]))" before writing FINAL counts.
  12. Update Pass H at corpus.commons/{corpus}/references/_audit/_ingest_pass_H_{prefix}-{slug}_verification.md with the refreshed image-classification counts (substantive / decorative).
  13. Verify and report: total images processed, substantive count, decorative count, decorative rate, anomalies, recovery actions needed (e.g., orphaned files on disk without index entries).

Hard rule (Source Integrity)

Every entry shipped in IMAGE-INDEX.yaml must have been visually inspected. Do not bulk-classify by filename, file extension, file size, or any metadata-only signal. Those heuristics are unreliable across publishers: the same format carries diagrams in one source and photographs in another. Visual inspection is non-negotiable.

If a source has hundreds of images and one session won't finish them, stop and report progress; the operator re-invokes in a fresh session. Do not silently truncate. Label partial IMAGE-INDEX entries as such (header comment naming what's classified and what isn't).

Hard rule (Staging file integrity)

The per-wave staging file at /tmp/{slug}-wave{N}.yaml is the agent's actual classification output. If a session is summarized and resumes later, do NOT overwrite that file from memory. Either (a) re-dispatch the agent for that wave, or (b) confirm the staging file's contents match real reads by spot-checking 3-5 images against the description. The Write tool happily overwrites without warning: losing the agent's verified output and replacing it with reconstruction from page-number inference is the most common source-integrity failure mode in this skill.

Sonnet failure modes to override

Read-only Sonnet dispatches reliably classify images but consistently make the same three errors. The operator (Opus) must override these:

  • Exercise data tables flagged as decorative because they lack diagrams or annotation. Override: any data table the student computes from is substantive.
  • Variance / source-document scans (invoices, checks, payslips) flagged as decorative. Override: documents that carry pedagogical data are substantive.
  • Cumulative count drift: Sonnet's summary totals can be off-by-one or off-by-two across a wave's classification. Override: recount decoratives yourself from the actual flagged entries, don't trust the summary number.

Related skills

  • ingesting-resources, full ingestion of a new source (text axis plus image axis at pre-flight scope).
  • matching-references, search the corpus for an existing source by topic.
  • creating-distillations, project one existing verified deep reference onto one existing task axis (per-distillation projection).
  • creating-applications, assemble a new application across multiple distillations.
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 197kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall