Lambenthan/empiricalwiki / i18n/en/skills/ingest
ingest
VerifiedIngest a paper into the wiki — creates pages (papers + concepts + people + claims) and builds all cross-references and graph edges. Trigger whenever the user says "ingest", "add this paper", drops a `.pdf` / `.tex` / arXiv URL, or asks to fold a paper into the knowledge base.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 15.6 KB |
| references/cross-references.md | markdown | 7.0 KB |
| references/dedup-policy.md | markdown | 5.2 KB |
| references/error-handling.md | markdown | 3.9 KB |
| references/init-mode.md | markdown | 5.1 KB |
| references/pdf-preprocessing.md | markdown | 3.9 KB |
references/error-handling.md
View on GitHub# /ingest Error Handling
Open this reference when a step fails. `/ingest` prefers to degrade gracefully: record what happened, continue with what remains, and surface the gap in the final report.
## Source parsing
- **`.tex` parse fails**: fall back to the PDF if one is available in the same source directory.
- **PDF text extraction fails**: fall back to a vision-API pass on the first few pages to recover the title and abstract, then run the preprocessing pipeline in `references/pdf-preprocessing.md` with the recovered title.
- **No readable source at all**: stop and report. Do not create a paper page from a title alone — a paper page without grounded content is noise.
- **INIT MODE input unreadable**: do not attempt to re-prepare the source (INIT MODE is read-only on `raw/`). Stop, record the failure, and let the parent `/init` retry or skip the paper at fan-in.
## External APIs
- **Semantic Scholar unavailable** (`fetch_s2.py paper` errors): skip S2 enrichment, default `importance` to 3, and note in the report that the paper's importance is provisional. Skip the citation backfill step entirely for this ingest.
- **DeepXiv unavailable** (`fetch_deepxiv.py` errors): skip enrichment silently. DeepXiv is optional; its absence is not degraded ingest, just plainer ingest. Do not surface this in the user report unless the user asked about DeepXiv specifically.
- **arXiv source fetch fails**: if the paper is on arXiv but the source archive does not exist or times out, fall through to the PDF path. Record a warning in the final report.
## Slug collisions
- **Generated slug matches an existing page with a different arXiv ID or title**: stop and report. Do not append a numeric suffix silently — a collision between two different papers at the same slug is a signal the wiki has a naming problem that the user should resolve.
- **Generated slug matches an existing page with the same paper**: the paper is already ingested. Report and exit.
- **Within a single ingest, a generated concept or claim slug collides with a different existing page**: append a numeric suffix (`-2`, `-3`, ...) via the tool's built-in collision handling. This is the one case where suffixing is correct — it happens when two genuinely different ideas produce the same slug under the deterministic rule.
## Wiki not initialized
If `wiki/` is missing or empty, run:
```bash
"$PYTHON_BIN" tools/research_wiki.py init wiki/
```
Then retry `/ingest`. Do not attempt to create pages in a non-initialized wiki; `index.md` and `graph/` scaffolding must exist first.
## Partial failure mid-ingest
If an ingest fails after some writes have landed (paper page written, but concept dedup or graph edge fails):
- do not roll back the writes that succeeded
- append a log entry via `tools/research_wiki.py log` describing which steps completed and which are incomplete
- surface the incomplete steps in the user report so the user can run `/edit` or `/check --fix` to finish the job
- in INIT MODE, if the ingest completed successfully, commit inside the worktree before exiting (see `references/init-mode.md`). If the ingest partially failed, do **not** commit the incomplete state; let the parent `/init` handle the failed worktree at fan-in
## When to stop vs. continue
Stop outright when:
- no source can be read at all
- the paper is already ingested (slug + arXiv ID match an existing page)
- a slug collision would silently overwrite a different existing paper
Continue with a warning when:
- one enrichment source (S2 or DeepXiv) is down
- the reference list cannot be parsed (skip step 5; paper ingest still works)
- a single concept or claim dedup call fails transiently (retry once; if it still fails, skip that candidate and note it)
The guiding principle: a partial ingest that preserves a well-shaped paper page is more useful than a clean abort that leaves the wiki unchanged. Partial state is recoverable via `/check` and `/edit`. Lost partial state is not.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| canvas-designanthropics/skillsCreate beautiful visual art in .png and .pdf documents using design philosophy. You should use this skill when the user asks to create a poster, piece of art, design, or other static piece. Create original visual designs, never copying existing artists' work to avoid copyright violations. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| docxanthropics/skillsUse this skill whenever the user wants to create, read, edit, or manipulate Word documents (.docx files) or Word templates (.dotx files). Triggers include: any mention of 'Word doc', 'word document', '.docx', '.dotx', or requests to produce professional documents with formatting like tables of contents, headings, page numbers, or letterheads. Also use when extracting or reorganizing content from .docx or .dotx files, inserting or replacing images in documents, performing find-and-replace in Word files, working with tracked changes or comments, or converting content into a polished Word document. If the user asks for a 'report', 'memo', 'letter', 'template', or similar deliverable as a Word or .docx file, use this skill. Do NOT use for PDFs, spreadsheets, Google Docs, or general coding tasks unrelated to document generation. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| pdfanthropics/skillsUse this skill whenever the user wants to do anything with PDF files. This includes reading or extracting text/tables from PDFs, combining or merging multiple PDFs into one, splitting PDFs apart, rotating pages, adding watermarks, creating new PDFs, filling PDF forms, encrypting/decrypting PDFs, extracting images, and OCR on scanned PDFs to make them searchable. If the user mentions a .pdf file or asks to produce one, use this skill. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| pptxanthropics/skillsUse this skill any time a .pptx or .potx file is involved in any way — as input, output, or both. This includes: creating slide decks, pitch decks, or presentations; reading, parsing, or extracting text from any .pptx or .potx file (even if the extracted content will be used elsewhere, like in an email or summary); editing, modifying, or updating existing presentations; combining or splitting slide files; working with templates (.potx), layouts, speaker notes, or comments. Trigger whenever the user mentions "deck," "slides," "presentation," or references a .pptx or .potx filename, regardless of what they plan to do with the content afterward. If a .pptx or .potx file needs to be opened, created, or touched, use this skill. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |