# Literature Connector Skill
Trigger this skill when the user asks to **find recent papers**, **check the literature**, **connect a finding to PubMed**, or **search for related publications** on a given topic in relation to a specific lab result.
## What this skill does
Searches both **PubMed** (peer-reviewed publications) and preprint servers indexed by Europe PMC (bioRxiv, Authorea Preprints, Research Square, medRxiv, etc.) for literature published in the last 12 months matching a user-supplied topic, fetches abstracts, and writes a brief (2–3 sentence, no direct quotes) characterization of each paper's relationship to a user-supplied lab finding. Relationships are categorized as one of: **Supports**, **Conflicts**, **Adds context**, or **Suggests next step**. Every entry is clearly labeled by source — **PubMed** or the actual preprint publisher name (e.g., **bioRxiv preprint**, **Authorea Preprints preprint**) — since preprints have not been peer-reviewed. If no genuinely relevant recent literature is found, the skill says so plainly — it does not stretch a loose match.
## Inputs
The skill takes two inputs from the user's message:
- **TOPIC** — the biological/scientific subject to search (e.g., `"PolyIC immune priming in Pacific oysters"`).
- **FINDING** — one or more sentences describing the specific lab result to compare against (e.g., `"HSP70 is driven by temperature stress, VIPERIN is driven by PolyIC, and cGAS-STING is unresponsive to PolyIC in Magallana gigas"`).
If the user omits the finding, ask for it before proceeding — the skill cannot categorize papers without it.
## Caching (disk cache for API responses)
To avoid re-hitting the PubMed and Europe PMC APIs for identical searches, this skill caches the **raw API response** of each live call on disk. Raw responses (not the filtered/categorized result) are cached because the relevance filter and categorization depend on FINDING, which can change between runs while the underlying query stays the same — so a cached raw response for a given TOPIC query can be re-analyzed against a new FINDING.
**Cache location:** `.cache/literature-connector/` (relative to the repo root). Create the directory if it does not exist. This path is git-ignored — cached responses are never committed.
**What gets cached:** each of the three live API calls independently — the PubMed `esearch` call (Step 2), the PubMed `efetch` call (Step 3), and the Europe PMC `search` call (Step 4).
**Cache key:** the SHA-256 hash of a canonical string combining the call type, the exact query string sent, and the date window used:
```
<call-type>|<exact query string>|<date window>
```
where:
- `<call-type>` is one of `pubmed-esearch`, `pubmed-efetch`, `europepmc-search`.
- `<exact query string>` is the literal term/id sent to the API (the `term=` value for esearch, the comma-joined PMID list for efetch, the `query=` value for Europe PMC), before URL-encoding.
- `<date window>` is `reldate=365;today=YYYY-MM-DD` for the PubMed calls, or `START_DATE..END_DATE` for Europe PMC.
The cache filename is `<sha256-hex>.json`.
**Cache entry format** (JSON):
```json
{
"call_type": "pubmed-esearch | pubmed-efetch | europepmc-search",
"query": "<exact query string>",
"date_window": "<date window string>",
"request_url": "<full request URL, for debugging>",
"fetched_at": "<ISO 8601 UTC timestamp of the original successful live fetch>",
"response": "<raw response body, verbatim>"
}
```
**Read/write procedure — apply to every one of the three API calls:**
1. **Before** making the live call, compute the cache key and check for `.cache/literature-connector/<key>.json`.
2. **Cache hit (fresh):** if the file exists and `now - fetched_at < 48 hours`, use its `response` field in place of the live call. Do **not** make the live request. Record that this call was served from cache and remember its `fetched_at` timestamp for the output header.
3. **Cache miss or stale:** if the file is absent, or `fetched_at` is 48 hours old or older, make the live call as normal.
- **On a successful (HTTP 200) fetch:** write/overwrite the cache entry with a fresh `fetched_at` timestamp and the raw response body, then proceed using the live response.
- **On a failed fetch (non-200, timeout, etc.):** follow the existing failed-fetch handling in Steps 3/5 and Step 10 exactly — report the record as "unable to retrieve" and exclude it. **Do not** write a cache entry, and **do not** fall back to a stale cached entry to fill the gap. A stale cache entry is never a substitute for a failed live fetch — caching must never be used as a workaround for a failed fetch, and must never let stale content masquerade as a live result.
**Suggested Bash helpers** (run via the shell; adjust as needed):
```bash
mkdir -p .cache/literature-connector
# compute key:
KEY=$(printf '%s' "pubmed-esearch|${QUERY}|reldate=365;today=$(date -u +%F)" | sha256sum | cut -d' ' -f1)
CACHE_FILE=".cache/literature-connector/${KEY}.json"
# freshness check (48h = 172800s), reading fetched_at from the JSON:
if [ -f "$CACHE_FILE" ]; then
FETCHED=$(python3 -c "import json,sys;print(json.load(open('$CACHE_FILE'))['fetched_at'])")
# compare epoch of $FETCHED to now; use cache if (now - fetched) < 172800s
fi
# on a fresh live 200, write:
# fetched_at = $(date -u +%Y-%m-%dT%H:%M:%SZ)
```
## Steps
Run Steps 1–3 (PubMed) and Steps 4–5 (bioRxiv) in parallel, then merge at Step 6.
---
### PubMed Search (Steps 1–3)
### 1. Build the PubMed search query
Transform the TOPIC into a PubMed query string. Apply the following rules:
- Identify the key species name and the key process/assay/molecule, and combine them with AND.
- Use `reldate=365` in the API call to restrict to the last 12 months — do not embed literal date strings in the query.
- Example: topic `"PolyIC immune priming in Pacific oysters"` → query `PolyIC AND "Pacific oyster" AND immune priming`.
- Keep the query short — 3–5 terms, no wildcards unless clearly needed.
### 2. Run esearch
**Check the cache first** (see the Caching section): compute the `pubmed-esearch` key from the exact `term` value and the date window, and if a fresh (< 48h) entry exists, use its stored response instead of calling the API. Otherwise fetch live and, on HTTP 200, write a fresh cache entry.
Fetch PMIDs via:
```
https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pubmed&term=QUERY&datetype=pdat&reldate=365&retmax=15&retmode=json&usehistory=n
```
Replace `QUERY` with the URL-encoded query string from step 1.
Parse `esearchresult.idlist` from the JSON response. If the list is empty:
- Try one fallback: broaden the query by dropping the least-specific term (e.g., drop the assay name, keep organism + molecule).
- If the fallback also returns an empty list, note "No papers matching [TOPIC] indexed on PubMed in the last 12 months." and continue to the bioRxiv search — do not stop the whole skill.
Cap processing at the first 10 PMIDs even if more are returned.
### 3. Fetch titles and abstracts via efetch
**Check the cache first** (see the Caching section): compute the `pubmed-efetch` key from the comma-joined PMID list and the date window, and if a fresh (< 48h) entry exists, use its stored response instead of calling the API. Otherwise fetch live and, on HTTP 200, write a fresh cache entry. If the live efetch fails, apply the failure handling below — do **not** fall back to a stale cache entry.
For the list of PMIDs, fetch full records in abstract text format:
```
https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&id=PMID1,PMID2,...&rettype=abstract&retmode=text
```
**Fetch failure handling:** If the efetch call returns a non-200 HTTP status (including 403 Forbidden), do not generate any summary or characterization for those records. Report each affected PMID as "Unable to retrieve abstract — excluded from results" in the output header counts. Do not use the paper's title, any prior knowledge, or contextual inference to reconstruct what the abstract might say.
Parse the plain-text response. Each record begins with `PMID- ` and contains `TI - ` (title), `AB - ` (abstract), `DP - ` (publication date), and `AU - ` (authors).
Extract for each record:
- PMID
- Title
- Publication date (year + month)
- First author's last name
- Abstract text (full)
If an individual record in the response is missing its `AB` field, treat it the same as a failed fetch: mark it "unable to retrieve" and exclude it from all downstream processing. Do not summarize a paper based on its title alone.
---
### bioRxiv Search (Steps 4–5)
**Note on API choice:** `api.biorxiv.org` retrieves preprints by date range only (no keyword search), returning ~4,000–5,000 records per 3-week window — impractical for topic-specific retrieval. Instead, use the **Europe PMC API**, which indexes all bioRxiv preprints and supports keyword search. Results with `source: PPR` in the response are preprints from bioRxiv or medRxiv. This is the standard machine-access method for searching bioRxiv content by keyword.
### 4. Build the Europe PMC preprint query
Use the same core keywords as the PubMed query. Compute the 12-month start date as today's date minus 365 days, formatted as `YYYY-MM-DD`.
**Check the cache first** (see the Caching section): compute the `europepmc-search` key from the exact `query` value (keywords + `SRC:PPR` + date filter) and the `START_DATE..END_DATE` date window, and if a fresh (< 48h) entry exists, use its stored response instead of calling the API. Otherwise fetch live and, on HTTP 200, write a fresh cache entry. If the live fetch fails, apply the existing failure handling — do **not** fall back to a stale cache entry.
Construct the URL:
```
https://www.ebi.ac.uk/europepmc/webservices/rest/search?query=KEYWORDS+SRC:PPR+AND+FIRST_PDATE:[START_DATE+TO+END_DATE]&resultType=core&pageSize=10&format=json
```
- Replace `KEYWORDS` with the URL-encoded topic keywords (same terms as PubMed, without field tags).
- Replace `START_DATE` with today minus 365 days (e.g., `2025-07-21`).
- Replace `END_DATE` with today's date (e.g., `2026-07-21`).
- `SRC:PPR` restricts results to preprints.
- `resultType=core` returns full abstracts.
If the query returns zero results, try one fallback by dropping the least-specific term (same strategy as PubMed step 2). If the fallback is also empty, note "No preprints matching [TOPIC] found on bioRxiv in the last 12 months." and continue with PubMed results only.
Cap processing at the first 10 preprint results.
### 5. Extract preprint metadata
From the `resultList.result` array in the JSON response, extract for each record:
- `id` (e.g., `PPR1225898`) — used as the preprint identifier
- `title`
- `firstPublicationDate`
- First author name from `authorList.author[0]` (use `fullName` or `lastName`)
- `abstractText`
- `doi`
- **Publisher name** — check these fields in order and use the first non-empty value: `publisher`, `bookOrReportDetails.publisher`, `journalInfo.journal.title`. This is the actual preprint server name (e.g., "bioRxiv", "Authorea Preprints", "Research Square", "medRxiv"). If none of these fields are present, fall back to `"Preprint"`.
- Confirm `source` is `PPR`; discard any record where source is not `PPR`
- Skip any record where `abstractText` is missing — do not guess content from the title alone
**Authoritative abstract source:** The `abstractText` field returned by Europe PMC is the sole authoritative basis for preprint summaries. Do **not** attempt to fetch the preprint source URL (bioRxiv.org, Authorea, medRxiv.org, etc.) to retrieve or supplement the abstract — these fetches are unreliable and frequently return 403 errors or authentication walls. If `abstractText` is absent or empty in the Europe PMC response, skip that record entirely; report it as "abstract not available via Europe PMC — excluded." Never infer or reconstruct abstract content from a failed fetch, from the preprint ID, from the title, or from prior knowledge about what a paper with that title would likely say.
---
### Merge and Filter (Steps 6–8)
### 6. Deduplication
Before applying the relevance filter, check for overlap between the PubMed results and the bioRxiv preprint results:
- For each bioRxiv preprint, check whether its title closely matches any PubMed paper title (high word overlap, allowing for minor wording differences between preprint and published versions).
- If a match is found: **keep the PubMed entry only**, and append to its citation line: `(originally posted as bioRxiv preprint: https://doi.org/[bioRxiv DOI])`.
- Remove the matching bioRxiv entry from the preprint pool.
### 7. Relevance filter
Apply the same strict relevance test to all remaining PubMed papers and bioRxiv preprints:
**A paper is relevant if and only if** its abstract discusses at least one specific entity shared with FINDING or TOPIC (same gene, protein, pathway, organism species, or experimental method). Shared vague domain (e.g., "marine biology", "immunity") does not qualify.
Discard papers that fail this test. If all papers from a given source are discarded, note this plainly (e.g., "Retrieved N bioRxiv preprints on [TOPIC] but none specifically addressed [key entity from FINDING].").
### 8. Categorize and summarize each relevant paper
For each paper that passes the relevance filter, write a single entry with:
1. **Source badge** — one of:
- `[PubMed]` for peer-reviewed publications
- `[PUBLISHER preprint — not peer-reviewed]` for preprints, where `PUBLISHER` is the actual publisher name extracted in Step 5 (e.g., `[bioRxiv preprint — not peer-reviewed]`, `[Authorea Preprints preprint — not peer-reviewed]`, `[Research Square preprint — not peer-reviewed]`). Never hardcode "bioRxiv" — always use the value from the API response.
2. **Relationship tag** — choose exactly one:
- `Supports` — abstract reports a result consistent with or confirming the FINDING
- `Conflicts` — abstract reports a result that contradicts the FINDING (note what specifically differs)
- `Adds context` — abstract provides background, mechanism, or comparative data that informs interpretation of the FINDING without directly confirming or contradicting it
- `Suggests next step` — abstract identifies a gap, method, or question that follows naturally from the FINDING
3. **Summary** — 2–3 sentences written in your own words. No direct quotes from the abstract. Cover: what the paper did, what it found, and why that matters for the FINDING.
4. **Citation line**:
- For PubMed: `[First Author et al., Year] · PMID: [PMID] · https://pubmed.ncbi.nlm.nih.gov/[PMID]/`
- For bioRxiv: `[First Author et al., Year] · DOI: [doi] · https://doi.org/[doi]`
---
### 9. Assemble the output
Format the final output as:
```
# Literature Connector — [TOPIC]
**Lab finding:** [FINDING]
**PubMed query:** [exact query string]
**bioRxiv query:** [exact keywords + date range used]
**Date range:** last 12 months (from [today minus 365 days] to [today])
**PubMed papers retrieved:** [N] · **relevant:** [M]
**bioRxiv preprints retrieved:** [N] · **relevant:** [M]
**Data freshness:** [For each of the three API calls, state "live" or "from cache". For any served from cache, give the original fetch timestamp, e.g. "PubMed esearch/efetch: from cache (fetched 2026-08-02T14:31:09Z); Europe PMC: live". If all live, write "all live (no cache used)".]
**Excluded (retrieval failed):** [N total — list each as: PMID/DOI · reason (e.g. HTTP 403, abstract field absent, timeout); or "none" if all fetches succeeded]
---
## Relevant Literature
### [Relationship tag] [Source badge]: [Paper Title]
[First Author et al., Year] · [PMID or DOI] · [link]
[2–3 sentence summary]
---
[repeat for each relevant paper — PubMed entries first, then bioRxiv preprints]
## Summary
[2–4 sentences synthesizing what the literature as a whole says about the FINDING. Explicitly call out which supporting findings come from peer-reviewed publications vs. preprints — e.g., "Two peer-reviewed papers support this finding; one additional preprint points in the same direction but has not yet been peer-reviewed."]
```
If no relevant papers passed the filter from either source, replace the "Relevant Literature" section with a plain-language explanation.
---
### 10. Caveats
Always include at the bottom of the output:
> Coverage limited to papers indexed on PubMed and preprints discoverable via Europe PMC (which indexes bioRxiv, Authorea Preprints, Research Square, medRxiv, and other preprint servers), restricted to the last 12 months. **Preprints have not been peer-reviewed** and should be interpreted with appropriate caution. Preprints that have since been published in a peer-reviewed journal are reported in their published form where detected.
Additional rules:
- Do not speculate about paper content beyond what is stated in the retrieved abstract.
- Never fabricate a PMID, DOI, or paper title.
- **Failed fetches — hard rule:** If any HTTP request returns a non-200 response (including 403 Forbidden, 401 Unauthorized, 500, etc.), that record must be reported as "unable to retrieve" and excluded from all results. Do not generate any summary, characterization, relationship tag, or inference from a failed fetch under any circumstances — not from the title, not from the DOI, not from contextual knowledge about the authors or topic, not from assumptions about what such a paper would likely say. A paper only appears in the results section if its abstract was successfully retrieved and is being directly cited.
- **Caching is not a fetch fallback:** the disk cache (see Caching section) may only supply a response when the matching entry is **fresh (< 48 hours old)**. A stale or missing cache entry combined with a failed live fetch is still a failed fetch — report it as "unable to retrieve" and exclude it. Never reach for stale cached content, and never fabricate content, to paper over a failed live fetch.