wanshuiyin/Auto-claude-code-research-in-sleep / skills/citation-crosscheck
citation-crosscheck
VerifiedDouble-blind bibliographic cross-check: one agent extracts each cited entry verbatim from the local .bib, a second independently re-fetches the same paper's record from the web while blind to the .bib, and a third extractor emits a mechanical field-by-field delta to catch dropped/re-ordered authors, wrong years, wrong venues, and bad DOIs/arXiv ids. The main agent (never a shard) assigns the MATCH/MINOR/MISMATCH verdict per reference, and exports a side-by-side spreadsheet. Use when user says "citation crosscheck", "double-blind citation check", "verify bib against scholar", "bib field audit", "引用交叉核对", or before submission when bibliography FIELD accuracy (not just existence) must be certified.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 35.7 KB |
| scripts/cx_verify.py | script | 82.5 KB |
scripts/cx_verify.py
View on GitHub#!/usr/bin/env python3
"""cx_verify.py — the deterministic verifier gate for /citation-crosscheck.
Requirement 4 of `shared-references/fan-out-pattern.md` requires a fan-out skill to
end at "a single cross-model jury step -- OR a deterministic verifier gate -- that is
identical across all three tiers". `citation-crosscheck` has no cross-model jury (it is
an evidence instrument, not an acquittal step), so THIS SCRIPT is that gate.
It is a real external process, not an LLM self-report: it decides field equality by
string normalization plus a declared venue-alias table, and it FAILS CLOSED. Only two
outcomes are decided here --
MATCH every field equal after declared normalization (cosmetic differences only)
MINOR the sole difference is preprint-vs-published venue for the same work
-- and EVERYTHING else becomes
ESCALATE routed to the main agent's own primary-source fetch (Step 4)
`MISMATCH` is never emitted by this script. A discrepancy becomes a finding only when
the main agent reproduces it against the publisher of record. That keeps the accept
path mechanical and the reject path externally evidenced.
Usage:
python3 cx_verify.py --a cx_A_ourbib.json --b cx_B_web.json --c cx_C_fielddiff.json \
--expect <CITED_KEY_COUNT> --expect-keys /tmp/cx_dedup_keys.txt \
--expect-manifest /tmp/cx_manifest.tsv \
--json /tmp/cx_gate.json
python3 cx_verify.py --selftest # built-in cases (verify + build_entry), exit 2 on failure
Exit codes: 0 = ran (read the verdicts); 1 = bad/empty/incomplete input, or A/B/C
dedup_key sets disagree (a broken run, NOT a clean bill of health); 2 = selftest failure.
Trust model (this is the point of the rewrite):
* The gate RECOMPUTES the field diff from A's and B's raw records itself. C is a model
shard, so a faulty/adversarial C that omits a differing field or forges `blind:true`
could otherwise obtain a MATCH — the exact shard-trust flaw the gate exists to remove.
* B's `blind` attestation is read PER-SHARD from B's own artifact header, never from C.
* Routing flags are recomputed from A/B; C's per-entry flags are advisory additions.
Invalid C flags fail closed, and C never states the authoritative diff.
Artifact schemas the gate reads:
A: {"shard_id":"A", "entries":[{"dedup_key","key","bibtex"}, …]}
B: {"shard_id":"B", "blind":<bool>,
"entries":[{"dedup_key","title_queried","found","source_url","bibtex","sources_tried","note"}, …]}
C: {"shard_id":"C",
"entries":[{"dedup_key","key","found","sources_tried","url","flags":[…],
"fields":{"<field>":{"status":"same|differs|absent","ours","web"}}}, …]}
Keep these in sync with both SKILL.md files.
"""
import argparse
import hashlib
import json
import re
import sys
import unicodedata
# --------------------------------------------------------------- normalization
# LaTeX accent forms: \"o, \'e, \^{a}, \~n, \c{c}, \v{s} ...
# accent macros only: symbol forms (\`o \'e \^a \"u \~n \=o \.z) and the
# letter-named accents (\c{c} \v{s} \u{g} \H{o} \k{a} \r{u} \t{oo} \d{h} \b{t}). A
# blanket \\[a-z]+\{X\} would eat \sqrt{x}/\vec{x} and erase math from a title.
ACCENT = re.compile(r"\\[`'\"^~=.]\s*\{?([A-Za-z])\}?|\\[cvuHkrtdb]\{([A-Za-z])\}")
BRACES = re.compile(r'[{}]')
NONALNUM = re.compile(r'[^a-z0-9]+')
def _alnum(s):
"""Unicode-aware 'keep only alphanumerics'. Unlike NONALNUM ([^a-z0-9]) it preserves
non-Latin scripts, so a CJK/Cyrillic surname is not erased to '' (which silently
dropped the coauthor). Used for author tokens; ASCII-id fields still use NONALNUM.
"""
return ''.join(c for c in (s or '') if c.isalnum())
def _fold(s):
"""Lowercase, strip LaTeX accents/braces, drop combining marks."""
s = ACCENT.sub(lambda m: m.group(1) or m.group(2), s or '')
s = BRACES.sub('', s)
s = unicodedata.normalize('NFKD', s)
return ''.join(c for c in s if not unicodedata.combining(c)).lower()
_TEXT_SEP = re.compile(r'[\s\-‐-―_.,;:!?()\[\]{}\'"`/\\]+')
def norm_text(s):
"""Title/venue comparison form.
Collapses case, whitespace and *separator* punctuation, but KEEPS every
alphanumeric and non-ASCII character. A blanket `[^a-z0-9]+` strip would fold
'C++' and 'C#' together, and would erase CJK/Cyrillic titles entirely so two
unrelated non-ASCII titles compared equal -- both are wrong-accepts.
Digit-adjacent punctuation is KEPT SIGNIFICANT: a single separator char sitting
directly between two digits is a version number / range / decimal ('Model 4.1' vs
'Model 4-1', '10.5' vs '10-5'), not formatting -- collapsing it to a space folded
substantively different titles together. Every other separator run still collapses.
"""
s = _fold(s)
def _repl(m):
run = m.group(0)
if run.strip() == '':
return ' ' # pure whitespace is always formatting
if len(run) == 1:
before = s[m.start() - 1] if m.start() > 0 else ''
after = s[m.end()] if m.end() < len(s) else ''
if before.isdigit() and after.isdigit():
return run # e.g. '4.1' / '4-1' — keep it verbatim
return ' '
s = _TEXT_SEP.sub(_repl, s)
return re.sub(r'\s+', ' ', s).strip()
def norm_id(s):
"""Year / volume / pages.
Dash STYLE is cosmetic ('12-20' == '12--20' == '12–20'), but the range BOUNDARY is
not: collapsing every separator turns '1-234' and '12-34' both into '1234' and the
gate accepts a wrong page range. So normalize each dash run to a single '-' and keep
it, rather than deleting it.
"""
s = _fold(s)
s = re.sub(r'\s+', '', s) # only whitespace is pure formatting; '_' is significant
# Dash STYLE is cosmetic ('12-20' == '12--20' == '12–20'); collapse a dash RUN to one
# '-' and keep it (incl. a trailing one: '12-' is not the single page '12').
s = re.sub(r'[\-‐-―]+', '-', s)
# '.'/',' are NOT formatting inside a number: '20.24' != '2024', '1.2' != '12',
# 'e1.23' != 'e123'. Keep every other char verbatim.
return s
def norm_ident(s):
"""Strict identifier form for url / eprint / isbn / issn and any UNKNOWN field.
Truly fail-closed: collapse only leading/trailing whitespace; keep case, diacritics,
and every other character EXACTLY. It deliberately does NOT call `_fold` -- URL paths
are case-sensitive ('/Method.pdf' != '/method.pdf') and diacritics are meaningful
('Resume' != 'Résumé'), so folding them would let an ordinary wrong value pass. Inner
whitespace is preserved too (rare in an id; if present, a difference should surface).
This is the default for any field the gate does not specifically understand, so an
unmodelled field can never reach MATCH through a lossy normalizer -- only `title` gets
the lenient norm_text treatment.
"""
return (s or '').strip()
def norm_doi(s):
"""DOI / arXiv id -> 'namespace:bare-id', or '' if there is no id at all.
Inner punctuation is SIGNIFICANT: '10.5555/abc-def' and '10.5555/abcdef' are
different DOIs, so they must not fold together.
The namespace is KEPT. Stripping both the doi.org and the arxiv.org wrappers to a
bare id made 'https://arxiv.org/abs/2301.1' and 'https://doi.org/2301.1' compare
equal -- two different identifier spaces accepted as the same id.
A wrapper with nothing after it ('doi:', 'https://doi.org/') yields '', and an empty
identifier must never satisfy the identifier requirement -- callers treat '' as absent.
"""
s = _fold(s).strip()
ns = 'doi'
m = re.match(r'^(?:https?://arxiv\.org/abs/|arxiv:\s*)(.*)$', s)
if m:
ns, s = 'arxiv', m.group(1)
else:
m = re.match(r'^(?:https?://(?:dx\.)?doi\.org/|doi:\s*)(.*)$', s)
if m:
s = m.group(1)
elif re.match(r'^\d{4}\.\d{4,5}(v\d+)?$', s):
ns = 'arxiv' # bare arXiv id
s = re.sub(r'\s+', '', s)
return ('%s:%s' % (ns, s)) if s else ''
# Generational suffixes are not surnames; nobiliary particles belong TO the surname.
SUFFIXES = {'jr', 'sr', 'ii', 'iii', 'iv'} # NOT phd/md: they collide with real given names (e.g. 'Md Ali')
PARTICLES = {'van', 'von', 'der', 'den', 'de', 'del', 'della', 'di', 'da', 'dos', 'das',
'du', 'la', 'le', 'ter', 'ten', 'al', 'bin', 'ibn', 'st', 'mac', 'mc', 'o'}
# "and others" / "et al." are conventional truncation markers, not a person. Match
# 'others' only in its conventional trailing position, never as a bare surname -- a real
# author surnamed 'Others' must not be erased.
# 'et al.'/'and others' are truncation markers ONLY when terminal (optionally followed by
# punctuation) — 'and Others, Ann' is a real coauthor named Others, not a truncation, and
# must NOT be erased (that would let a dropped coauthor compare equal).
ETAL = re.compile(r'[,;]?\s*\b(?:et\s+al|and\s+others)\b\s*\.?\s*$', re.I)
def _split_bib_and(s):
"""Split an author field on a whitespace-delimited ` and ` at brace depth 0 ONLY.
This is the BibTeX rule, and getting it right closes two wrong-accepts:
* A `{...}`-grouped `and` is NOT a separator — a corporate author
`{Research and Development Team}` (or double-braced `{{...}}`) is ONE author,
not two people. Folding braces away *before* the split (the old bug) let a
corporate author normalize to the same list as two split names.
* `;` and `&` are NOT BibTeX separators — `Alice Smith & Bob Jones` is one
(malformed) author, not two clean ones. The old `re.split(..|;|&)` split them.
Splitting the RAW string (braces intact) and folding each chunk afterwards is what
makes both hold.
"""
parts, buf, depth, i, n = [], [], 0, 0, len(s)
while i < n:
c = s[i]
if c == '{':
depth += 1; buf.append(c); i += 1
elif c == '}':
depth = max(0, depth - 1); buf.append(c); i += 1
elif depth == 0 and c.isspace():
m = re.match(r'\s+and\s+', s[i:], re.I)
if m and ''.join(buf).strip():
parts.append(''.join(buf)); buf = []; i += m.end()
else:
buf.append(c); i += 1
else:
buf.append(c); i += 1
if ''.join(buf).strip():
parts.append(''.join(buf))
return parts
def norm_authors(s):
"""Order-PRESERVING surname list.
'Vaswani, Ashish and Shazeer, Noam' -> ['vaswani','shazeer']
'Ashish Vaswani and Noam Shazeer' -> ['vaswani','shazeer']
'van der Berg, Jan' == 'Jan van der Berg' -> ['vanderberg']
'Smith, John Jr.' compares equal to 'John Smith Jr.' via authors_equal().
Order is preserved deliberately: a re-ordered author list is one of the exact
defects this skill exists to catch, so it must NOT normalize to equal.
Truncation markers ('and others', 'et al.') are dropped, so a truncated `.bib`
list and a full web list compare on the names they share. That means truncation
alone does not escalate -- but it also cannot hide a *changed* name, because every
listed position is still compared in order.
"""
s = s or ''
# Strip a TERMINAL truncation marker ('et al.'/'and others') BEFORE splitting, so a
# truncated .bib list and a full web list compare on the names they share (a sentinel
# below still stops a truncated list matching a complete one). Split on a brace-depth-0
# ` and ` only — braces protect a corporate author's internal 'and', and ';'/'&' are
# not separators — then fold each chunk.
stripped = ETAL.sub(' ', s)
truncated = _fold(stripped) != _fold(s) # a terminal 'et al.'/'and others' was stripped
out = []
for raw_chunk in _split_bib_and(stripped):
chunk = _fold(raw_chunk).strip(' .,')
if not chunk:
continue
suffix = ''
if ',' in chunk: # "Last, First" -> surname is before the comma
surname_words = chunk.split(',')[0].split()
given = chunk.split(',', 1)[1]
# Only comma form identifies the surname unambiguously. A suffix token
# inside that surname (e.g. "Ii, Naosuke") is never removed.
suffix = ''.join(sorted(_alnum(w) for w in given.split()
if _alnum(w) in SUFFIXES))
given = ' '.join(w for w in given.split() if _alnum(w) not in SUFFIXES)
else: # "First M. Last" -> surname is the tail
words = [w for w in chunk.split() if w]
# Keep the literal tail: "Naosuke Ii" has surname Ii, not Naosuke.
# authors_equal folds a possible suffix only when the other record
# explicitly identifies the same suffix outside its comma-form surname.
# Walk left from the tail while the preceding word is a nobiliary particle,
# so "jan van der berg" yields "van der berg", not "berg".
i = len(words) - 1
while i > 0 and _alnum(words[i - 1]) in PARTICLES:
i -= 1
surname_words = words[i:]
given = ' '.join(words[:i]) if i > 0 else ''
surname = _alnum(''.join(surname_words))
# Surname alone accepts 'John Smith' vs 'Jane Smith' and 'Plato' vs 'John
# Plato', so carry the given name too and let authors_equal decide what is
# cosmetic (abbreviation and dropped middles) versus a real difference.
# Keep the full given-name token sequence. Comparison (authors_equal) decides
# what is cosmetic: it tolerates abbreviation and a DROPPED middle name, but a
# PRESENT-on-both-sides middle name must agree ('John P. Smith' vs 'John Q.
# Smith' is a different person), and a given name absent on one side while the
# other spells one out is a real difference, not a wildcard.
g = '.'.join(_alnum(w) for w in given.split() if _alnum(w))
# Keep the FULL given name, not just the initial. Comparison below downgrades to
# initial-only when EITHER side is abbreviated, so 'J. Smith' still matches 'John
# Smith' while 'John Smith' vs 'Jane Smith' now differs.
if surname:
out.append(surname + '|' + g + '|' + suffix)
if truncated:
# The list claimed MORE authors than it named. Carry that as a sentinel entry so a
# truncated list never compares equal to a complete list of the same visible names
# (which would hide a dropped/extra coauthor). Two truncated lists still agree.
out.append('|||truncated')
return out
def _looks_abbreviated(tok):
"""A one-character Latin token is a typical initial abbreviation ('J.' -> 'j').
A one-character NON-LATIN token is not: '伟' is a full CJK given name, not an
abbreviation of '伟明'. Comparing those by first character alone silently accepted
dropped/changed CJK given-name characters."""
return len(tok) == 1 and 'a' <= tok <= 'z'
def _given_eq(gx, gy):
"""Compare two dot-joined given-name token sequences.
Tolerated as cosmetic:
- Latin abbreviation: 'j' vs 'john' -> equal
- dropped middles: 'john' vs 'john.a' -> equal
NOT tolerated:
- different name: 'john' vs 'jane' -> differ
- both middles present and different: 'john.p' vs 'john.q' -> differ
- one side names nobody while the other does: '' vs 'jane' -> differ
- one-char NON-LATIN vs longer non-Latin: '伟' vs '伟明' -> differ (see
`_looks_abbreviated`).
"""
if not gx and not gy:
return True
if not gx or not gy:
# A given name on one side and none on the other is a real difference: 'Smith'
# is not evidence for 'Jane Smith'. Treating the empty side as a wildcard let a
# wrong author through.
return False
tx, ty = gx.split('.'), gy.split('.')
# First given name must agree; abbreviation is cosmetic ONLY for Latin initials.
fx, fy = tx[0], ty[0]
if _looks_abbreviated(fx) or _looks_abbreviated(fy):
if fx[0] != fy[0]:
return False
elif fx != fy:
return False
# Dropped-middle tolerance is a Latin-name convention (a .bib legitimately omits a
# middle name or initial: 'John Smith' vs 'John A. Smith'). For a CJK-style given name
# like '伟' vs '伟明' — or '伟' vs '伟 明' — a difference in the token count IS the
# name difference and must not be tolerated as a dropped middle. Detect that any token
# on either side contains a non-Latin letter and require equal token counts then.
def _any_non_latin(toks):
return any(any(c.isalpha() and not ('a' <= c <= 'z') for c in t) for t in toks)
if (_any_non_latin(tx) or _any_non_latin(ty)) and len(tx) != len(ty):
return False
# Middles: for the positions BOTH sides supply, they must agree (Latin abbreviation
# is cosmetic, non-Latin must match exactly).
for mx, my in zip(tx[1:], ty[1:]):
if _looks_abbreviated(mx) or _looks_abbreviated(my):
if mx[0] != my[0]:
return False
elif mx != my:
return False
return True
def authors_equal(a, b):
"""Compare two norm_authors() lists. Order matters; abbreviation is cosmetic."""
def explicit_suffix(surname, given, suffix, other_suffix):
# "John Smith Jr." may be suffix form only when the other record says
# "Smith, John Jr." (or "Smith, Jr., John"). Without that corroboration,
# treating its literal surname as a suffix could erase a real surname.
words = given.split('.') if given else []
if not suffix and surname in SUFFIXES and surname == other_suffix and len(words) >= 2:
i = len(words) - 1
while i > 0 and words[i - 1] in PARTICLES:
i -= 1
return ''.join(words[i:]), '.'.join(words[:i]), surname
return surname, given, suffix
if len(a) != len(b):
return False
for x, y in zip(a, b):
px, py = x.split('|'), y.split('|')
sx, gx, sufx = px[0], px[1] if len(px) > 1 else '', px[2] if len(px) > 2 else ''
sy, gy, sufy = py[0], py[1] if len(py) > 1 else '', py[2] if len(py) > 2 else ''
sx, gx, sufx = explicit_suffix(sx, gx, sufx, sufy)
sy, gy, sufy = explicit_suffix(sy, gy, sufy, sufx)
if sx != sy or not _given_eq(gx, gy):
return False
# A conflicting generational suffix is a different person (Jr vs Sr); a suffix
# present on one side only is a tolerable omission.
if sufx and sufy and sufx != sufy:
return False
return True
# DECLARED venue alias classes. Same class => same event, so differing wording is
# cosmetic. This table is deliberately explicit and small: a venue pair that is NOT
# in it escalates rather than being guessed at.
#
# Membership is WHOLE-STRING equality against a class, never substring containment.
# Substring matching silently collapses distinct conferences -- 'acl' is a substring of
# 'naacl' and of 'eacl', so ACL/NAACL/EACL would all become one class and a wrong-venue
# citation would be ACCEPTED. That is the one direction this gate must never fail in.
VENUE_ALIASES = [
{'neurips', 'nips', 'advances in neural information processing systems',
'conference on neural information processing systems'},
{'icml', 'international conference on machine learning'},
{'iclr', 'international conference on learning representations'},
{'cvpr', 'ieee conference on computer vision and pattern recognition',
'computer vision and pattern recognition',
'ieee cvf conference on computer vision and pattern recognition'},
{'iccv', 'international conference on computer vision'},
{'eccv', 'european conference on computer vision'},
{'acl', 'annual meeting of the association for computational linguistics'},
{'naacl', 'north american chapter of the association for computational linguistics',
'conference of the north american chapter of the association for computational linguistics'},
{'eacl', 'european chapter of the association for computational linguistics'},
{'emnlp', 'conference on empirical methods in natural language processing'},
{'aaai', 'aaai conference on artificial intelligence'},
{'ijcai', 'international joint conference on artificial intelligence'},
{'jmlr', 'journal of machine learning research'},
{'tmlr', 'transactions on machine learning research'},
]
PREPRINT = {'arxiv', 'arxiv preprint', 'corr', 'openreview', 'preprint', 'biorxiv'}
# A satellite/secondary track is NOT the main conference. Without this, 'ICML Workshop
# on Foo' would alias to ICML and a workshop-vs-main-track error would be accepted.
SUBVENUE = re.compile(r'\b(workshop|workshops|companion|demo|demos|tutorial|tutorials|'
r'short\s+papers|student|doctoral|findings|track|symposium|poster)\b')
# A difference in any of these is never cosmetic.
IDENTIFYING = ('authors', 'year', 'doi', 'arxiv', 'title') # differing here is never cosmetic
COMPARED = ('title', 'year', 'doi', 'arxiv', 'volume', 'pages')
DOI_LIKE = ('doi', 'arxiv') # inner punctuation significant
NUM_LIKE = ('year', 'volume', 'pages', 'issue', 'number', 'chapter') # dash-style cosmetic, rest significant
def venue_class(v):
n = norm_text(v)
if not n:
return None
# SUBVENUE first: 'arXiv Workshop on Foo' is a satellite event, not the arXiv
# preprint server, and must not take the PREPRINT shortcut into a MINOR verdict.
if SUBVENUE.search(n):
return 'SUB:' + n # satellite track: only ever equal to itself
# PREPRINT only for the exact preprint-server names, not any string beginning 'arxiv'
# ('arXiv Press' is a different venue). arXiv-with-an-id ('arxiv 2301.00001') is also
# a preprint -- but the EMBEDDED ID must be kept, or two different arXiv ids in the
# venue string ('arXiv:2401.00001' vs 'arXiv:2401.99999') both map to bare PREPRINT and
# compare equal (different works accepted as MATCH). norm_text preserves a dot BETWEEN
# digits (so '2401.00001' survives) but spaces out ':' and a leading dot, so accept
# either a dot or whitespace between the two digit groups and rejoin the id.
m = re.match(r'^arxiv[\s:.]*(\d{4})[\s.]*(\d{4,5})(?:[\s.]*(v\d+))?$', n)
if m:
return 'PREPRINT:%s.%s%s' % (m.group(1), m.group(2), m.group(3) or '')
if n in PREPRINT:
return 'PREPRINT' # a bare 'arXiv preprint' with no id: only equal to itself
for i, cls in enumerate(VENUE_ALIASES):
if n in cls: # WHOLE-STRING membership; see the note above
return 'V%d' % i
return 'RAW:' + n # unknown to the table -> only ever equal to itself
REQUIRED_KEYS = ('dedup_key', 'key', 'found', 'fields')
# --------------------------------------------------------------- BibTeX parsing
# The gate recomputes the field diff from A's and B's RAW records; it does not trust C's
# restatement (C is a model shard — a faulty/adversarial C that omits a differing field or
# forges `blind` could otherwise obtain a MATCH, which is the exact shard-trust flaw the
# gate exists to remove). C is consulted ONLY for routing flags.
# @type{key, field = {value} | "value" | bareword, ...}
_BIB_ENTRY = re.compile(r'@\s*([A-Za-z]+)\s*\{\s*([^,\s]*)\s*,(.*)\}\s*$', re.S)
def _split_bib_fields(body):
"""Yield (name, raw_value) from a bibtex entry body, respecting nested {} and quotes."""
i, n = 0, len(body)
while i < n:
# field name
m = re.compile(r'([A-Za-z][A-Za-z0-9_-]*)\s*=\s*').match(body, i)
if not m:
i += 1
continue
name = m.group(1).lower()
i = m.end()
if i >= n:
break
if body[i] == '{':
depth, j = 0, i
while j < n:
if body[j] == '{':
depth += 1
elif body[j] == '}':
depth -= 1
if depth == 0:
j += 1
break
j += 1
val = body[i + 1:j - 1]
i = j
elif body[i] == '"':
# Inside "..." bibtex still recognizes {...} groups. A `"` inside a brace
# group is text, not a value terminator. Track brace depth to close at the
# right `"` — otherwise `"M{\"u}ller..."` truncates at the inner quote.
j, depth = i + 1, 0
while j < n:
c = body[j]
if c == '{':
depth += 1
elif c == '}':
depth = max(0, depth - 1)
elif c == '"' and depth == 0:
break
j += 1
val = body[i + 1:j]
i = j + 1
else: # bareword (number / macro)
m2 = re.compile(r'[^,]*').match(body, i)
val = m2.group(0).strip()
i = m2.end()
# Handle bibtex string concatenation: value # value # value ...
while True:
k = i
while k < n and body[k] in ' \t\r\n':
k += 1
if k >= n or body[k] != '#':
break
k += 1
while k < n and body[k] in ' \t\r\n':
k += 1
if k >= n:
break
if body[k] == '{':
depth, j = 0, k
while j < n:
if body[j] == '{':
depth += 1
elif body[j] == '}':
depth -= 1
if depth == 0:
j += 1
break
j += 1
val += body[k + 1:j - 1]
i = j
elif body[k] == '"':
j, depth = k + 1, 0
while j < n:
c = body[j]
if c == '{':
depth += 1
elif c == '}':
depth = max(0, depth - 1)
elif c == '"' and depth == 0:
break
j += 1
val += body[k + 1:j]
i = j + 1
else:
# bareword: read to the next , or #
j = k
while j < n and body[j] not in ',#':
j += 1
val += body[k:j].strip()
i = j
yield name, val.strip()
# advance past the trailing comma
while i < n and body[i] in ' \t\r\n,':
i += 1
# Normalize the many bibtex venue/id field names to the gate's canonical field set.
_FIELD_ALIASES = {
'journal': 'venue', 'booktitle': 'venue', 'venue': 'venue',
'eprint': 'arxiv', 'archiveprefix': None, 'primaryclass': None,
'url': 'url', 'doi': 'doi', 'author': 'authors', 'authors': 'authors',
'title': 'title', 'year': 'year', 'date': 'year', 'volume': 'volume',
'number': 'issue', 'issue': 'issue', # both mean 'issue-of-journal'
'pages': 'pages', 'chapter': 'chapter',
'publisher': 'publisher', 'editor': 'editor', 'isbn': 'isbn', 'issn': 'issn',
'howpublished': 'howpublished', 'note': 'note', 'month': 'month', 'series': 'series',
}
def _put_field(out, conflicts, canon, val):
"""First value for a canonical field wins (matching the historical parser behaviour),
but a DIFFERING duplicate is recorded as a conflict rather than silently dropped: two
disagreeing `author=` (or any field) lines in one entry are ambiguous evidence and must
ESCALATE, not vanish. Recorded conflicts surface via build_entry as a routing flag."""
if not val:
return
if canon in out:
if out[canon] != val:
conflicts.add(canon)
else:
out[canon] = val
def parse_bibtex(bibtex):
"""Parse a single bibtex record -> {canonical_field: value}. Unknown fields are kept
under their own lowercased name so the gate still compares them (fail-closed).
Duplicate fields with differing values are recorded under the reserved `__conflicts__`
key (a list of conflicted canonical field names), which build_entry turns into an
ESCALATE-forcing flag.
Strict: if we cannot find a well-formed `@type{key, ...}` entry, return {}. An
old permissive fallback parsed the whole string, so a stray '}' or an @string{...}
macro definition produced phantom fields (e.g. '2020}' as year, 'tcs: TCS' as field).
An unparseable record is not evidence.
"""
if not bibtex:
return {}
m = _BIB_ENTRY.search(bibtex.strip())
if not m:
return {} # not a real bibtex record; caller sees empty -> ESCALATE
kind = m.group(1).lower()
if kind == 'string': # @string{FOO = "…"} is a MACRO defn, not an entry
return {}
body = m.group(3)
out, conflicts, archive_prefix, eprint_raw = {}, set(), None, None
for name, val in _split_bib_fields(body):
if name == 'archiveprefix':
if archive_prefix is not None and archive_prefix != val:
conflicts.add('arxiv')
archive_prefix = val
continue
if name == 'primaryclass':
continue
if name == 'eprint':
if eprint_raw is not None and eprint_raw != val:
conflicts.add('eprint')
eprint_raw = val
continue
canon = _FIELD_ALIASES.get(name, name) # unknown -> its own name (still compared)
if canon is None:
continue
_put_field(out, conflicts, canon, val)
# Route `eprint` by its archivePrefix. `arxiv` for arXiv; anything else (HAL, bioRxiv,
# OSF, …) is kept as `eprint:<prefix>:<id>` so two different eprint spaces do NOT compare
# equal. Without a prefix, only an obvious arXiv id form gets the arxiv namespace.
if eprint_raw:
pfx = (archive_prefix or '').strip().lower()
if pfx == 'arxiv' or (not pfx and re.match(r'(?i)^arxiv', eprint_raw)):
_put_field(out, conflicts, 'arxiv', eprint_raw if re.match(r'(?i)^arxiv', eprint_raw)
else 'arXiv:' + eprint_raw)
elif not pfx and re.match(r'^\d{4}\.\d{4,5}(v\d+)?$', eprint_raw):
_put_field(out, conflicts, 'arxiv', 'arXiv:' + eprint_raw)
else:
_put_field(out, conflicts, 'eprint', '%s:%s' % (pfx or 'unknown', eprint_raw))
if conflicts:
out['__conflicts__'] = sorted(conflicts)
return out
def build_entry(dedup_key, key, a_rec, b_rec, blind, found, flags):
"""Assemble the {fields:{status,ours,web}} entry verify() consumes, from A and B RAW
records (parsed here) -- NOT from C. `blind` comes from B; `flags`/`found` route."""
ours = parse_bibtex(a_rec)
web = parse_bibtex(b_rec)
# A duplicate field whose two values disagree is ambiguous evidence -> force ESCALATE
# (never silently keep the first). Carried as a flag, which verify() escalates on.
conflicts = sorted(set(ours.pop('__conflicts__', [])) | set(web.pop('__conflicts__', [])))
fields = {}
for f in sorted(set(ours) | set(web)):
ev = {}
if f in ours:
ev['ours'] = ours[f]
if f in web:
ev['web'] = web[f]
fields[f] = ev
all_flags = list(flags or [])
if conflicts:
all_flags.append('field_conflict:' + ','.join(conflicts))
return {'dedup_key': dedup_key, 'key': key, 'found': found, 'blind': blind,
'flags': all_flags, 'fields': fields}
def recompute_routing_flags(a_rec, b_entry):
"""Routing flags RE-DERIVED from A's and B's own raw artifacts, so the gate never
depends on C to raise them (C is a model shard; see assemble_flags):
* web_record_absent — B's own `found` is not true.
* degraded_search — B's own `sources_tried` marks a source down/rate-limited.
* url_only_source_unqueried — A carries a url/howpublished but no arXiv id / DOI, B did
not confirm, and B never queried a general-web/landing source (only indexers that
structurally cannot see that url). This forces the main-agent url fetch in Step 4.
"""
b_entry = b_entry if isinstance(b_entry, dict) else {}
flags = set()
found = b_entry.get('found') is True
sources = b_entry.get('sources_tried')
joined = (' '.join(str(x) for x in sources) if isinstance(sources, list)
else str(sources or '')).lower()
# Broad on purpose: a MISSED degradation token is the dangerous direction (a genuinely
# degraded search reads as a clean not-found / clean match), so over-flag rather than
# under-flag. Covers HTTP 4xx/5xx and the common bot-block / quota / transport failures.
if re.search(r'(4\d\d|5\d\d|\bdown\b|time.?d?.?out|\berror\b|rate.?limit|throttl|block|'
r'unavailable|unreachable|refused|reset|forbidden|captcha|quota|denied|'
r'exceeded|\bssl\b|handshake|\bfail)', joined):
flags.add('degraded_search')
if not found:
flags.add('web_record_absent')
ours = parse_bibtex(a_rec)
has_url = bool(ours.get('url') or ours.get('howpublished'))
has_id = bool(ours.get('arxiv') or ours.get('doi'))
queried_web = bool(re.search(r'(\bweb\b|\burl\b|landing|google|bing|duckduckgo|http)', joined))
if has_url and not has_id and not queried_web:
flags.add('url_only_source_unqueried')
return flags
def assemble_flags(c_entry, a_rec, b_entry):
"""Final routing flags = C's advisory flags (sanitized) ∪ flags recomputed from A/B.
C is NOT in the trust path: a C that OMITS a flag cannot wave an entry through, because
recompute_routing_flags re-derives from A/B what the routing conditions actually are; and
a C SCHEMA VIOLATION (`flags` not a list, or a non-string/empty item) fails CLOSED by
adding `c_schema_invalid` (→ ESCALATE) instead of being silently coerced to no-flags.
"""
raw = c_entry.get('flags', []) if isinstance(c_entry, dict) else None
flags, bad = set(), False
if isinstance(raw, list):
for x in raw:
if isinstance(x, str) and x.strip():
flags.add(x.strip())
else:
bad = True # non-string / empty flag item is a schema violation
else:
bad = True # `flags` present but not a list is a schema violation
if bad:
flags.add('c_schema_invalid')
flags |= recompute_routing_flags(a_rec, b_entry)
return sorted(flags)
def read_sides(entry):
"""Extract {field: value} for each provenance from C's DOCUMENTED schema.
Step 3 emits nested per-field evidence:
{"fields": {"<field>": {"status": ..., "ours": ..., "web": ...}}}
Read exactly that. (An earlier revision of this gate read top-level `ours`/`web`
dicts, which do not exist in a conforming artifact -- every entry escalated, so the
gate was inert. Keep the shapes in sync: schema change here means schema change in
both SKILL.md files.)
"""
ours, web = {}, {}
for fname, ev in (entry.get('fields') or {}).items():
if not isinstance(ev, dict):
continue
if ev.get('ours') not in (None, ''):
ours[fname] = ev.get('ours')
if ev.get('web') not in (None, ''):
web[fname] = ev.get('web')
return ours, web
def verify(entry):
"""-> (verdict, reason, per_field_status). verdict in MATCH|MINOR|ESCALATE."""
fields = {}
# Schema validation is part of the gate: a malformed entry is not evidence, so it
# escalates rather than being silently treated as "nothing differs".
if not isinstance(entry, dict):
return 'ESCALATE', 'entry is not an object', fields
missing = [k for k in REQUIRED_KEYS if k not in entry]
if missing:
return 'ESCALATE', 'entry missing required key(s): ' + ','.join(missing), fields
if not isinstance(entry.get('fields'), dict) or not entry['fields']:
return 'ESCALATE', 'entry has no per-field evidence', fields
flags = entry.get('flags') or []
if flags:
return 'ESCALATE', 'flags: ' + ','.join(sorted(flags)), fields
if entry.get('found') is not True:
return 'ESCALATE', 'web_record_absent (B did not confirm)', fields
ours, web = read_sides(entry)
if not ours or not web:
return 'ESCALATE', 'one provenance carries no fields at all', fields
differs, absent, cosmetic = [], [], []
# Compare EVERY field either side carries, not just a hardcoded list. A field the gate
# does not know about (issue, number, editor, publisher, chapter, ...) is compared as
# text and, if it differs, escalates — otherwise a wrong `issue`/`number` would pass
# silently just because it is not in COMPARED. `authors`/`venue` have bespoke handling
# below, so they are excluded here.
for f in sorted((set(ours) | set(web)) - {'authors', 'venue'}):
o, w = ours.get(f), web.get(f)
if not o or not w:
if o or w:
fields[f] = 'absent'
absent.append(f)
continue
n = (norm_doi if f in DOI_LIKE else norm_id if f in NUM_LIKE
else norm_text if f == 'title' else norm_ident)
if not n(o) or not n(w):
# A value that normalizes to nothing (a bare 'doi:' resolver, a
# punctuation-only title like '---') is not evidence of anything. Treat it as
# absent -- which escalates -- rather than letting '' == '' read as agreement.
fields[f] = 'absent'
absent.append(f)
continue
if n(o) == n(w):
fields[f] = 'same'
if o != w:
cosmetic.append(f)
else:
fields[f] = 'differs'
differs.append(f)
ao, aw = norm_authors(ours.get('authors')), norm_authors(web.get('authors'))
if not ao or not aw:
fields['authors'] = 'absent'
absent.append('authors')
elif authors_equal(ao, aw):
fields['authors'] = 'same'
else:
fields['authors'] = 'differs'
differs.append('authors')
co, cw = venue_class(ours.get('venue')), venue_class(web.get('venue'))
if co is None or cw is None:
fields['venue'] = 'absent'
absent.append('venue')
elif co == cw:
fields['venue'] = 'same'
if norm_text(ours.get('venue')) != norm_text(web.get('venue')):
cosmetic.append('venue')
elif (co.startswith('PREPRINT') or cw.startswith('PREPRINT')) and (
str(co).startswith('V') or str(cw).startswith('V')):
# Preprint (bare or id-bearing) on one side, a KNOWN published venue on the other
# -> the benign preprint-vs-published case. arXiv vs an unknown/RAW venue
# ('arXiv Press') is NOT this; it falls through and escalates. Two DIFFERENT
# preprint ids ('PREPRINT:2401.00001' vs 'PREPRINT:2401.99999') are caught by the
# `co == cw` test above failing, so they reach the `differs` branch -> ESCALATE.
fields['venue'] = 'preprint_vs_published'
elif co.startswith(('RAW:', 'SUB:')) or cw.startswith(('RAW:', 'SUB:')):
fields['venue'] = 'differs_unknown_alias'
return 'ESCALATE', 'venue pair not resolvable from the declared alias table', fields
else:
fields['venue'] = 'differs'
differs.append('venue')
# Fail-closed ladder: anything not provably cosmetic leaves as ESCALATE.
# A one-sided ABSENCE is not evidence of agreement — the two records simply were not
# compared on that field, so calling it MATCH would overstate what the gate checked.
# Only the fields both provenances carry can support an accept, and the identifying
# set must be fully covered on both sides.
if differs:
kind = 'identifying' if any(f in IDENTIFYING for f in differs) else 'secondary'
return 'ESCALATE', '%s field differs: %s' % (kind, ','.join(differs)), fields
if absent:
kind = 'identifying' if any(f in IDENTIFYING for f in absent) else 'secondary'
return 'ESCALATE', '%s field absent on one side: %s' % (kind, ','.join(absent)), fields
# Core identity must be covered on both sides. `doi` and `arxiv` are ALTERNATIVES
# (a venue paper has a DOI, a preprint an arXiv id, rarely both), so require at
# least one of them rather than every one.
missing_core = [f for f in ('title', 'authors', 'year', 'venue') if f not in fields]
if missing_core:
return 'ESCALATE', 'no evidence for core field(s): ' + ','.join(missing_core), fields
if not ({'doi', 'arxiv'} & set(fields)):
return 'ESCALATE', 'no identifier evidence (neither doi nor arxiv)', fields
if fields.get('venue') == 'preprint_vs_published':
# Reached only when nothing else differs and nothing is absent, so this really
# is the sole difference.
return 'MINOR', 'preprint vs published venue is the only difference', fields
if cosmetic:
return 'MATCH', 'cosmetic only: ' + ','.join(cosmetic), fields
return 'MATCH', 'all compared fields identical', fields
# ------------------------------------------------------ coverage manifest + I/O safety
def _safe(s):
"""Strip control chars (newline, ANSI ESC, etc.) from shard-derived text before it is
printed, so a crafted cite key / field value cannot inject terminal escapes or forge a
log line. Ordinary printable content (incl. non-ASCII) is preserved."""
return re.sub(r'[\x00-\x1f\x7f]', ' ', str(s))
def manifest_title_hash(title):
"""Stable title fingerprint for the Step-1 coverage manifest.
Computed by --build-manifest from the .bib at Step 1 AND from A's parsed title at verify
time — both via THIS function and THIS parser — so a drop-and-substitute that preserves
the ordinal *set* still changes the (cite_key, title_hash) MAPPING and fails the gate.
Case/whitespace-insensitive so cosmetic .bib reflow does not spuriously fail coverage.
"""
t = re.sub(r'\s+', ' ', (title or '').strip().lower())
return hashlib.sha1(t.encode('utf-8')).hexdigest()[:16]
def iter_bib_entries(text):
"""Yield (cite_key, raw_entry_text) for each brace-matched @type{key,...} in a .bib.
@string/@comment/@preamble macro blocks are skipped (not real entries)."""
i, n = 0, len(text)
while i < n:
at = text.find('@', i)
if at < 0:
break
m = re.match(r'@\s*([A-Za-z]+)\s*\{', text[at:])
if not m:
i = at + 1
continue
kind = m.group(1).lower()
depth, j = 0, at + m.end() - 1 # j = index of the opening '{'
while j < n:
if text[j] == '{':
depth += 1
elif text[j] == '}':
depth -= 1
if depth == 0:
j += 1
break
j += 1
raw, i = text[at:j], j
if kind in ('string', 'comment', 'preamble'):
continue
km = re.match(r'@\s*[A-Za-z]+\s*\{\s*([^,\s]+)\s*,', raw)
if km:
yield km.group(1), raw
def build_manifest(bib_path, keymap_path, out_path):
"""Emit the Step-1 coverage manifest `dedup_key<TAB>cite_key<TAB>title_hash` from the
.bib + the ordinal->cite_key keymap, using the gate's own parser so the gate can later
verify A against it as a MAPPING (not just a set)."""
try:
text = open(bib_path, encoding='utf-8', errors='replace').read()
except OSError as exc:
print('ERROR: cannot read --bib %s: %s' % (bib_path, exc), file=sys.stderr)
return 1
by_key = {}
for key, raw in iter_bib_entries(text):
by_key.setdefault(key, raw) # first definition wins (bibtex behaviour)
try:
km = open(keymap_path, encoding='utf-8').read()
except OSError as exc:
print('ERROR: cannot read --keymap %s: %s' % (keymap_path, exc), file=sys.stderr)
return 1
lines, missing = [], []
seen_ordinals, seen_cite_keys = set(), set()
for raw_line in km.splitlines():
if not raw_line.strip():
continue
parts = raw_line.split('\t')
if len(parts) < 2 or not parts[0].strip() or not parts[1].strip():
print("ERROR: --keymap line is not '<dedup_key>\\t<cite_key>': %r" % raw_line, file=sys.stderr)
return 1
dk, ck = parts[0].strip(), parts[1].strip()
if dk in seen_ordinals:
print('ERROR: duplicate dedup_key in --keymap: %s' % _safe(dk), file=sys.stderr)
return 1
if ck in seen_cite_keys:
print('ERROR: duplicate cite key in --keymap: %s' % _safe(ck), file=sys.stderr)
return 1
seen_ordinals.add(dk)
seen_cite_keys.add(ck)
raw = by_key.get(ck)
if raw is None:
missing.append(ck)
th = ''
else:
th = manifest_title_hash(parse_bibtex(raw).get('title') or '')
lines.append('%s\t%s\t%s' % (dk, ck, th))
if missing:
print('ERROR: --build-manifest: cite key(s) not found in %s: %s'
% (bib_path, ','.join(missing)), file=sys.stderr)
return 1
try:
open(out_path, 'w', encoding='utf-8').write('\n'.join(lines) + '\n')
except OSError as exc:
print('ERROR: cannot write --out-manifest %s: %s' % (out_path, exc), file=sys.stderr)
return 1
print('wrote %s (%d entries)' % (out_path, len(lines)))
return 0
def load_manifest(path):
"""Read a `dedup_key<TAB>cite_key<TAB>title_hash` manifest -> {dedup_key:(cite_key,hash)}."""
try:
raw = open(path, encoding='utf-8').read()
except OSError as exc:
print('ERROR: cannot read --expect-manifest %s: %s' % (path, exc), file=sys.stderr)
return None
want, seen_cite_keys = {}, set()
for line in raw.splitlines():
if not line.strip():
continue
parts = line.split('\t')
if len(parts) < 3:
print('ERROR: --expect-manifest line is not 3 tab-separated columns: %r' % line, file=sys.stderr)
return None
dk, ck, title_hash = (part.strip() for part in parts[:3])
if dk in want:
print('ERROR: duplicate dedup_key in --expect-manifest: %s' % _safe(dk), file=sys.stderr)
return None
if ck in seen_cite_keys:
print('ERROR: duplicate cite key in --expect-manifest: %s' % _safe(ck), file=sys.stderr)
return None
want[dk] = (ck, title_hash)
seen_cite_keys.add(ck)
return want
def manifest_mismatches(want, ai):
"""Verify A against the Step-1 manifest as a MAPPING: same dedup_key set, and for each
ordinal the SAME cite key and SAME title hash. Catches a drop-and-substitute that keeps
the count/ordinal set identical (which --expect / --expect-keys cannot)."""
errs = []
if set(want) != set(ai):
miss = sorted(set(want) - set(ai))
extra = sorted(set(ai) - set(want))
if miss:
errs.append('dedup_key(s) in manifest missing from A: ' + ','.join(miss))
if extra:
errs.append('dedup_key(s) in A not in manifest: ' + ','.join(extra))
for dk in sorted(set(want) & set(ai)):
exp_key, exp_hash = want[dk]
a_e = ai[dk]
a_key = str(a_e.get('key') or '').strip()
a_hash = manifest_title_hash(parse_bibtex(a_e.get('bibtex')).get('title') or '')
if a_key != exp_key:
errs.append('%s: cite key A=%r manifest=%r (substituted downstream?)' % (dk, _safe(a_key), _safe(exp_key)))
elif a_hash != exp_hash:
errs.append('%s (%s): title hash A=%s manifest=%s (entry changed downstream?)'
% (dk, _safe(a_key), a_hash, exp_hash))
return errs
# --------------------------------------------------------------------- selftest
def _entry(ours, web, found=True, flags=None, key='k', blind=True):
"""Build a SCHEMA-CONFORMING entry (nested fields.<f>.ours/web).
The fixtures deliberately use the same shape Step 3 documents. An earlier revision
of this selftest used flat top-level ours/web dicts, so it passed while the gate was
inert against real artifacts -- do not reintroduce that shape.
`blind` defaults True so these field-diff cases exercise the comparison logic; the
blindness-enforcement path has its own dedicated cases below.
"""
fields = {}
for f in set(ours) | set(web):
ev = {'status': 'same' if ours.get(f) == web.get(f) else 'differs'}
if f in ours:
ev['ours'] = ours[f]
if f in web:
ev['web'] = web[f]
if f not in ours or f not in web:
ev['status'] = 'absent'
fields[f] = ev
e = {'dedup_key': '01', 'key': key, 'found': found, 'fields': fields, 'blind': blind}
if flags:
e['flags'] = flags
return e
FULL = ('title', 'authors', 'year', 'venue', 'doi')
def _both(**kw):
"""ours == web on every identifying field unless overridden via web_*.
A non-core field named in kw (e.g. pages='1-234') is placed on BOTH sides, so a
fixture testing that field's comparison does not accidentally test one-sided absence.
"""
ours = {f: kw.get(f, 'X') for f in FULL}
for k, v in kw.items():
if not k.startswith('web_'):
ours[k] = v
web = dict(ours)
for k, v in kw.items():
if k.startswith('web_'):
web[k[4:]] = v
return ours, web
SELFTEST = [
("identical", _entry(*_both(title='Attention Is All You Need', authors='Vaswani, Ashish and Shazeer, Noam',
year='2017', venue='NeurIPS', doi='10.1/a')), 'MATCH'),
("brace/case cosmetic", _entry(*_both(title='{BERT}: Pre-training', web_title='BERT: Pre-Training',
authors='Devlin, Jacob', web_authors='Jacob Devlin',
year='2019', venue='NAACL', doi='10.1/b')), 'MATCH'),
("venue alias is cosmetic", _entry(*_both(venue='Advances in Neural Information Processing Systems',
web_venue='NeurIPS', authors='Smith, A', web_authors='A Smith',
year='2020', doi='10.1/c')), 'MATCH'),
("latex accent in surname", _entry(*_both(authors=r'Schl{\"o}gl, Anna', web_authors='Anna Schlogl',
year='2021', venue='ACL', doi='10.1/d')), 'MATCH'),
("nobiliary particle", _entry(*_both(authors='van der Berg, Jan', web_authors='Jan van der Berg',
year='2020', venue='ICML', doi='10.1/e')), 'MATCH'),
("generational suffix", _entry(*_both(authors='Smith, John Jr.', web_authors='John Smith Jr.',
year='2020', venue='ICML', doi='10.1/f')), 'MATCH'),
("truncated author list", _entry(*_both(authors='Vaswani, Ashish and others', web_authors='Ashish Vaswani et al.',
year='2017', venue='NeurIPS', doi='10.1/g')), 'MATCH'),
("NAACL long form aliases", _entry(*_both(venue='NAACL', authors='A, B', web_authors='B A', year='2021', doi='10.1/h',
web_venue='North American Chapter of the Association for Computational Linguistics')), 'MATCH'),
("preprint vs published", _entry(*_both(venue='arXiv preprint', web_venue='ICML', authors='A, B',
web_authors='B A', year='2020', doi='10.1/i')), 'MINOR'),
# --- must ESCALATE (fail-closed) ---
("dropped author", _entry(*_both(authors='Smith, A', web_authors='A Smith and B Jones')), 'ESCALATE'),
("reordered authors", _entry(*_both(authors='Zhang, X and Li, Y', web_authors='Y Li and X Zhang')), 'ESCALATE'),
("changed name in truncated list", _entry(*_both(authors='Smith, J and others', web_authors='J Jones et al.')), 'ESCALATE'),
("wrong year", _entry(*_both(year='2019', web_year='2020')), 'ESCALATE'),
("wrong doi", _entry(*_both(doi='10.5555/abc-def', web_doi='10.5555/abcdef')), 'ESCALATE'),
("title word changed", _entry(*_both(title='Deep Residual Learning', web_title='Deep Residual Networks')), 'ESCALATE'),
("ACL vs NAACL is not an alias", _entry(*_both(venue='ACL', web_venue='NAACL')), 'ESCALATE'),
("ACL vs EACL is not an alias", _entry(*_both(venue='ACL', web_venue='EACL')), 'ESCALATE'),
("workshop is not the main track", _entry(*_both(venue='ICML', web_venue='ICML Workshop on Foo')), 'ESCALATE'),
("NeurIPS track is not main", _entry(*_both(venue='NeurIPS', web_venue='NeurIPS Datasets and Benchmarks Track')), 'ESCALATE'),
("findings is not main", _entry(*_both(venue='EMNLP', web_venue='Findings of EMNLP')), 'ESCALATE'),
("different event", _entry(*_both(venue='ICML', web_venue='ICLR')), 'ESCALATE'),
("Cpp vs Csharp title", _entry(*_both(title='C++ Patterns', web_title='C# Patterns')), 'ESCALATE'),
("distinct CJK titles", _entry(*_both(title='深度学习', web_title='机器学习')), 'ESCALATE'),
("identifying absent one side", _entry({'title': 'T', 'authors': 'A, B', 'year': '2020', 'venue': 'ICML', 'doi': '10.1/x'},
{'title': 'T', 'year': '2020', 'venue': 'ICML', 'doi': '10.1/x'}), 'ESCALATE'),
("secondary absent one side", _entry({'title': 'T', 'authors': 'A, B', 'year': '2020', 'venue': 'ICML', 'doi': '10.1/x', 'pages': '1-9'},
{'title': 'T', 'authors': 'B A', 'year': '2020', 'venue': 'ICML', 'doi': '10.1/x'}), 'ESCALATE'),
("no evidence for an identifying field", _entry({'title': 'T', 'authors': 'A, B', 'year': '2020'},
{'title': 'T', 'authors': 'B A', 'year': '2020'}), 'ESCALATE'),
("flag present", _entry(*_both(), flags=['url_only_source_unqueried']), 'ESCALATE'),
("not found", _entry(*_both(), found=False), 'ESCALATE'),
("no per-field evidence", {'dedup_key': '01', 'key': 'k', 'found': True, 'fields': {}}, 'ESCALATE'),
("missing required key", {'dedup_key': '01', 'found': True, 'fields': {'title': {'status': 'same', 'ours': 'T', 'web': 'T'}}}, 'ESCALATE'),
# --- round-4 wrong-accept regressions. Each of these once returned MATCH/MINOR.
("page range boundary moved", _entry(*_both(pages='1-234', web_pages='12-34')), 'ESCALATE'),
("dash style is still cosmetic", _entry(*_both(pages='12-20', web_pages='12--20')), 'MATCH'),
("arxiv wrapper vs doi wrapper", _entry(*_both(doi='https://arxiv.org/abs/2301.00001',
web_doi='https://doi.org/2301.00001')), 'ESCALATE'),
("doi resolver prefix is cosmetic", _entry(*_both(doi='https://doi.org/10.1/abc', web_doi='10.1/abc')), 'MATCH'),
("empty identifier is not an identifier", _entry(*_both(doi='doi:', web_doi='https://doi.org/')), 'ESCALATE'),
("same-initial given names differ", _entry(*_both(authors='Smith, John', web_authors='Jane Smith')), 'ESCALATE'),
("abbreviated given name still matches", _entry(*_both(authors='Smith, J.', web_authors='John Smith')), 'MATCH'),
("surname 'Others' is not a truncation marker", _entry(*_both(authors='Others, Ann', web_authors='Ann Others')), 'MATCH'),
# --- blindness enforcement: an accept that is not blind-attested must NOT certify.
("clean match but not blind-attested downgrades",
_entry(*_both(title='T', authors='A, B', year='2020', venue='ICML', doi='10.1/x'), blind=False), 'ESCALATE'),
("blind flag absent downgrades",
{'dedup_key': '01', 'key': 'k', 'found': True,
'fields': {'title': {'status': 'same', 'ours': 'T', 'web': 'T'},
'authors': {'status': 'same', 'ours': 'A, B', 'web': 'B A'},
'year': {'status': 'same', 'ours': '2020', 'web': '2020'},
'venue': {'status': 'same', 'ours': 'ICML', 'web': 'ICML'},
'doi': {'status': 'same', 'ours': '10.1/x', 'web': '10.1/x'}}}, 'ESCALATE'),
("a real discrepancy still ESCALATEs even when not blind",
_entry(*_both(authors='Smith, A', web_authors='A Smith and B Jones'), blind=False), 'ESCALATE'),
("middle initial is cosmetic", _entry(*_both(authors='Smith, John A.', web_authors='John Smith')), 'MATCH'),
("initials vs full given name", _entry(*_both(authors='Smith, J A', web_authors='John Andrew Smith')), 'MATCH'),
("same-initial permutation still differs",
_entry(*_both(authors='Smith, J and Doe, J', web_authors='J Doe and J Smith')), 'ESCALATE'),
# --- round-5 wrong-accept regressions.
# --- round-6 wrong-accept regressions (all were once MATCH/MINOR).
# --- round-7 wrong-accepts (found by cross-model re-check on blind-attested fixtures).
# --- round-8 STRUCTURAL wrong-accepts (ordinary input, cross-model round 7).
("different arXiv ids embedded in venue", _entry(*_both(venue='arXiv:2401.00001', web_venue='arXiv:2401.99999')), 'ESCALATE'),
("same arXiv id in venue, cosmetic form", _entry(*_both(venue='arXiv:2401.00001', web_venue='arxiv 2401.00001')), 'MATCH'),
("preprint-with-id vs known published", _entry(*_both(venue='arXiv:2401.00001', web_venue='ICML')), 'MINOR'),
("url path case is significant", _entry(*_both(url='http://x/Method.pdf', web_url='http://x/method.pdf')), 'ESCALATE'),
("diacritics in an id are significant", _entry(*_both(eprint='Resume', web_eprint='Résumé')), 'ESCALATE'),
("wrong url punctuation", _entry(*_both(url='https://arxiv.org/abs-2401/00001',
web_url='https://arxiv.org/abs/2401.00001')), 'ESCALATE'),
("wrong eprint punctuation", _entry(*_both(eprint='2401-00001', web_eprint='2401.00001')), 'ESCALATE'),
("false truncation hides a coauthor",
_entry(*_both(authors='Rivera, Elena and Patel, Omar and others', web_authors='Elena Rivera and Omar Patel')), 'ESCALATE'),
("both truncated still match",
_entry(*_both(authors='Vaswani, A and others', web_authors='A Vaswani et al.')), 'MATCH'),
("conflicting generational suffix Jr vs Sr", _entry(*_both(authors='Smith, John Jr.', web_authors='John Smith Sr.')), 'ESCALATE'),
("dropped coauthor named Others", _entry(*_both(authors='Smith, A and Others, Ann', web_authors='A Smith')), 'ESCALATE'),
("underscore in year is significant", _entry(*_both(year='20_24', web_year='2024')), 'ESCALATE'),
("underscore in pages is significant", _entry(*_both(pages='e1_23', web_pages='e123')), 'ESCALATE'),
("unknown field (issue) differing escalates", _entry(*_both(issue='2', web_issue='3')), 'ESCALATE'),
("arXiv vs arXiv Press is not preprint-vs-published", _entry(*_both(venue='arXiv', web_venue='arXiv Press')), 'ESCALATE'),
("differing spelled middle name", _entry(*_both(authors='Smith, John Peter', web_authors='John Paul Smith')), 'ESCALATE'),
("abbreviated middle still cosmetic", _entry(*_both(authors='Smith, John P.', web_authors='John Peter Smith')), 'MATCH'),
("year dot is significant", _entry(*_both(year='20.24', web_year='2024')), 'ESCALATE'),
("volume dot is significant", _entry(*_both(volume='1.2', web_volume='12')), 'ESCALATE'),
("arXiv Press is not the preprint server", _entry(*_both(venue='arXiv Press', web_venue='ICML')), 'ESCALATE'),
("open-ended page range", _entry(*_both(pages='12-', web_pages='12')), 'ESCALATE'),
("bare surname is not a wildcard", _entry(*_both(authors='Smith', web_authors='Jane Smith')), 'ESCALATE'),
("differing middle initial", _entry(*_both(authors='Smith, John P.', web_authors='John Q. Smith')), 'ESCALATE'),
("same-first-initial reorder", _entry(*_both(authors='Smith, Alex A and Smith, Alex B',
web_authors='Alex B Smith and Alex A Smith')), 'ESCALATE'),
("punctuation-only title is not evidence", _entry(*_both(title='---', web_title='...')), 'ESCALATE'),
("arXiv workshop is not the preprint server",
_entry(*_both(venue='arXiv Workshop on Foo', web_venue='ICML')), 'ESCALATE'),
("dropped middle is still cosmetic", _entry(*_both(authors='Smith, John A.', web_authors='John Smith')), 'MATCH'),
("lying status is recomputed, not trusted",
{'dedup_key': '01', 'key': 'liar', 'found': True,
'fields': {'title': {'status': 'same', 'ours': 'Real Title', 'web': 'Different Title'},
'authors': {'status': 'same', 'ours': 'A, B', 'web': 'B A'},
'year': {'status': 'same', 'ours': '2020', 'web': '2020'},
'venue': {'status': 'same', 'ours': 'ICML', 'web': 'ICML'},
'doi': {'status': 'same', 'ours': '10.1/x', 'web': '10.1/x'}}}, 'ESCALATE'),
# --- round-9 wrong-accept regressions (each once returned MATCH/MINOR). ---
("version-number title punctuation is significant",
_entry(*_both(title='Model 4.1', web_title='Model 4-1')), 'ESCALATE'),
("version-number title case-fold still matches",
_entry(*_both(title='Model 4.1', web_title='MODEL 4.1')), 'MATCH'),
("decimal vs integer title token differs",
_entry(*_both(title='Scaling to 10.5 B', web_title='Scaling to 105 B')), 'ESCALATE'),
("corporate braced author is one unit, not two people",
_entry(*_both(authors='{{Research and Development Team}}', web_authors='Research and Development Team')), 'ESCALATE'),
("ampersand is not an author separator",
_entry(*_both(authors='Alice Smith and Bob Jones', web_authors='Alice Smith & Bob Jones')), 'ESCALATE'),
("semicolon is not an author separator",
_entry(*_both(authors='Alice Smith and Bob Jones', web_authors='Alice Smith; Bob Jones')), 'ESCALATE'),
# --- round-10: suffix-token surname must not delete a coauthor (adversarial false-accept). ---
("suffix-token surname coauthor is not dropped",
_entry(*_both(authors='Tanaka, Hiro and Ii, Naosuke', web_authors='Hiro Tanaka')), 'ESCALATE'),
("roman-numeral surname coauthor is not dropped",
_entry(*_both(authors='Smith, Alice and III, Robert', web_authors='Alice Smith')), 'ESCALATE'),
("suffix-vs-published does not hide a dropped coauthor",
_entry(*_both(authors='Tanaka, H and Ii, N', web_authors='H Tanaka', venue='arXiv', web_venue='ICML')), 'ESCALATE'),
("identical suffix-like surname still matches",
_entry(*_both(authors='Ii, Naosuke', web_authors='Ii, Naosuke')), 'MATCH'),
("no-comma surname Ii is not erased",
_entry(*_both(authors='Alice Smith and Naosuke Ii', web_authors='Alice Smith and Naosuke')), 'ESCALATE'),
("surname Ii agrees across name formats",
_entry(*_both(authors='Ii, Naosuke', web_authors='Naosuke Ii')), 'MATCH'),
("middle name does not make surname Ii a suffix",
_entry(*_both(authors='Naosuke Middle Ii', web_authors='Naosuke Middle')), 'ESCALATE'),
("ambiguous no-comma suffix omission escalates",
_entry(*_both(authors='John Smith Jr.', web_authors='John Smith')), 'ESCALATE'),
("explicit comma-form suffix omission is cosmetic",
_entry(*_both(authors='Smith, John Jr.', web_authors='John Smith')), 'MATCH'),
("explicit Roman suffix agrees across name formats",
_entry(*_both(authors='Smith, III, John', web_authors='John Smith III')), 'MATCH'),
]
def classify(entry):
"""verify() + blindness enforcement — the SINGLE decision path used by both the CLI
and the selftest, so the selftest covers exactly what runs. A MATCH/MINOR that is not
blind-attested downgrades to ESCALATE: flag, never certify.
"""
verdict, reason, fields = verify(entry)
if verdict in ('MATCH', 'MINOR') and (not isinstance(entry, dict) or entry.get('blind') is not True):
reason = 'not blind-attested (B may have seen the .bib) — would be %s: %s' % (verdict, reason)
verdict = 'ESCALATE'
return verdict, reason, fields
# A+B recompute path: (name, a_bibtex, b_bibtex, b_blind, found, flags, want). These build
# the entry the SAME way main() does -- from A and B RAW records via build_entry -- so the
# parser and the "C is not trusted" property are covered by --selftest, not just by hand.
AB_SELFTEST = [
("recompute clean match",
"@article{k, title={Deep Nets}, author={Smith, Alice and Jones, Bob}, year={2020}, journal={ICML}, doi={10.1/x}}",
"@misc{w, title={Deep Nets}, author={Alice Smith and Bob Jones}, year={2020}, journal={ICML}, doi={10.1/x}}",
True, True, [], 'MATCH'),
("recompute catches a diff C could have hidden (pages)",
"@article{k, title={T}, author={Smith, A}, year={2020}, journal={ICML}, pages={1--9}, doi={10.1/x}}",
"@misc{w, title={T}, author={A Smith}, year={2020}, journal={ICML}, pages={10--19}, doi={10.1/x}}",
True, True, [], 'ESCALATE'),
("recompute catches a dropped coauthor",
"@article{k, title={T}, author={Smith, A and Jones, B}, year={2020}, journal={ICML}, doi={10.1/x}}",
"@misc{w, title={T}, author={A Smith}, year={2020}, journal={ICML}, doi={10.1/x}}",
True, True, [], 'ESCALATE'),
("recompute: eprint bibtex field -> arxiv id, wrong id escalates",
"@article{k, title={T}, author={Smith, A}, year={2020}, journal={ICML}, eprint={2401.00001}, archivePrefix={arXiv}}",
"@misc{w, title={T}, author={A Smith}, year={2020}, journal={ICML}, eprint={2401.99999}, archivePrefix={arXiv}}",
True, True, [], 'ESCALATE'),
("recompute: b_blind false never certifies",
"@article{k, title={T}, author={Smith, A}, year={2020}, journal={ICML}, doi={10.1/x}}",
"@misc{w, title={T}, author={A Smith}, year={2020}, journal={ICML}, doi={10.1/x}}",
False, True, [], 'ESCALATE'),
("recompute: a flag escalates",
"@article{k, title={T}, author={Smith, A}, year={2020}, journal={ICML}, doi={10.1/x}}",
"@misc{w, title={T}, author={A Smith}, year={2020}, journal={ICML}, doi={10.1/x}}",
True, True, ['url_only_source_unqueried'], 'ESCALATE'),
# --- round-12 parser bug regressions
("quoted-form braced accent parses full author list",
r"""@article{k, author="M{\"u}ller, Anna and Smith, Bob", title={T}, year=2020, journal={ICML}, doi={10.1/x}}""",
r"""@misc{w, author={M{\"u}ller, Anna}, title={T}, year=2020, journal={ICML}, doi={10.1/x}}""",
True, True, [], 'ESCALATE'), # A has 2 authors, B has 1 -> dropped-coauthor caught
("HAL eprint != arXiv eprint",
r"@article{k, title={T}, author={Smith, A}, year=2020, journal={ICML}, doi={10.1/x}, eprint={hal-12345}, archivePrefix={HAL}}",
r"@misc{w, title={T}, author={A Smith}, year=2020, journal={ICML}, doi={10.1/x}, eprint={2401.00001}, archivePrefix={arXiv}}",
True, True, [], 'ESCALATE'),
("string concatenation preserved",
r"@article{k, title={Robust} # { and Reliable}, year=2020, author={Smith, A}, journal={ICML}, doi={10.1/x}}",
r"@misc{w, title={Wrong Method}, year=2020, author={A Smith}, journal={ICML}, doi={10.1/x}}",
True, True, [], 'ESCALATE'), # A concat = 'Robust and Reliable', differs from 'Wrong Method'
("CJK given-name integrity: 伟 vs 伟明 differ",
"@article{k, author={李, 伟}, title={T}, year=2020, journal={ICML}, doi={10.1/x}}",
"@misc{w, author={伟明 李}, title={T}, year=2020, journal={ICML}, doi={10.1/x}}",
True, True, [], 'ESCALATE'),
("CJK spaced-middle substitution: 伟 vs 伟 明",
"@article{k, author={李, 伟}, title={T}, year=2020, journal={ICML}, doi={10.1/x}}",
"@misc{w, author={伟 明 李}, title={T}, year=2020, journal={ICML}, doi={10.1/x}}",
True, True, [], 'ESCALATE'),
("CJK given-name identity: 伟 vs 伟 match",
"@article{k, author={李, 伟}, title={T}, year=2020, journal={ICML}, doi={10.1/x}}",
"@misc{w, author={伟 李}, title={T}, year=2020, journal={ICML}, doi={10.1/x}}",
True, True, [], 'MATCH'),
("month contradiction is not silently discarded",
r"@article{k, title={T}, author={Smith, A}, year=2020, month={May}, journal={ICML}, doi={10.1/x}}",
r"@misc{w, title={T}, author={A Smith}, year=2020, month={November}, journal={ICML}, doi={10.1/x}}",
True, True, [], 'ESCALATE'),
("recompute: found false escalates",
"@article{k, title={T}, author={Smith, A}, year={2020}, journal={ICML}, doi={10.1/x}}",
"", True, False, [], 'ESCALATE'),
# --- round-9 parser regressions ---
("recompute: duplicate conflicting author field escalates (not silently first-wins)",
"@article{k, title={T}, author={Smith, Alice}, author={Jones, Bob}, year={2020}, journal={ICML}, doi={10.1/x}}",
"@misc{w, title={T}, author={Alice Smith}, year={2020}, journal={ICML}, doi={10.1/x}}",
True, True, [], 'ESCALATE'),
("recompute: corporate braced author is not two people",
"@article{k, title={T}, author={{OpenAI and Friends}}, year={2020}, journal={ICML}, doi={10.1/x}}",
"@misc{w, title={T}, author={OpenAI and Friends}, year={2020}, journal={ICML}, doi={10.1/x}}",
True, True, [], 'ESCALATE'),
("recompute: identical duplicate field is not a conflict",
"@article{k, title={T}, author={Smith, A}, author={Smith, A}, year={2020}, journal={ICML}, doi={10.1/x}}",
"@misc{w, title={T}, author={A Smith}, year={2020}, journal={ICML}, doi={10.1/x}}",
True, True, [], 'MATCH'),
]
# Routing-flag recompute path (C is advisory, A/B are authoritative):
# (name, c_entry, a_bibtex, b_entry, want_flags). Exercises assemble_flags directly so the
# "C not trusted / fail-closed on C schema" property is covered by --selftest, not just prose.
_DOI_REC = "@article{k, title={T}, author={Smith, A}, year={2020}, journal={ICML}, doi={10.1/x}}"
_MISC_URL_REC = "@misc{k, title={T}, author={Smith, A}, year={2020}, url={http://example.org/tr}}"
FLAG_SELFTEST = [
("flags clean: no routing flags", {}, _DOI_REC,
{'found': True, 'sources_tried': ['arxiv', 'crossref']}, []),
("recompute degraded_search from B even if C omits it", {}, _DOI_REC,
{'found': True, 'sources_tried': ['arxiv', 'semantic-scholar(429)']}, ['degraded_search']),
("recompute web_record_absent from B.found even if C omits it", {}, _DOI_REC,
{'found': False, 'sources_tried': ['arxiv', 'crossref']}, ['web_record_absent']),
("recompute url_only for a url-bearing misc B could not index", {}, _MISC_URL_REC,
{'found': False, 'sources_tried': ['arxiv', 'dblp']}, ['url_only_source_unqueried', 'web_record_absent']),
("C flags as a string fails closed (schema violation)", {'flags': 'url_only_source_unqueried'}, _DOI_REC,
{'found': True, 'sources_tried': ['arxiv']}, ['c_schema_invalid']),
("C flags null fails closed (schema violation)", {'flags': None}, _DOI_REC,
{'found': True, 'sources_tried': ['arxiv']}, ['c_schema_invalid']),
("C flags empty string fails closed (schema violation)", {'flags': ''}, _DOI_REC,
{'found': True, 'sources_tried': ['arxiv']}, ['c_schema_invalid']),
("C flags with a non-string item fails closed but keeps valid flags", {'flags': ['degraded_search', 123]}, _DOI_REC,
{'found': True, 'sources_tried': ['arxiv']}, ['c_schema_invalid', 'degraded_search']),
("valid C advisory flag is honored", {'flags': ['url_only_source_unqueried']}, _DOI_REC,
{'found': True, 'sources_tried': ['arxiv']}, ['url_only_source_unqueried']),
# --- round-10: broadened degradation tokens C might omit (403/forbidden/captcha/quota/reset). ---
("recompute degraded from a 403/forbidden token C omitted", {}, _DOI_REC,
{'found': True, 'sources_tried': ['arxiv', 'crossref(403 forbidden)']}, ['degraded_search']),
("recompute degraded from captcha/quota/reset tokens C omitted", {}, _DOI_REC,
{'found': True, 'sources_tried': ['s2(quota exceeded)', 'web(captcha)', 'dblp(connection reset)']}, ['degraded_search']),
]
def _amanifest(key, title):
"""A-index entry fixture (key + bibtex) for the manifest selftest."""
return {'key': key, 'bibtex': '@article{%s, title={%s}, author={Smith, A}, year={2020}}' % (key, title)}
# Coverage-manifest path: (name, want_manifest, a_index, expect_clean). Verifies the gate
# catches a drop-and-substitute the ordinal-set check cannot (blocker #4).
MANIFEST_SELFTEST = [
("manifest clean mapping",
{'01': ('foo', manifest_title_hash('Deep Nets')), '02': ('bar', manifest_title_hash('Wide Nets'))},
{'01': _amanifest('foo', 'Deep Nets'), '02': _amanifest('bar', 'Wide Nets')}, True),
("manifest catches a substituted cite key (same ordinal set)",
{'01': ('foo', manifest_title_hash('Deep Nets')), '02': ('bar', manifest_title_hash('Wide Nets'))},
{'01': _amanifest('foo', 'Deep Nets'), '02': _amanifest('baz', 'Wide Nets')}, False),
("manifest catches a changed title under the same key",
{'01': ('foo', manifest_title_hash('Deep Nets'))},
{'01': _amanifest('foo', 'Deep Networks')}, False),
("manifest catches a dropped ordinal",
{'01': ('foo', manifest_title_hash('Deep Nets')), '02': ('bar', manifest_title_hash('Wide Nets'))},
{'01': _amanifest('foo', 'Deep Nets')}, False),
("manifest title hash tolerates whitespace/case",
{'01': ('foo', manifest_title_hash('Deep NETS'))},
{'01': _amanifest('foo', 'Deep Nets')}, True),
]
def selftest():
bad = 0
for name, entry, want in SELFTEST:
got, reason, _ = classify(entry)
ok = got == want
bad += 0 if ok else 1
print('%s %-38s -> %-9s (%s)' % ('PASS' if ok else 'FAIL', name, got, reason))
for name, a_rec, b_rec, blind, found, flags, want in AB_SELFTEST:
entry = build_entry('01', 'k', a_rec, b_rec, blind, found, flags)
got, reason, _ = classify(entry)
ok = got == want
bad += 0 if ok else 1
print('%s %-38s -> %-9s (%s)' % ('PASS' if ok else 'FAIL', name, got, reason))
for name, c_e, a_rec, b_e, want in FLAG_SELFTEST:
got = assemble_flags(c_e, a_rec, b_e)
ok = got == sorted(want)
bad += 0 if ok else 1
print('%s %-38s -> %-9s (%s)' % ('PASS' if ok else 'FAIL', name, 'FLAGS', ','.join(got) or '(none)'))
for name, want, ai, expect_clean in MANIFEST_SELFTEST:
errs = manifest_mismatches(want, ai)
ok = (errs == []) == expect_clean
bad += 0 if ok else 1
print('%s %-38s -> %-9s (%s)' % ('PASS' if ok else 'FAIL', name, 'MANIFEST',
'clean' if not errs else '; '.join(errs)))
total = len(SELFTEST) + len(AB_SELFTEST) + len(FLAG_SELFTEST) + len(MANIFEST_SELFTEST)
print('\n%d/%d passed' % (total - bad, total))
return 0 if bad == 0 else 2
def _load(path, label):
try:
with open(path) as fh:
data = json.load(fh)
except (OSError, ValueError) as exc:
print('ERROR: cannot read %s (%s): %s' % (path, label, exc), file=sys.stderr)
return None
ents = data.get('entries') if isinstance(data, dict) else None
if not isinstance(ents, list):
print('ERROR: %s (%s) has no "entries" list' % (path, label), file=sys.stderr)
return None
return data
def main():
ap = argparse.ArgumentParser(description='Deterministic verifier gate for /citation-crosscheck.')
ap.add_argument('--a', dest='a', help="path to cx_A_ourbib.json (A: local .bib records)")
ap.add_argument('--b', dest='b', help="path to cx_B_web.json (B: web records, blind)")
ap.add_argument('--c', dest='c', help="path to cx_C_fielddiff.json (C: routing flags only)")
ap.add_argument('--json', dest='out', help='write machine-readable verdicts here')
ap.add_argument('--expect', type=int, default=None,
help="CITED_KEY_COUNT from Step 1; errors if the entry count differs")
ap.add_argument('--expect-keys', default=None,
help="file of whitespace-separated dedup_keys from Step 1; errors if the "
"set differs (catches a substitution that --expect's count cannot)")
ap.add_argument('--expect-manifest', default=None,
help="Step-1 coverage manifest (dedup_key<TAB>cite_key<TAB>title_hash). The "
"gate verifies the ordinal->cite_key/title MAPPING against A, catching a "
"drop-and-substitute that --expect-keys' set check cannot. PREFERRED "
"coverage arg; build it with --build-manifest.")
ap.add_argument('--build-manifest', action='store_true',
help='build the Step-1 coverage manifest from --bib + --keymap into --out-manifest')
ap.add_argument('--bib', default=None, help='path to the .bib (with --build-manifest)')
ap.add_argument('--keymap', default=None,
help='TSV of dedup_key<TAB>cite_key in resolved order (with --build-manifest)')
ap.add_argument('--out-manifest', default=None, help='manifest output path (with --build-manifest)')
ap.add_argument('--selftest', action='store_true', help='run built-in cases and exit')
args = ap.parse_args()
if args.selftest:
return selftest()
if args.build_manifest:
if not (args.bib and args.keymap and args.out_manifest):
ap.error('--build-manifest requires --bib, --keymap and --out-manifest')
return build_manifest(args.bib, args.keymap, args.out_manifest)
if not (args.a and args.b and args.c):
ap.error('all of --a, --b, --c are required (or use --selftest). The gate recomputes '
'from A and B raw records and uses C only for flags; it will not trust C alone.')
if not (args.expect_manifest or args.expect_keys):
print('ERROR: coverage is MANDATORY — pass --expect-manifest (preferred, checks the '
'ordinal->key/title mapping) or --expect-keys. The gate must not run coverage-blind: '
'a dropped/substituted citation would otherwise read as "clean".', file=sys.stderr)
return 1
A, B, C = _load(args.a, 'A'), _load(args.b, 'B'), _load(args.c, 'C')
if A is None or B is None or C is None:
return 1
# Index each shard by dedup_key, catching duplicates PER shard (a duplicate is a broken
# join and must not silently overwrite).
def index(data, label):
idx, dupes = {}, set()
for e in data['entries']:
if not isinstance(e, dict):
print('ERROR: %s has a non-object entry' % label, file=sys.stderr)
return None, None
k = str(e.get('dedup_key') or '').strip()
if not k:
print('ERROR: %s has an entry with empty/whitespace/null dedup_key' % label, file=sys.stderr)
return None, None
if k in idx:
dupes.add(k)
idx[k] = e
return idx, dupes
ai, ad = index(A, 'A'); bi, bd = index(B, 'B'); ci, cd = index(C, 'C')
if ai is None or bi is None or ci is None:
return 1
alldupes = ad | bd | cd
if alldupes:
print('ERROR: duplicate dedup_key(s) within a shard: %s — the Step-2 join is broken.'
% ','.join(_safe(x) for x in sorted(alldupes)), file=sys.stderr)
return 1
# The THREE shards must cover the SAME dedup_key set — a key present in one and missing
# in another means the join dropped or invented a citation.
if not (set(ai) == set(bi) == set(ci)):
miss_b = sorted(set(ai) - set(bi)); miss_a = sorted(set(bi) - set(ai))
miss_c = sorted((set(ai) | set(bi)) - set(ci)); extra_c = sorted(set(ci) - (set(ai) | set(bi)))
print('ERROR: A/B/C dedup_key sets disagree — the Step-2 join is broken.', file=sys.stderr)
for lbl, v in (('in A not B', miss_b), ('in B not A', miss_a),
('missing from C', miss_c), ('only in C', extra_c)):
if v:
print(' %s: %s' % (lbl, ','.join(_safe(x) for x in v)), file=sys.stderr)
return 1
keys = ai # same set as bi/ci
if not keys:
print('ERROR: zero entries — a shard produced nothing. Re-run; do NOT treat this as '
'"no problems found".', file=sys.stderr)
return 1
if args.expect is not None and len(keys) != args.expect:
print('ERROR: coverage mismatch — %d entries but --expect %d cited keys. A citation '
'was dropped between Step 1 and the shards.' % (len(keys), args.expect), file=sys.stderr)
return 1
if args.expect_keys:
try:
want = {k.strip() for k in open(args.expect_keys).read().split() if k.strip()}
except OSError as exc:
print('ERROR: cannot read --expect-keys %s: %s' % (args.expect_keys, exc), file=sys.stderr)
return 1
got = set(keys)
if got != want:
missing, extra = sorted(want - got), sorted(got - want)
print('ERROR: dedup_key set mismatch vs Step 1.', file=sys.stderr)
if missing:
print(' missing (dropped): ' + ','.join(_safe(x) for x in missing), file=sys.stderr)
if extra:
print(' unexpected (invented downstream): ' + ','.join(_safe(x) for x in extra), file=sys.stderr)
return 1
if args.expect_manifest:
want_map = load_manifest(args.expect_manifest)
if want_map is None:
return 1
errs = manifest_mismatches(want_map, ai)
if errs:
print('ERROR: coverage manifest mismatch vs Step 1 — a citation was dropped and/or '
'substituted downstream (the ordinal set alone would not catch this):', file=sys.stderr)
for e in errs:
print(' ' + e, file=sys.stderr)
return 1
# B's blindness attestation is authoritative and read PER SHARD from B itself (a single
# bool for the whole B run), never from C. If B did not attest blindness, nothing
# certifies.
b_blind = B.get('blind') is True
results, counts = [], {'MATCH': 0, 'MINOR': 0, 'ESCALATE': 0}
downgraded = 0
for k in sorted(keys):
a_e, b_e, c_e = ai[k], bi[k], ci[k]
# A supplies the local .bib key + record; B the web record + found; C ONLY flags.
cite_key = a_e.get('key') or c_e.get('key')
a_rec = a_e.get('bibtex')
b_rec = b_e.get('bibtex')
found = b_e.get('found') is True
# C's flags are ADVISORY only: recompute routing from A/B raw artifacts and fail
# closed on a C schema violation, so a faulty/omitting/adversarial C cannot route.
flags = assemble_flags(c_e, a_rec, b_e)
entry = build_entry(k, cite_key, a_rec, b_rec, b_blind, found, flags)
verdict, reason, fields = classify(entry)
if verdict == 'ESCALATE' and 'not blind-attested' in reason:
downgraded += 1
counts[verdict] += 1
results.append({'dedup_key': k, 'key': cite_key, 'verdict': verdict,
'reason': reason, 'fields': fields, 'blind': b_blind})
print('%-10s %-28s %s' % (verdict, _safe(cite_key)[:28], _safe(reason)))
print('\nMATCH=%(MATCH)d MINOR=%(MINOR)d ESCALATE=%(ESCALATE)d' % counts)
if not b_blind:
print('B did not attest blindness (blind != true) — NO entry is certified; this run '
'FLAGS but does not CERTIFY.')
elif downgraded:
print('%d entr%s downgraded to ESCALATE for lack of a blindness attestation.'
% (downgraded, 'y' if downgraded == 1 else 'ies'))
print('ESCALATE entries REQUIRE the main agent\'s own primary-source fetch (Step 4).')
print('This gate recomputes the diff from A and B directly (C supplies only flags) and '
'never emits MISMATCH; only a main-agent fetch can produce one.')
if args.out:
with open(args.out, 'w') as fh:
json.dump({'gate': 'cx_verify.py', 'deterministic': True, 'recomputed_from': 'A+B',
'b_blind': b_blind, 'counts': counts, 'entries': results}, fh, indent=2)
print('wrote %s' % args.out)
return 0
if __name__ == '__main__':
sys.exit(main())
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |