Skip to content

zrwangems/dsh-zrwang-plugin / ars/skills/sr-screener

sr-screener

Flagged

系统综述文献筛选:把方案或 PROSPERO 协议转成资格规则,双盲筛选标题摘要与全文,第三人裁决冲突,输出 PRISMA 2020 计数、EndNote/Zotero RIS 分组与可交付给论文技能的 literature_corpus。Triggers: systematic review screening, screen papers, PRISMA counts, pilot screening, adjudicate conflicts | 文献筛选, 系统综述筛选, PRISMA 计数, 筛选冲突裁决。

FileKindSize
SKILL.mdskill md33.1 KB
_shared/ARCHITECTURE.mdmarkdown51.7 KB
_shared/MODE_REGISTRY.mdmarkdown6.7 KB
_shared/agents/compliance_agent.mdmarkdown9.4 KB
_shared/artifact_reproducibility_pattern.mdmarkdown9.8 KB
_shared/benchmark_report.schema.jsonconfig2.7 KB
_shared/benchmark_report_pattern.mdmarkdown8.8 KB
_shared/bibliographic_integrity_signals.mdmarkdown8.1 KB
_shared/collaboration_depth_rubric.mdmarkdown11.5 KB
_shared/compliance_checkpoint_protocol.mdmarkdown8.9 KB
_shared/compliance_report.schema.jsonconfig8.1 KB
_shared/contracts/README.mdmarkdown54.7 KB
_shared/contracts/activity/adjudication_activity_input.schema.jsonconfig18.5 KB
_shared/contracts/activity/adjudication_activity_store.schema.jsonconfig18.7 KB
_shared/contracts/audit/audit_jsonl.schema.jsonconfig5.4 KB
_shared/contracts/audit/audit_sidecar.schema.jsonconfig8.4 KB
_shared/contracts/audit/audit_verdict.schema.jsonconfig5.4 KB
_shared/contracts/audit/cross_document_consistency_advisory.schema.jsonconfig20.4 KB
_shared/contracts/audit/cross_document_consistency_advisory_draft.schema.jsonconfig23.4 KB
_shared/contracts/audit/cross_document_source_manifest.schema.jsonconfig6.3 KB
_shared/contracts/audit/tortured_phrase_advisory.schema.jsonconfig30.9 KB
_shared/contracts/audit/tortured_phrase_snapshot.schema.jsonconfig4.9 KB
_shared/contracts/audit/tortured_phrase_snapshot_manifest.schema.jsonconfig7.3 KB
_shared/contracts/capability/stage_capability_matrix.jsonconfig23.5 KB
_shared/contracts/claim_standing/candidate_ledger.schema.jsonconfig23.1 KB
_shared/contracts/claim_standing/query_plan.schema.jsonconfig12.7 KB
_shared/contracts/claim_standing/query_plan_v1_1.schema.jsonconfig20.3 KB
_shared/contracts/claim_standing/retrieval_input.schema.jsonconfig14.4 KB
_shared/contracts/claim_standing/stance_record.schema.jsonconfig11.7 KB
_shared/contracts/claim_standing/transmission_ledger.schema.jsonconfig7.6 KB
_shared/contracts/cross_model/codex_citation_receipt.schema.jsonconfig4.7 KB
_shared/contracts/cross_model/codex_citation_request.schema.jsonconfig986 B
_shared/contracts/cross_model/promotion_bakeoff_sealed_commitment.schema.jsonconfig1.5 KB
_shared/contracts/cross_model/promotion_bakeoff_sealed_reveal.schema.jsonconfig1.5 KB
_shared/contracts/degradation_registry.jsonconfig42.5 KB
_shared/contracts/evaluator/full.jsonconfig6.3 KB
_shared/contracts/evidence/claim_registry.schema.jsonconfig2.3 KB
_shared/contracts/evidence/claim_registry_coverage_report.schema.jsonconfig4.8 KB
_shared/contracts/evidence/evidence_row.schema.jsonconfig14.3 KB
_shared/contracts/evidence/evidence_row_v1_1.schema.jsonconfig11.4 KB
_shared/contracts/evidence/evidence_row_v1_2.schema.jsonconfig21.8 KB
_shared/contracts/evidence/evidence_row_v1_3.schema.jsonconfig10.7 KB
_shared/contracts/human_subjects/authority_profile_registry.schema.jsonconfig16.7 KB
_shared/contracts/human_subjects/committee_correspondence.schema.jsonconfig9.9 KB
_shared/contracts/human_subjects/content_coverage_advisory.schema.jsonconfig19.9 KB
_shared/contracts/human_subjects/irb_context_record.schema.jsonconfig12.1 KB
_shared/contracts/human_subjects/resolved_authority_context.schema.jsonconfig11.8 KB
_shared/contracts/human_subjects/review_pathway_rule_trace.schema.jsonconfig9.5 KB
_shared/contracts/human_subjects/review_pathway_trace_request.schema.jsonconfig4.8 KB
_shared/contracts/human_subjects/submission_packet_inventory.schema.jsonconfig8.5 KB
_shared/contracts/human_subjects/submission_packet_manifest.schema.jsonconfig32.2 KB
_shared/contracts/passport/audit_artifact_entry.schema.jsonconfig10.8 KB
_shared/contracts/passport/bibliographic_integrity_signal.schema.jsonconfig48.9 KB
_shared/contracts/passport/citation_provenance.schema.jsonconfig4.8 KB
_shared/contracts/passport/citation_verification_summary.schema.jsonconfig8.2 KB
_shared/contracts/passport/claim_audit_result.schema.jsonconfig7.8 KB
_shared/contracts/passport/claim_drift.schema.jsonconfig3.8 KB
_shared/contracts/passport/claim_intent_manifest.schema.jsonconfig6.6 KB
_shared/contracts/passport/constraint_violation.schema.jsonconfig3.6 KB
_shared/contracts/passport/excluded_source_entry.schema.jsonconfig1.7 KB
_shared/contracts/passport/experiment_alignment_result.schema.jsonconfig4.8 KB
_shared/contracts/passport/experiment_provenance_entry.schema.jsonconfig10.0 KB
_shared/contracts/passport/human_read_log.schema.jsonconfig3.8 KB
_shared/contracts/passport/inquiry_ledger_ref.schema.jsonconfig881 B
_shared/contracts/passport/literature_corpus_entry.schema.jsonconfig30.3 KB
_shared/contracts/passport/preregistration_artifact.schema.jsonconfig4.3 KB
_shared/contracts/passport/rejection_log.schema.jsonconfig3.3 KB
_shared/contracts/passport/reset_ledger_entry.schema.jsonconfig7.1 KB
_shared/contracts/passport/run_ledger.schema.jsonconfig8.0 KB
_shared/contracts/passport/standing_constraint_entry.schema.jsonconfig2.6 KB
_shared/contracts/passport/temporal_audit_results.schema.jsonconfig8.8 KB
_shared/contracts/passport/terminal_policies.schema.jsonconfig8.1 KB
_shared/contracts/passport/timeline.schema.jsonconfig4.7 KB
_shared/contracts/passport/uncited_assertion.schema.jsonconfig3.3 KB
_shared/contracts/passport/uncited_audit_failure.schema.jsonconfig4.4 KB
_shared/contracts/passport/user_attested_read_resolution.schema.jsonconfig3.2 KB
_shared/contracts/passport/version_records.schema.jsonconfig4.9 KB
_shared/contracts/patch/block_manifest.schema.jsonconfig2.1 KB
_shared/contracts/patch/legacy/v1_0/revision_patch.schema.jsonconfig5.3 KB
_shared/contracts/patch/revision_patch.schema.jsonconfig8.9 KB
_shared/contracts/pdf/pdf_content_classifier_diagnostic.schema.jsonconfig1.2 KB
_shared/contracts/pdf/pdf_content_classifier_worker.schema.jsonconfig2.2 KB
_shared/contracts/pdf/pdf_read_preflight.schema.jsonconfig6.8 KB
_shared/contracts/re_review/input_manifest.schema.jsonconfig7.6 KB
_shared/contracts/re_review/legacy/v1_0/input_manifest.schema.jsonconfig6.2 KB
_shared/contracts/re_review/legacy/v1_0/precommitment.schema.jsonconfig8.9 KB
_shared/contracts/re_review/legacy/v1_0/traceability.schema.jsonconfig34.2 KB
_shared/contracts/re_review/legacy/v1_0/verdict_record.schema.jsonconfig10.5 KB
_shared/contracts/re_review/precommitment.schema.jsonconfig8.6 KB
_shared/contracts/re_review/traceability.schema.jsonconfig37.7 KB
_shared/contracts/re_review/verdict_record.schema.jsonconfig10.5 KB
_shared/contracts/research_workflow/inquiry_branch_ledger.schema.jsonconfig17.0 KB
_shared/contracts/research_workflow/research_workflow_profile.schema.jsonconfig7.2 KB
_shared/contracts/research_workflow/research_workflow_profile_selection_receipt.schema.jsonconfig3.9 KB
_shared/contracts/review_target/constructive_review_findings.schema.jsonconfig7.0 KB
_shared/contracts/review_target/criteria_registry.schema.jsonconfig4.0 KB
_shared/contracts/review_target/review_criteria_binding_manifest.schema.jsonconfig7.9 KB
_shared/contracts/review_target/review_criteria_source_receipt.schema.jsonconfig5.6 KB
_shared/contracts/review_target/review_target_context.schema.jsonconfig4.5 KB
_shared/contracts/review_target/review_target_declaration.schema.jsonconfig4.6 KB
_shared/contracts/reviewer/full.jsonconfig3.7 KB
_shared/contracts/reviewer/methodology_focus.jsonconfig2.2 KB
_shared/contracts/reviewer/review_panel_provenance.schema.jsonconfig7.8 KB
_shared/contracts/reviewer/review_panel_provenance_carrier.schema.jsonconfig4.1 KB
_shared/contracts/reviewer/review_panel_provenance_input.schema.jsonconfig3.5 KB
_shared/contracts/revision/author_adjudication.schema.jsonconfig7.8 KB
_shared/contracts/revision/author_adjudication_input.schema.jsonconfig1.2 KB
_shared/contracts/revision/claim_strength_drift_disposition.schema.jsonconfig3.6 KB
_shared/contracts/revision/claim_strength_drift_disposition_input.schema.jsonconfig3.1 KB
_shared/contracts/revision/claim_strength_drift_findings.schema.jsonconfig4.2 KB
_shared/contracts/revision/claim_surface_manifest.schema.jsonconfig3.3 KB
_shared/contracts/revision/integrity_correction_authorization.schema.jsonconfig4.3 KB
_shared/contracts/revision/integrity_correction_authorization_input.schema.jsonconfig1.5 KB
_shared/contracts/revision/integrity_correction_list.schema.jsonconfig1.8 KB
_shared/contracts/revision/integrity_pass_receipt.schema.jsonconfig809 B
_shared/contracts/revision/revision_evidence_bundle.schema.jsonconfig4.9 KB
_shared/contracts/revision/revision_roadmap.schema.jsonconfig11.2 KB
_shared/contracts/submission/format_profile.example.yamlconfig971 B
_shared/contracts/submission/format_profile.schema.jsonconfig4.6 KB
_shared/contracts/submission/submission_verification_report.schema.jsonconfig10.3 KB
_shared/contracts/submission/venue_profile.schema.jsonconfig4.1 KB
_shared/contracts/writer/full.jsonconfig3.6 KB
_shared/cross_model_verification.mdmarkdown83.3 KB
_shared/description-original.mdmarkdown1.1 KB
_shared/dsh-runtime.mdmarkdown1.5 KB
_shared/evals_lift_report.schema.jsonconfig5.0 KB
_shared/ground_truth_isolation_pattern.mdmarkdown16.0 KB
_shared/handoff_schemas.mdmarkdown96.9 KB
_shared/human_subjects_authority_registry.jsonconfig49.5 KB
_shared/mode_spectrum.mdmarkdown5.2 KB
_shared/model_tiering.mdmarkdown12.6 KB
_shared/output_language_pair.mdmarkdown7.7 KB
_shared/policy_data/nature_policy.mdmarkdown3.9 KB
_shared/prisma_trAIce_protocol.mdmarkdown10.2 KB
_shared/raise_framework.mdmarkdown7.5 KB
_shared/references/authority_content_coverage_advisory_protocol.mdmarkdown14.0 KB
_shared/references/claim_standing_candidate_ledger_protocol.mdmarkdown19.2 KB
_shared/references/claim_strength_ladder.mdmarkdown5.5 KB
_shared/references/cross_document_consistency_advisory_protocol.mdmarkdown13.5 KB
_shared/references/evidence_row_protocol.mdmarkdown14.4 KB
_shared/references/firm_rules.mdmarkdown11.0 KB
_shared/references/human_subjects_authority_protocol.mdmarkdown13.1 KB
_shared/references/intent_clarification_protocol.mdmarkdown11.9 KB
_shared/references/irb_terminology_glossary.mdmarkdown12.4 KB
_shared/references/per_source_method_weaknesses.mdmarkdown4.2 KB
_shared/references/protected_hedging_phrases.mdmarkdown8.9 KB
_shared/references/psychometric_terminology_glossary.mdmarkdown8.7 KB
_shared/references/review_criteria_consumer_protocol.mdmarkdown10.3 KB
_shared/references/review_form_note.mdmarkdown11.3 KB
_shared/references/review_pathway_rule_trace_protocol.mdmarkdown8.3 KB
_shared/references/routing_core.mdmarkdown5.7 KB
_shared/references/submission_packet_manifest_protocol.mdmarkdown13.6 KB
_shared/references/word_count_conventions.mdmarkdown9.0 KB
_shared/research_workflow_profiles/field_general.jsonconfig1.2 KB
_shared/review_criteria_registry.jsonconfig11.5 KB
_shared/review_criteria_sources/msr-2027-technical-papers.2026-08-24.jsonconfig2.0 KB
_shared/review_criteria_sources/sigsoft-empirical-standards.2026-08-24.jsonconfig1.4 KB
_shared/sprint_contract.schema.jsonconfig19.0 KB
_shared/style_calibration_protocol.mdmarkdown6.9 KB
_shared/templates/codex_audit_multifile_template.mdmarkdown16.7 KB
agents/protocol_architect_agent.mdmarkdown6.4 KB
agents/qc_auditor_agent.mdmarkdown5.2 KB
agents/reporter_agent.mdmarkdown3.7 KB
agents/screening_reviewer_agent.mdmarkdown5.4 KB
examples/example_config_dta.jsonconfig1.8 KB
examples/example_protocol_dta.mdmarkdown3.8 KB
examples/quick_screen_example.mdmarkdown3.0 KB
references/decision_rules.mdmarkdown6.7 KB
references/failure_paths.mdmarkdown4.1 KB
references/orchestration.mdmarkdown12.4 KB
references/protocol_template.mdmarkdown6.2 KB
references/quality_control.mdmarkdown6.0 KB
references/reporting_and_handoff.mdmarkdown5.8 KB
references/reviewer_roles.mdmarkdown4.0 KB
scripts/_claim_audit_constants.pyscript13.7 KB
scripts/_markdown_lint_util.pyscript9.0 KB
scripts/_next_verified_at_ms.pyscript6.5 KB
scripts/_skill_lint.pyscript13.6 KB
scripts/ars_cache_invalidate.pyscript1.3 KB
scripts/ars_mark_read.pyscript18.2 KB
scripts/ars_write_scope_guard.pyscript30.9 KB
scripts/audit_snapshot.pyscript23.5 KB
scripts/bibliographic_integrity_signals.pyscript32.2 KB
scripts/build_claim_standing_candidate_ledger.pyscript54.3 KB
scripts/build_claim_standing_query_plan.pyscript25.0 KB
scripts/build_content_coverage_advisory.pyscript45.1 KB
scripts/build_cross_document_consistency_advisory.pyscript77.9 KB
scripts/build_outputs.pyscript33.0 KB
scripts/build_review_pathway_rule_trace.pyscript34.2 KB
scripts/build_submission_packet_manifest.pyscript88.1 KB
scripts/build_workflow.pyscript20.0 KB
scripts/check_audit_artifact_consistency.pyscript103.9 KB
scripts/check_benchmark_report.pyscript2.4 KB
scripts/check_claim_audit_consistency.pyscript68.9 KB
scripts/check_claim_standing_freshness.pyscript9.7 KB
scripts/check_claim_standing_transmissions.pyscript17.9 KB
scripts/check_committee_correspondence.pyscript14.9 KB
scripts/check_corpus_consumer_protocol.pyscript13.7 KB
scripts/check_cross_model_handoff_contract.pyscript9.9 KB
scripts/check_data_access_level.pyscript5.2 KB

scripts/check_benchmark_report.py

View on GitHub
#!/usr/bin/env python3
"""Validate an ARS benchmark report against benchmark_report.schema.json.

Usage: python scripts/check_benchmark_report.py path/to/report.json

Exit 0 on pass (warnings may still be printed to stderr).
Exit 1 on validation failure.
"""
from __future__ import annotations

import argparse
import json
import sys
from pathlib import Path

import jsonschema

SCHEMA_PATH = Path(__file__).resolve().parent.parent / "shared" / "benchmark_report.schema.json"


def load_schema() -> dict:
    return json.loads(SCHEMA_PATH.read_text(encoding="utf-8"))


def validate(report: dict) -> list[str]:
    schema = load_schema()
    validator = jsonschema.Draft202012Validator(schema)
    return [f"{list(e.absolute_path)}: {e.message}" for e in validator.iter_errors(report)]


def warn_self_scored(report: dict) -> list[str]:
    warnings = []
    scoring = report.get("metrics", {}).get("scoring_independence")
    if scoring == "self-scored":
        warnings.append(
            "WARNING: metrics.scoring_independence = 'self-scored'. "
            "Self-scoring is permitted but a known reward-hacking risk. "
            "Consider blind-scored or third-party-scored instead."
        )
    sample = report.get("human_baseline", {}).get("sample_size", 0)
    if isinstance(sample, int) and 1 <= sample <= 2:
        warnings.append(
            f"WARNING: human_baseline.sample_size = {sample}. "
            "n<=2 is dramatic, not statistical. Consider n>=5 or explicit disclosure in caveats."
        )
    return warnings


def main() -> int:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("report", type=Path, help="Path to the benchmark report JSON")
    args = parser.parse_args()

    try:
        report = json.loads(args.report.read_text(encoding="utf-8"))
    except (FileNotFoundError, json.JSONDecodeError) as exc:
        print(f"ERROR: failed to load {args.report}: {exc}")
        return 1

    errors = validate(report)
    if errors:
        for e in errors:
            print(f"ERROR: {e}")
        print(
            f"\n{len(errors)} schema violation(s). "
            f"See shared/benchmark_report_pattern.md for field rationale.",
            file=sys.stderr,
        )
        return 1

    for w in warn_self_scored(report):
        print(w, file=sys.stderr)

    print("OK: benchmark report validates.")
    return 0


if __name__ == "__main__":
    sys.exit(main())
add-product-e2e-evalpaperclipai/paperclipAdd or extend a Paperclip full-stack runner E2E workflow, fixture, matcher, or report evidence path for local or Daytona execution.Stars: 115kUpdated Claude CodeCodexVerifiedInstall
add-runner-evalpaperclipai/paperclipAdd or extend a Paperclip Runner protocol evaluation definition, roster, assertion, or report fixture with provenance and narrow validation.Stars: 115kUpdated Claude CodeCodexVerifiedInstall
diagnose-why-work-stoppedpaperclipai/paperclipCompatibility entry point for failed, stalled, looping, or over-recovered Paperclip work. Use fix-failed-paperclip-run for the full diagnosis, regression, repair, real verification, and green-PR workflow; honor an explicitly requested diagnosis-only scope.Stars: 115kUpdated Claude CodeCodexVerifiedInstall
fix-failed-paperclip-runpaperclipai/paperclipDiagnose and fix failed, stuck, slow, or incorrectly blocked Paperclip tasks and runs. Use with a task/run URL or failure report to trace the root cause, prove a regression red then green, check second-order effects, verify the real workflow, and deliver a PR with passing checks.Stars: 115kUpdated Claude CodeCodexVerifiedInstall