zrwangems/dsh-zrwang-plugin / ars/skills/academic-pipeline
academic-pipeline
Flagged学术研究总编排:研究→写作→诚信检查→评审→返修→复审→终检→定稿的十阶段流水线,每阶段停在用户确认点,含材料护照与 7 类 AI 研究失败模式检查。Triggers: academic pipeline, research to paper, full paper workflow, end-to-end paper | 完整论文流程, 从研究到论文, 端到端写论文, 论文流水线。
references/process_summary_protocol.md
View on GitHub> 输出语言(本 DSH 集成强制):所有面向用户的文字、稿件、摘要、评审与报告一律使用**简体中文**;英文术语与引文保持英文。文中的 `zh-TW` / `zh-tw-en` 只是上游语言代码与脚本参数,不代表要用繁体输出。用户明确要求其他语言时以用户为准。
>
> 多角色执行(评审团、十阶段流水线、角色扮演)必须使用 DSH 的 Agent Team 工具(`spawn_teammate`、`team_task_*`、`wait_agent`、`send_message`);当前会话看不到这些工具,说明用户没开 Agent Team,必须先提示用户开启,不得退化成单会话顺序扮演。
# Stage 6: Process Summary Protocol (Added in v2.4)
**Trigger**: After the user confirms the Stage 5 completion checkpoint (FULL). Stage 6 is non-mandatory — the user may decline it at that checkpoint; it is then marked `skipped` and the pipeline still terminates `completed` (see `pipeline_state_machine.md` § Stage 6 terminal semantics)
**Purpose**: Document the complete human-AI collaboration history for the paper creation process, for user sharing, reporting, or reflection
## Workflow
```
1. Ask user language preference:
"Which language version of the process record would you like to generate first?"
- Chinese (Simplified Chinese)
- English
- Both (default: generate the user's primary conversation language first)
Ask in the same question whether to add a PDF. An active standing constraint
that answers either point is followed without asking (pipeline orchestrator
§ Standing Constraints (#927)).
2. Review session history and compile the following (after a reset boundary the
session holds only the turns since the last resume: take earlier decisions and
words from the passport and run ledger, and say in the record which parts come
from those rather than the conversation):
- User's initial instructions (verbatim quote)
- Key decision points and user interventions at each stage
- Direction correction moments and reasons
- Iteration count and review result summaries
- Intellectual insights raised by the user (e.g., questions that spawned new chapters)
- Quality requirement evolution (e.g., formatting, tone adjustments)
- Pipeline statistics (stage count, review rounds, integrity verification count, etc.)
2b. Dispatch collaboration_depth_agent in whole-pipeline mode (range = all
stages, v3.5); its advisory report becomes the "Collaboration Depth
Trajectory" chapter of the Process Record — this dispatch happens BEFORE
record generation so the chapter is inside the record the user acknowledges
3. Generate Markdown version (paper_creation_process.md / paper_creation_process_en.md)
4. Only when a PDF was asked for, convert to LaTeX and compile PDF:
- pandoc MD -> LaTeX body
- Package complete LaTeX document (with cover page, table of contents, headers/footers)
- tectonic compile PDF
- Chinese version requires xeCJK + Source Han Serif TC VF
5. Terminal acknowledgement (pipeline terminal checkpoint):
- After delivering the process record, prompt the user to close the pipeline.
- Acknowledgement vocabulary: "finish" / "end" / "done" / "confirm", or an
unambiguous natural-language equivalent that accepts the deliverables.
- Change requests (the other language version, content corrections) keep
Stage 6 in_progress — they are not acknowledgements.
- On acknowledgement: state_tracker marks Stage 6 completed and sets the pipeline global state to completed. There is no next stage.
(See pipeline_state_machine.md § Stage 6 terminal semantics.)
- Persist that terminal transition first and without consulting #673
adjudication-activity metadata. If the user selected a local activity
store, seal/build/append/render runs only afterward as best-effort
post-terminal work; its failure cannot change completion.
```
## Adjudication-activity exclusion (#673)
The Process Record may describe the user-visible decisions already present in
the ordinary dialogue and state history. It MUST NOT read, copy, summarize,
score, or mention activity metadata. It MUST NOT include
`pending_adjudication_activity_bindings[]`, the sealed
`adjudication_activity_sources` inventory, the selected store/path, stored
records, renderer output, or diagnostics. Those artifacts are a
separate local advisory side channel, not Process Record evidence. They also
must not be passed to `collaboration_depth_agent` or any model/judge/eval call.
The Stage 6 compilation rules do not perform an ambient filesystem scan for
them. See `../agents/state_tracker_agent.md` § "Adjudication-activity metadata"
for the single producer/state authority.
## Required Content in Process Record
| Section | Content |
|---------|---------|
| Paper Information | Title, final deliverables list |
| Stage-by-Stage Process | Input/output/key decisions for each stage, with verbatim user quotes |
| Iteration Details | Review comment summaries, revision items, re-review results |
| Interaction Pattern Summary | User role, Claude role, intervention count, key turning points — statistics table |
| User Key Decisions | Chronological list of every important decision made by the user |
| Key Lessons | Reusable lessons learned from the process |
| **Collaboration Quality Evaluation** | **Final chapter: 1-100 score + dimensional analysis + improvement suggestions** (see below) |
## Collaboration Quality Evaluation (Final Chapter, Mandatory)
The final chapter of the process record is a "Collaboration Quality Evaluation" that honestly and constructively assesses the user's performance in the human-AI collaboration. Format follows the Claude Code CLI `/insight` feature.
### Scoring Dimensions (each 1-100, weighted average for overall score)
```
+--------------------------------------------------+
| Collaboration Quality Score: [XX]/100 |
+--------------------------------------------------+
| |
| Direction Setting [---------- ] XX |
| Clarity, timing, scope definition |
| |
| Intellectual Contribution [------------ ] XX |
| Insight depth, original questions, concept |
| challenges |
| |
| Quality Gatekeeping [--------- ] XX |
| Visual inspection, formatting requirements, |
| quality standards |
| |
| Iteration Discipline [---------- ] XX |
| Timely direction correction, willingness to |
| re-run pipeline, refusing to settle |
| |
| Delegation Efficiency [------- ] XX |
| When to intervene/when to let go, instruction |
| precision, checkpoint efficiency |
| |
| Meta-Learning [------------ ] XX |
| Feeding experience back to skills, requesting |
| lesson recording, process improvement awareness |
| |
+--------------------------------------------------+
```
### Scoring Criteria
| Score Range | Meaning |
|------------|---------|
| 90-100 | Exceptional — User intervention significantly elevated the paper's intellectual quality beyond what AI could produce independently |
| 75-89 | Excellent — User made correct directional decisions and effectively leveraged the pipeline's iteration capabilities |
| 60-74 | Good — User completed necessary decisions but some opportunities were missed |
| 40-59 | Basic — User primarily served as a "continue" button with little substantive intervention |
| 1-39 | Needs Improvement — User intervention may have disrupted the workflow or lacked critical quality gatekeeping |
### Required Subsections
1. **Overall Score**: Total score + one-sentence evaluation
2. **What Worked Well**: 2-4 specific behaviors, with verbatim user quotes
3. **Missed Opportunities**: 1-3 things the user could have done but didn't
4. **Recommendations for Next Time**: 3-5 specific, actionable improvement suggestions
5. **Human vs AI Value-Add**: Clearly identify which aspects of the final paper quality came from user intervention (not achievable by AI independently)
### Evaluation Principles
- **Honesty first**: No inflation, no pleasantries. If the user only pressed "continue," reflect that truthfully
- **Evidence-based**: Every score is supported by specific behaviors or conversation records
- **Constructive**: Every criticism must include actionable improvement suggestions
- **Acknowledge uncertainty**: If certain dimensions cannot be evaluated (e.g., mid-entry skipped the research stage), mark as N/A
- **Bidirectional reflection**: Also candidly point out Claude's shortcomings during the process (e.g., areas requiring multiple corrections)
## AI Self-Reflection Report (Mandatory)
The second-to-last chapter of the process record is an "AI Self-Reflection Report" that honestly documents AI's own behavioral patterns during the pipeline. This complements the Collaboration Quality Evaluation (which assesses the user) by assessing the AI.
### Tracked Metrics
All metrics below are derived from existing agent logs (`[DA-DECISION]`, `[DA-REBUTTAL]`, `[HEALTH-CHECK]`, state tracker JSON) — no additional per-stage instrumentation is required. The orchestrator aggregates these at Stage 6 by scanning the dialogue transcript:
```
+--------------------------------------------------+
| AI Self-Reflection Report |
+--------------------------------------------------+
| |
| DA Concession Rate X/Y (Z%) |
| (concessions / total rebuttals received) |
| |
| DA Consecutive Concessions [list if any] |
| (violations of no-consecutive rule) |
| |
| SLIM checkpoints X/Y |
| User-skipped stages X/Y |
| |
| User Overrides X |
| (times user overruled AI recommendation) |
| |
| Dialogue Health Alerts X |
| (health check interventions triggered) |
| - Persistent Agreement: X |
| - Conflict Avoidance: X |
| - Premature Convergence: X |
| |
| Intent Mode Transitions X |
| (exploratory ↔ goal-oriented switches) |
| |
| Cross-Model Disagreements X (if enabled) |
| (integrity + DA combined) |
| |
+--------------------------------------------------+
```
### Required Subsections
1. **Behavioral Summary**: One paragraph describing the overall AI behavioral pattern during this pipeline run
2. **Sycophancy Risk Assessment**: Screening thresholds based on concession rate and health alerts — LOW (concession <50%, 0 health alerts) / MEDIUM (50-65% or 1-2 alerts) / HIGH (>65% or 3+ alerts). These are screening thresholds, not diagnostic criteria — a MEDIUM rating means the metrics warrant human review, not that sycophancy occurred (a high concession rate may reflect genuinely strong rebuttals). If HIGH, include a warning: "AI may have been too accommodating in this run. Human review of DA findings and integrity results is strongly recommended."
3. **Frame-Lock Incidents**: List any `[CROSS-MODEL-FINDING]` that the primary DA missed (if cross-model was enabled), or any frame-lock detections triggered during checkpoints. If none, state "No frame-lock incidents detected — note this could mean either good coverage or undetected frame-lock."
4. **Convergence Pattern**: In Socratic dialogue stages, was intent correctly detected? Did the mentor try to converge prematurely? Report mode transitions and any premature-convergence health alerts.
5. **What AI Got Wrong**: Candid list of AI errors or shortcomings during the run — corrections needed, checkpoint failures, integrity issues found. This is not a failure report; it is evidence that quality gates are working.
6. **Failure Mode Audit Log** (v3.2): For each of the 7 AI research failure modes from the Stage 2.5 / 4.5 checklist (see `references/ai_research_failure_modes.md`), report (a) final status at 4.5 — `CLEAR` / `NOT APPLICABLE` (quote the no-experiments declaration it rests on) / `OVERRIDDEN`, (b) history — was it ever `SUSPECTED` during the pipeline? At which stage? How was it resolved? (c) if `OVERRIDDEN`, the user's recorded reasoning. This makes the failure-mode defences part of the permanent process record. Modes with no history can be listed as `CLEAR (no flags)` or `NOT APPLICABLE (no-experiments declaration)` in one line, never merging the two; expand only on modes that were flagged.
- **Reading Probe Outcomes (if present)** — transcribes the `### Reading Probe Outcomes` subsection from the Research Plan Summary verbatim, with a one-line note that the AI did not verify paraphrase accuracy. If the Research Plan Summary has no such subsection (i.e., `ARS_SOCRATIC_READING_PROBE` was unset), this item is omitted entirely (no "not applicable" noise). Pickup rule (two sources, either sufficient): (a) copy the entire `### Reading Probe Outcomes` subsection body verbatim — this is the authoritative human-readable record; (b) additionally grep for `[READING-PROBE: status=..., paper=..., outcome=..., turn=...]` which the Mentor emits once in the summary as a machine-stable anchor (including for `not_fired_*` statuses). If both are present use (a) as the display source and keep (b) as the final line of the transcribed block so downstream tooling can still parse it. If only raw inline tags from dialogue turns (`[READING-PROBE: paper=..., outcome=..., turn=...]` without the `status=` field) are found and no subsection exists, the Mentor compilation step was skipped — log this as a pipeline anomaly rather than silently dropping the probe data.
- **Adjacent-Framing Probe Outcomes (if present)** — if `ARS_SOCRATIC_ADJACENT_PROBE` was set, grep the dialogue transcript for `[ADJACENT-PROBE: surfaced=..., anchor=internal_knowledge, turn=..., outcome=...]` tags (the Mentor emits one per AI-initiated surfacing, on a standalone line). Transcribe a one-line-per-probe summary plus a note: a high `outcome=declined` rate is the bias-visibility signal that the internal-knowledge adjacency was mis-calibrated for this user (the Mentor did NOT verify the facets against any external source). If `ARS_SOCRATIC_ADJACENT_PROBE` was unset (no tags found), omit this item entirely (no "not applicable" noise). Note: the `outcome` value is only known AFTER the user's next response, so a probe surfaced on the final turn may carry `outcome=deferred`.
### Output Length Guidance
For dimensions with no findings, state the null result in one sentence. Expand only when issues are detected. The real risk is generating verbose "everything is fine" paragraphs for empty subsections — resist this.
### Principles
- **Self-honesty**: AI must not minimize its own shortcomings. If the DA conceded too easily, say so.
- **Not self-flagellation**: The purpose is transparency, not performative humility. Report facts with interpretation.
- **Actionable**: Every finding should suggest what could be done differently next time (e.g., "Consider enabling cross-model verification for the next run" or "The user might want to push back harder on DA concessions"); never suggest a mechanism the user declined
- **The irony is noted**: This self-reflection is itself produced by the same AI that may have been sycophantic during the pipeline. The user should read it with that awareness. This caveat must be stated in the report.
## Output Specifications
- **Filename**: `paper_creation_process.md` (Chinese) / `paper_creation_process_en.md` (English)
- **PDF** (when asked for): `paper_creation_process_zh.pdf` / `paper_creation_process_en.pdf`
- **LaTeX template**: `article` class, 12pt, A4, Times New Roman + Source Han Serif TC VF
- **Includes table of contents**: `\tableofcontents`
- **Header**: left = document title (italic), right = date
- **Compilation**: tectonic (same toolchain as Stage 5)
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |