LeoYeAI/agent-master-skills / skills/ai-powered-literature-review-skills
literature-reviewer-skill
Unverified根据用户提供的论文主题,进行系统性中英文文献回顾(Literature Survey)。 采用8阶段工作流,支持CNKI、Web of Science、ScienceDirect等主流数据库, 无需API配置,通过浏览器自动化获取文献信息。 输出包含GB/T 7714-2015引文、标题、摘要的Markdown文档。 当用户提到"文献回顾"、"文献综述"、"帮我找文献"、"中英文文献搜索"、"写综述"等关键词时触发。 NOTE: The 'examples/' folder contains sample outputs and is not included in skill installation.
| File | Kind | Size |
|---|---|---|
| AGENTS.md | markdown | 12.8 KB |
| README.md | markdown | 19.7 KB |
| SECURITY.md | markdown | 5.1 KB |
| SKILL.md | skill md | 22.1 KB |
| _meta.json | config | 316 B |
| agents/explore-agent.md | markdown | 4.2 KB |
| agents/orchestrator.md | markdown | 17.3 KB |
| agents/synthesize-agent.md | markdown | 12.4 KB |
| agents/verify-agent.md | markdown | 2.3 KB |
| docker-compose.yml | config | 3.3 KB |
| install.yaml | config | 2.6 KB |
| mcp.json | config | 5.6 KB |
| references/cnki-guide.md | markdown | 10.0 KB |
| references/database-access.md | markdown | 3.1 KB |
| references/gb-t-7714-2015.md | markdown | 10.8 KB |
| scripts/__init__.py | script | 519 B |
| scripts/citation_formatter.py | script | 5.3 KB |
| scripts/deduplicate_papers.py | script | 4.8 KB |
| scripts/models.py | script | 2.6 KB |
scripts/deduplicate_papers.py
View on GitHub#!/usr/bin/env python3
"""
文献去重工具
功能:
- 基于DOI精确去重
- 基于标题相似度去重
- 保留信息更完整的版本
"""
from typing import List, Dict, Tuple
from difflib import SequenceMatcher
def calculate_similarity(str1: str, str2: str) -> float:
"""
计算两个字符串的相似度
Args:
str1: 第一个字符串
str2: 第二个字符串
Returns:
相似度(0.0 - 1.0)
"""
if not str1 or not str2:
return 0.0
s1 = str1.lower().strip()
s2 = str2.lower().strip()
return SequenceMatcher(None, s1, s2).ratio()
def normalize_doi(doi: str) -> str:
"""标准化DOI"""
if not doi:
return ""
doi = doi.replace("https://doi.org/", "")
doi = doi.replace("http://doi.org/", "")
doi = doi.replace("doi.org/", "")
doi = doi.replace("DOI:", "")
doi = doi.replace("doi:", "")
return doi.strip().lower()
def calculate_paper_quality_score(paper: Dict) -> float:
"""
计算文献质量分数(用于决定保留哪个重复版本)
"""
score = 0.0
# 有DOI加分
if paper.get("doi"):
score += 10
# 有完整元数据加分
if paper.get("abstract"):
score += 5
if paper.get("volume") and paper.get("issue"):
score += 3
if paper.get("pages"):
score += 2
return score
def deduplicate_papers(
papers: List[Dict],
title_similarity_threshold: float = 0.85
) -> Tuple[List[Dict], List[Dict]]:
"""
文献去重主函数
去重策略:
1. DOI精确匹配
2. 标题相似度匹配
Args:
papers: 文献列表
title_similarity_threshold: 标题相似度阈值(默认0.85)
Returns:
(去重后的文献列表, 被移除的重复文献列表)
"""
if not papers:
return [], []
unique_papers = []
removed_papers = []
seen_dois = set()
for paper in papers:
# 检查DOI
doi = normalize_doi(paper.get("doi", ""))
if doi:
if doi in seen_dois:
removed_papers.append(paper)
continue
seen_dois.add(doi)
unique_papers.append(paper)
continue
# 没有DOI,检查标题相似度
title = paper.get("title", "").lower()
is_duplicate = False
for existing in unique_papers:
existing_title = existing.get("title", "").lower()
similarity = calculate_similarity(title, existing_title)
if similarity >= title_similarity_threshold:
# 标题相似,比较质量
if calculate_paper_quality_score(paper) > calculate_paper_quality_score(existing):
removed_papers.append(existing)
unique_papers.remove(existing)
unique_papers.append(paper)
else:
removed_papers.append(paper)
is_duplicate = True
break
if not is_duplicate:
unique_papers.append(paper)
return unique_papers, removed_papers
def print_deduplication_report(
original_count: int,
unique_papers: List[Dict],
removed_papers: List[Dict]
):
"""打印去重报告"""
print("=" * 60)
print("文献去重报告")
print("=" * 60)
print(f"\n原始文献数量: {original_count}")
print(f"去重后数量: {len(unique_papers)}")
print(f"移除重复: {len(removed_papers)}")
if original_count > 0:
print(f"去重率: {len(removed_papers) / original_count * 100:.1f}%")
if __name__ == "__main__":
# 测试示例
test_papers = [
{
"id": "E1",
"title": "Deep learning for image recognition",
"authors": ["Y LeCun"],
"year": 2015,
"journal": "Nature",
"doi": "10.1038/nature14539"
},
{
"id": "E2", # DOI重复的文献
"title": "Deep learning",
"authors": ["Y LeCun"],
"year": 2015,
"journal": "Nature",
"doi": "10.1038/nature14539"
},
{
"id": "E3", # 标题相似的文献
"title": "Deep learning for image recognition: A survey",
"authors": ["Y LeCun"],
"year": 2015,
"journal": "Nature",
"doi": ""
},
{
"id": "E4", # 完全不同的文献
"title": "Attention is all you need",
"authors": ["A Vaswani"],
"year": 2017,
"journal": "NIPS",
"doi": "10.48550/arXiv.1706.03762"
}
]
unique, removed = deduplicate_papers(test_papers)
print_deduplication_report(len(test_papers), unique, removed)
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |