Skip to content

brycewang-stanford/Auto-Empirical-Research-Skills / skills/35-bahayonghang-academic-writing-skills/skills/latex-paper-en

latex-paper-en

Verified

English LaTeX academic paper assistant for existing `.tex` projects. Use this skill whenever the user wants to compile, lint, audit, or improve an English LaTeX conference or journal paper such as IEEE, ACM, Springer, NeurIPS, or ICML submissions. Trigger even when the user only mentions one paper issue, such as bibliography errors, grammar cleanup, sentence splitting, logic review, expression polishing, translation, title optimization, figure checks, pseudocode review, algorithm block cleanup, de-AI editing, experiment-section review, table structure validation, three-line table generation, abstract structure diagnosis, or journal adaptation. Also trigger for "proofread my paper", "fix my LaTeX", "prepare for submission", "check my manuscript", "improve my writing", "three-line table", "booktabs", "check abstract", "reformat for another journal", "换投", `algorithm2e`, `algorithmicx`, `algpseudocodex`, `Require/Ensure`, or "Algorithm 1" when the user has a .tex file.

FileKindSize
SKILL.mdskill md8.5 KB
agents/openai.yamlconfig390 B
evals/evals.jsonconfig7.8 KB
examples/compile-and-bibliography.mdmarkdown551 B
examples/experiment-review.mdmarkdown527 B
examples/figures-and-title.mdmarkdown513 B
examples/grammar-and-logic.mdmarkdown687 B
examples/translation-and-deai.mdmarkdown630 B
references/ABSTRACT_STRUCTURE.mdmarkdown4.9 KB
references/BEST_PRACTICES.mdmarkdown456 B
references/CITATION_STYLES.mdmarkdown4.2 KB
references/CITATION_VERIFICATION.mdmarkdown4.1 KB
references/COMMON_ERRORS.mdmarkdown2.8 KB
references/COMPILATION.mdmarkdown2.0 KB
references/DEAI_GUIDE.mdmarkdown16.8 KB
references/FORBIDDEN_TERMS.mdmarkdown1.6 KB
references/JOURNAL_ABBREVIATIONS.mdmarkdown3.5 KB
references/JOURNAL_ADAPTATION_WORKFLOW.mdmarkdown3.6 KB
references/NUMBER_UNIT_GUIDE.mdmarkdown2.3 KB
references/REVIEWER_PERSPECTIVE.mdmarkdown4.9 KB
references/STYLE_GUIDE.mdmarkdown6.5 KB
references/TABLE_GUIDE.mdmarkdown3.7 KB
references/TERMINOLOGY.mdmarkdown9.9 KB
references/TRANSLATION_GUIDE.mdmarkdown11.4 KB
references/VENUES.mdmarkdown6.4 KB
references/WRITING_PHILOSOPHY.mdmarkdown6.7 KB
references/modules/ABSTRACT.mdmarkdown1.2 KB
references/modules/ADAPT.mdmarkdown1.3 KB
references/modules/BIBLIOGRAPHY.mdmarkdown1.3 KB
references/modules/CAPTION.mdmarkdown2.0 KB
references/modules/COMPILE.mdmarkdown2.8 KB
references/modules/DEAI.mdmarkdown1.0 KB
references/modules/EXPERIMENT.mdmarkdown3.8 KB
references/modules/EXPRESSION.mdmarkdown789 B
references/modules/FORMAT.mdmarkdown440 B
references/modules/GRAMMAR.mdmarkdown980 B
references/modules/LOGIC.mdmarkdown7.6 KB
references/modules/PSEUDOCODE.mdmarkdown1.7 KB
references/modules/SENTENCES.mdmarkdown575 B
references/modules/TABLES.mdmarkdown1.3 KB
references/modules/TITLE.mdmarkdown7.7 KB
references/modules/TRANSLATION.mdmarkdown1.1 KB
references/modules/WORKFLOW.mdmarkdown910 B
scripts/analyze_abstract.pyscript17.3 KB
scripts/analyze_experiment.pyscript18.1 KB
scripts/analyze_grammar.pyscript3.6 KB
scripts/analyze_logic.pyscript18.4 KB
scripts/analyze_sentences.pyscript3.6 KB
scripts/check_figures.pyscript6.4 KB
scripts/check_format.pyscript6.4 KB
scripts/check_pseudocode.pyscript11.2 KB
scripts/check_references.pyscript14.2 KB
scripts/check_tables.pyscript16.3 KB
scripts/compile.pyscript17.4 KB
scripts/deai_batch.pyscript9.9 KB
scripts/deai_check.pyscript16.4 KB
scripts/extract_prose.pyscript2.7 KB
scripts/generate_table.pyscript8.8 KB
scripts/improve_expression.pyscript3.2 KB
scripts/online_bib_verify.pyscript11.9 KB
scripts/optimize_title.pyscript15.4 KB
scripts/parsers.pyscript14.9 KB
scripts/translate_academic.pyscript5.1 KB
scripts/verify_bib.pyscript18.9 KB

scripts/parsers.py

View on GitHub
"""
Document Parsers for Academic Writing Skills
Support for LaTeX and Typst document parsing.
"""

import re
from abc import ABC, abstractmethod
from typing import Any


class DocumentParser(ABC):
    """Abstract base class for document parsers."""

    @abstractmethod
    def split_sections(self, content: str) -> dict[str, tuple[int, int]]:
        """
        Split document into sections.
        Returns map of {section_name: (start_line, end_line)}.
        """
        pass

    @abstractmethod
    def extract_visible_text(self, line: str) -> str:
        """
        Extract text visible to reader, preserving structure markers.
        Used for line-by-line AI trace checking.
        """
        pass

    @abstractmethod
    def clean_text(self, content: str, keep_structure: bool = False) -> str:
        """
        Extract pure prose text, removing all markup.
        Used for prose extraction and word counting.
        """
        pass

    @abstractmethod
    def get_comment_prefix(self) -> str:
        """Get the comment prefix for the language."""
        pass


class LatexParser(DocumentParser):
    """Parser for LaTeX documents."""

    # Section patterns
    SECTION_PATTERNS = {
        "abstract": r"\\begin{abstract}|\\section*?{abstract}?",
        "introduction": r"\\section*?{Introduction}|\\section*?{INTRODUCTION}",
        "related": r"\\section*?{Related\s+Work}|\\section*?{RELATED\s+WORK}",
        "method": r"\\section*?{.*(?:Method|Methodology|Approach)}",
        "experiment": r"\\section*?{.*(?:Experiment|Evaluation|Implementation)}",
        "result": r"\\section*?{.*(?:Result|Performance)}",
        "discussion": r"\\section*?{.*(?:Discussion|Analysis)}",
        "conclusion": r"\\section*?{.*(?:Conclusion|Conclusions)}",
    }

    # Preservation patterns for visible text extraction
    PRESERVE_PATTERNS = [
        r"\\cite{[^}]+}",  # Citations
        r"\\ref{[^}]+}",  # References
        r"\\label{[^}]+}",  # Labels
        r"\\eqref{[^}]+}",  # Equation references
        r"\\autoref{[^}]+}",  # Auto references
        r"\$\$[^$]*\$\$",  # Display math
        r"\$[^$]*\$",  # Inline math
        r"\\begin{equation}.*?\\end{equation}",  # Equations
        r"\\begin{align}.*?\\end{align}",  # Align environments
        r"\\begin{.*?}.*?\\end{.*?}",  # Generic environments
        r"\\includegraphics(?:\[[^]]*\])?\{[^}]+\}",  # Images
        r"\\caption{[^}]+}",  # Captions
    ]

    # Environments to skip entirely for clean text
    SKIP_ENVIRONMENTS = [
        "equation",
        "equation*",
        "align",
        "align*",
        "gather",
        "gather*",
        "multline",
        "multline*",
        "eqnarray",
        "eqnarray*",
        "displaymath",
        "figure",
        "figure*",
        "table",
        "table*",
        "tabular",
        "tabular*",
        "lstlisting",
        "verbatim",
        "minted",
        "algorithm",
        "algorithmic",
    ]

    def get_comment_prefix(self) -> str:
        return "%"

    def split_sections(self, content: str) -> dict[str, tuple[int, int]]:
        lines = content.split("\n")
        sections = {}
        current_section = "preamble"
        start_line = 0

        for i, line in enumerate(lines, 1):
            for section_name, pattern in self.SECTION_PATTERNS.items():
                if re.search(pattern, line, re.IGNORECASE):
                    if current_section != "preamble":
                        sections[current_section] = (start_line, i - 1)
                    current_section = section_name
                    start_line = i
                    break

        # Last section
        if current_section != "preamble":
            sections[current_section] = (start_line, len(lines))

        return sections

    def extract_visible_text(self, line: str) -> str:
        # Preserve structure markers logic
        preserved = []
        temp_line = line

        for pattern in self.PRESERVE_PATTERNS:
            matches = list(re.finditer(pattern, temp_line, re.DOTALL))
            for match in reversed(matches):
                preserved.append(
                    {"start": match.start(), "end": match.end(), "text": match.group()}
                )
                # Replace with placeholder of same length to keep indices valid
                placeholder = " " * (match.end() - match.start())
                temp_line = temp_line[: match.start()] + placeholder + temp_line[match.end() :]

        preserved.sort(key=lambda x: x["start"])

        visible_parts = []
        last_end = 0

        for item in preserved:
            if item["start"] > last_end:
                visible_parts.append(temp_line[last_end : item["start"]])
            last_end = item["end"]

        if last_end < len(temp_line):
            visible_parts.append(temp_line[last_end:])

        return " ".join(visible_parts).strip()

    def clean_text(self, content: str, keep_structure: bool = False) -> str:
        # Remove comments
        content = re.sub(r"(?<!\\)%.*", "", content)

        # Remove skip environments
        for env in self.SKIP_ENVIRONMENTS:
            pattern = rf"\\begin{{{env}}}.*?\\end{{{env}}}"
            content = re.sub(pattern, "", content, flags=re.DOTALL)

        # Remove inline math
        content = re.sub(r"\$[^$]+\$", "", content)
        content = re.sub(r"\\[^]]*\\]", "", content, flags=re.DOTALL)
        content = re.sub(r"\\\(.*?\\\)", "", content, flags=re.DOTALL)

        # Formatting
        if keep_structure:
            content = re.sub(r"\\section\*?{([^}]+)}", r"\n\n## \1\n\n", content)
            content = re.sub(r"\\subsection\*?{([^}]+)}", r"\n\n### \1\n\n", content)
        else:
            content = re.sub(r"\\(?:sub)*section\*?{[^}]+}", "", content)

        # Remove commands
        content = re.sub(r"\\[a-zA-Z]+\*?(?:\[[^\]]*\])?{([^}]*)}", r"\1", content)
        content = re.sub(r"\\[a-zA-Z]+\*?", "", content)
        content = re.sub(r"[{}]", "", content)

        # Cleanup whitespace
        content = re.sub(r"\n+", "\n", content)
        content = re.sub(r" +", " ", content)
        content = re.sub(r"\.(\s*\.)+", ".", content)  # Fix multiple periods

        return content.strip()


class TypstParser(DocumentParser):
    """Parser for Typst documents."""

    # Section patterns (Heading 1-3)
    # Matches: = Introduction or == Related Work
    SECTION_PATTERNS = {
        "introduction": r"^=\s+(?:Introduction|INTRODUCTION|绪论|引言)",
        "related": r"^=\s+(?:Related\s+Work|RELATED\s+WORK|相关工作|文献综述)",
        "method": r"^=\s+.*(?:Method|Methodology|Approach|方法|原理|设计)",
        "experiment": r"^=\s+.*(?:Experiment|Evaluation|Implementation|实验|实现|测试)",
        "result": r"^=\s+.*(?:Result|Performance|结果|性能)",
        "discussion": r"^=\s+.*(?:Discussion|Analysis|讨论|分析)",
        "conclusion": r"^=\s+.*(?:Conclusion|Conclusions|结论|总结与展望)",
        "abstract": r"#abstract\[",
    }

    PRESERVE_PATTERNS = [
        r"@[a-zA-Z0-9_-]+",  # Citations @key
        r"#cite\([^)]+\)",  # Function calls #cite()
        r"#figure\([^)]+\)",  # Figures
        r"#table\([^)]+\)",  # Tables
        r"\$[^$]+\$",  # Math $...$
        r"//.*",  # Line comments
        r"/\*.*?\*/",  # Block comments
        r"<[a-zA-Z0-9_-]+>",  # Labels <label>
        r"#link\([^)]+\)",  # Links
    ]

    def get_comment_prefix(self) -> str:
        return "//"

    def split_sections(self, content: str) -> dict[str, tuple[int, int]]:
        lines = content.split("\n")
        sections = {}
        current_section = "preamble"
        start_line = 0

        for i, line in enumerate(lines, 1):
            line = line.strip()
            # Ignore comments
            if line.startswith("//"):
                continue

            for section_name, pattern in self.SECTION_PATTERNS.items():
                if re.search(pattern, line, re.IGNORECASE):
                    if current_section != "preamble":
                        sections[current_section] = (start_line, i - 1)
                    current_section = section_name
                    start_line = i
                    break

        if current_section != "preamble":
            sections[current_section] = (start_line, len(lines))

        return sections

    def extract_visible_text(self, line: str) -> str:
        # Same logic as LatexParser but with Typst patterns
        temp_line = line

        # Remove comments first for Typst
        if "//" in temp_line:
            temp_line = temp_line.split("//")[0]

        preserved = []
        for pattern in self.PRESERVE_PATTERNS:
            matches = list(re.finditer(pattern, temp_line, re.DOTALL))
            for match in reversed(matches):
                preserved.append(
                    {"start": match.start(), "end": match.end(), "text": match.group()}
                )
                placeholder = " " * (match.end() - match.start())
                temp_line = temp_line[: match.start()] + placeholder + temp_line[match.end() :]

        preserved.sort(key=lambda x: x["start"])

        # Extract visible logic... (duplicated, could refactor to base)
        # But keeping separate for now to be safe
        visible_parts = []
        last_end = 0
        for item in preserved:
            if item["start"] > last_end:
                visible_parts.append(temp_line[last_end : item["start"]])
            last_end = item["end"]
        if last_end < len(temp_line):
            visible_parts.append(temp_line[last_end:])

        return " ".join(visible_parts).strip()

    def clean_text(self, content: str, keep_structure: bool = False) -> str:
        # Remove comments
        content = re.sub(r"//.*", "", content)
        content = re.sub(r"/\*.*?\*/", "", content, flags=re.DOTALL)

        # Remove math
        content = re.sub(r"\$[^$]+\$", "", content)

        # Handle headers
        if keep_structure:
            content = re.sub(r"^=+\s+(.+)$", r"\n\n## \1\n\n", content, flags=re.MULTILINE)
        else:
            content = re.sub(r"^=+\s+.+$", "", content, flags=re.MULTILINE)

        # Remove function calls #func(...) - basic support
        # This is hard with regex due to nested parenthesis, simplified here
        content = re.sub(r"#[a-zA-Z0-9_]+\([^)]*\)", "", content)
        content = re.sub(r"@[a-zA-Z0-9_-]+", "", content)
        content = re.sub(r"<[a-zA-Z0-9_-]+>", "", content)

        # Cleanup
        content = re.sub(r"\n+", "\n", content)
        content = re.sub(r" +", " ", content)
        content = re.sub(r"\.(\s*\.)+", ".", content)  # Fix multiple periods
        return content.strip()


def get_parser(file_path: Any) -> DocumentParser:
    """Factory method to get appropriate parser."""
    path_str = str(file_path).lower()
    if path_str.endswith(".typ"):
        return TypstParser()
    return LatexParser()


def _normalize_whitespace(text: str) -> str:
    return re.sub(r"\s+", " ", text).strip()


def _extract_balanced_block(content: str, start_idx: int, opener: str, closer: str) -> str:
    """Extract a balanced block body from content starting at opener index."""
    if start_idx < 0 or start_idx >= len(content) or content[start_idx] != opener:
        return ""

    depth = 0
    block_start = -1

    for i in range(start_idx, len(content)):
        char = content[i]
        if char == opener:
            depth += 1
            if depth == 1:
                block_start = i + 1
        elif char == closer:
            depth -= 1
            if depth == 0 and block_start >= 0:
                return content[block_start:i]
            if depth < 0:
                return ""
    return ""


def _strip_typst_markup(text: str) -> str:
    """Strip lightweight Typst markup for title/abstract extraction."""
    cleaned = text
    cleaned = re.sub(r"#\w+\([^)]*\)", " ", cleaned)

    # Collapse simple bracket macros repeatedly (e.g., #emph[Paper], nested forms).
    while True:
        collapsed = re.sub(r"#\w+\[([^\[\]]*)\]", r"\1", cleaned)
        if collapsed == cleaned:
            break
        cleaned = collapsed

    cleaned = re.sub(r"[\[\]]", " ", cleaned)
    return _normalize_whitespace(cleaned)


def _strip_latex_markup(text: str) -> str:
    """Strip lightweight LaTeX markup for title/abstract extraction."""
    cleaned = text
    cleaned = re.sub(r"(?<!\\)%.*", "", cleaned)
    cleaned = re.sub(r"\$[^$]*\$", " ", cleaned)
    cleaned = re.sub(r"\\[a-zA-Z]+\*?(?:\[[^\]]*\])?{([^{}]*)}", r"\1", cleaned)
    cleaned = re.sub(r"\\[a-zA-Z]+\*?", " ", cleaned)
    cleaned = re.sub(r"[{}]", " ", cleaned)
    return _normalize_whitespace(cleaned)


def extract_title(content: str) -> str:
    """Extract document title from LaTeX/Typst source content."""
    # LaTeX: \title{...}
    latex_match = re.search(r"\\title(?:\[[^\]]*\])?\{(.+?)\}", content, re.DOTALL)
    if latex_match:
        return _strip_latex_markup(latex_match.group(1))

    # Typst common: #set document(title: "...")
    typst_str = re.search(
        r"#set\s+document\s*\(\s*title\s*:\s*\"([^\"]+)\"",
        content,
        re.DOTALL,
    )
    if typst_str:
        return _normalize_whitespace(typst_str.group(1))

    # Typst bracket style: #set document(title: [ ... ])
    typst_block = re.search(r"#set\s+document\s*\(\s*title\s*:\s*\[", content, re.DOTALL)
    if typst_block:
        bracket_idx = typst_block.end() - 1
        text = _extract_balanced_block(content, bracket_idx, "[", "]")
        if text:
            return _strip_typst_markup(text)

    return ""


def extract_abstract(content: str) -> str:
    """Extract abstract text from LaTeX/Typst source content."""
    # LaTeX abstract environment
    latex_abs = re.search(r"\\begin{abstract}(.*?)\\end{abstract}", content, re.DOTALL)
    if latex_abs:
        return _strip_latex_markup(latex_abs.group(1))

    # LaTeX section-based abstract
    sec_abs = re.search(
        r"\\section\*?\{Abstract\}(.*?)(?=\\section\*?\{|\\end\{document\}|\Z)",
        content,
        re.DOTALL | re.IGNORECASE,
    )
    if sec_abs:
        return _strip_latex_markup(sec_abs.group(1))

    # Typst: #abstract[...]
    typst_abs = re.search(r"#abstract\[", content, re.DOTALL)
    if typst_abs:
        bracket_idx = typst_abs.end() - 1
        text = _extract_balanced_block(content, bracket_idx, "[", "]")
        if text:
            return _strip_typst_markup(text)

    typst_heading_abs = re.search(
        r"^=\s+摘要\s*\n(.*?)(?=^=\s+|\Z)",
        content,
        re.DOTALL | re.MULTILINE,
    )
    if typst_heading_abs:
        return _strip_typst_markup(typst_heading_abs.group(1))

    return ""


def extract_latex_citation_keys(content: str) -> set[str]:
    """
    Extract citation keys from LaTeX source.
    Supports \\cite, \\citep, \\citet, \\nocite and variants with optional arguments.
    """
    keys: set[str] = set()
    pattern = re.compile(r"\\(?:cite\w*|nocite)\*?(?:\[[^\]]*\]\s*)*\{([^}]*)\}")
    for match in pattern.finditer(content):
        raw = match.group(1)
        for key in raw.split(","):
            normalized = key.strip()
            if normalized:
                keys.add(normalized)
    return keys
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 196kUpdated Claude CodeCodexVerifiedInstall