Skip to content

dleerdefi/claude-code-construction / skills/spec-splitter

spec-splitter

Verified

Split a bound project manual PDF into individual spec section PDFs and extract searchable text. Triggers: 'split specs', 'break up the project manual', 'separate spec sections', 'extract spec text'. Prerequisite for /submittal-log-generator.

FileKindSize
SKILL.mdskill md14.5 KB
agents/openai.yamlconfig235 B
scripts/extract_spec_text.pyscript7.4 KB
scripts/split_spec_manual.pyscript20.6 KB

scripts/split_spec_manual.py

View on GitHub
#!/usr/bin/env python3
"""Split a bound project manual PDF into individual spec section PDFs.

Without --sections-file, finds each section's first page from its
"SECTION XX XX XX" heading in the text layer (titles from the Table of
Contents or the heading), then splits the PDF into one file per section.
A manual with no text layer, or no such headings, cannot be split this way.

With --sections-file, splits exactly the sections the file lists and reads
nothing from the pages. The file is tab-separated text:

  # manual: 02 - Specifications/Project Manual.pdf
  # pages: 1-based
  section	title	first	last
  08 71 00	DOOR HARDWARE	412	431
  09 65 13	RESILIENT BASE AND ACCESSORIES	455	460

- `# manual:` names the PDF being split (the same file as the pdf argument).
- `# pages: 1-based` states the page base: page 1 is the manual's first page,
  as AgentCM's INDEX.md and specs/ files give pages. No other base is accepted
  and nothing is converted.
- Then a header line, and one line per section: section number, title, first
  page, last page (inclusive). Gaps between sections are allowed.

The file is refused, with the reason and exit code 2, and nothing is written,
if a header is missing, a line does not have four fields, a page is not a whole
number, a section runs backwards (first > last), falls outside the manual, or
overlaps another, or a section number appears twice.

Usage:
  python split_spec_manual.py <project_manual.pdf> [--output-dir <specs/>]
  python split_spec_manual.py <project_manual.pdf> --sections-file <sections.tsv> [--output-dir <specs/>]
  python split_spec_manual.py <project_manual.pdf> --dry-run

Output:
  specs/01 10 00 - SUMMARY.pdf
  specs/03 30 00 - CAST-IN-PLACE CONCRETE.pdf
  specs/08 71 00 - DOOR HARDWARE.pdf
  ...
  specs/spec_index.yaml
"""

import argparse
import os
import re
import sys
from pathlib import Path

import fitz  # PyMuPDF


def _same_as_source(src, new, first_page):
    """True if every page of `new` renders and reads like its source page."""
    for i, page in enumerate(new):
        orig = src[first_page + i]
        if sorted(w[4] for w in orig.get_text("words")) != sorted(w[4] for w in page.get_text("words")):
            return False
        a, b = orig.get_pixmap(dpi=30), page.get_pixmap(dpi=30)
        if (a.width, a.height, a.n) != (b.width, b.height, b.n):
            return False
        if a.samples != b.samples:
            differing = sum(1 for x, y in zip(a.samples, b.samples) if abs(x - y) > 32)
            if differing > len(a.samples) * 0.001:
                return False
    return True


def save_pages(src, first_page, last_page, out_path, clean=True):
    """Write pages first_page..last_page of `src` to out_path, as small as possible.

    insert_pdf copies each page's resource dictionary, and bound sets often
    share one dictionary listing every sheet's fonts and images, so each split
    file came out nearly as large as the whole set. clean_contents(sanitize=True)
    drops resources the page never uses. It rewrites the page's content stream,
    so cleaned pages are checked against the source (low-res render and words)
    and the uncleaned pages are kept if anything differs. Saving with
    garbage=4/deflate is lossless. Returns True if the cleaned pages were kept.
    """
    def extract(do_clean):
        new = fitz.open()
        new.insert_pdf(src, from_page=first_page, to_page=last_page)
        if do_clean:
            for page in new:
                page.clean_contents(sanitize=True)
        return new

    new, cleaned = extract(clean), clean
    if clean and not _same_as_source(src, new, first_page):
        new.close()
        new, cleaned = extract(False), False
        print(f"  NOTE: kept {Path(out_path).name} uncleaned (cleaned copy did not match the source)")
    new.save(str(out_path), garbage=4, deflate=True)
    new.close()
    return cleaned


def parse_toc(pdf_path, max_pages=None):
    """Parse the Table of Contents to extract section → page mapping.

    Returns list of (section_number, section_title, page_number) sorted by page.
    Scans all pages by default — ToC may be located deep in the document
    (e.g., page 60+) when front matter precedes the project manual.
    """
    doc = fitz.open(str(pdf_path))
    entries = []

    # Strategy: In many PDFs, PyMuPDF extracts TOC entries as:
    #   "01 10 00\n"  (section number on one line)
    #   "SUMMARY\n"   (title on the next line)
    # OR as single-line: "01 10 00    SUMMARY"
    # Handle both formats.

    single_line_pattern = re.compile(
        r"(?:SECTION\s+)?(\d{2}\s+\d{2}\s+\d{2}(?:\.\d+)?)\s+([\w][\w\s,/&()\-]+)",
    )
    section_num_pattern = re.compile(
        r"^(\d{2}\s+\d{2}\s+\d{2}(?:\.\d+)?)$",
    )

    total = len(doc) if max_pages is None else min(max_pages, len(doc))
    for page_idx in range(total):
        page = doc[page_idx]
        text = page.get_text()
        lines = [l.strip() for l in text.split("\n")]

        i = 0
        while i < len(lines):
            line = lines[i]

            # Try single-line match first
            m = single_line_pattern.match(line)
            if m:
                sec = m.group(1).strip()
                title = m.group(2).strip()
                if len(title) > 3 and not title.replace(".", "").replace(" ", "").isdigit():
                    entries.append((sec, title, None))
                i += 1
                continue

            # Try two-line match: section number alone, title on next line
            m = section_num_pattern.match(line)
            if m and i + 1 < len(lines):
                sec = m.group(1).strip()
                next_line = lines[i + 1].strip()
                # Next line should be a title (alpha characters, not a number or header)
                if (next_line and len(next_line) > 3
                        and next_line[0].isalpha()
                        and not next_line.startswith("DIVISION")
                        and not next_line.startswith("VOLUME")
                        and not next_line.startswith("TABLE")
                        and not next_line.startswith("Grimm")):
                    entries.append((sec, next_line, None))
                    i += 2
                    continue

            i += 1

    doc.close()

    # Deduplicate by section number (keep first occurrence)
    seen = set()
    unique = []
    for sec, title, pg in entries:
        if sec not in seen:
            seen.add(sec)
            unique.append((sec, title, pg))

    return sorted(unique, key=lambda x: x[0])


def find_section_pages(pdf_path, toc_entries):
    """Find actual page numbers for each section by searching for section headers.

    Scans all pages for lines matching "SECTION XX XX XX" to get precise boundaries.
    Filters out TOC/index pages that list many section numbers but aren't actual content.
    """
    doc = fitz.open(str(pdf_path))
    total_pages = len(doc)

    # Strategy: find section starts using multiple methods
    # Method A: "SECTION XX XX XX" headers (standard CSI format)
    # Method B: Page headers with section numbers like "05 1200 - 1" (page 1 of section)
    # Filter out TOC/index pages that list many sections

    section_pages = {}
    sections_per_page = {}

    # Pattern A: SECTION keyword — matches spaced (03 30 00), compact (033000), and mixed (03 3000)
    section_keyword = re.compile(
        r"SECTION\s+(\d{2}\s*\d{2}\s*\d{2}(?:\.\d+)?)\b",
        re.MULTILINE,
    )
    # Pattern B: Page header with section number and page count (e.g., "05 1200 - 1")
    page_header = re.compile(
        r"^(\d{2}\s+\d{2}\s*\d{2})\s*-\s*1\s*$",
        re.MULTILINE,
    )

    for page_idx in range(total_pages):
        text = doc[page_idx].get_text()

        # Count SECTION keyword occurrences (for TOC detection)
        keyword_matches = section_keyword.findall(text)
        sections_per_page[page_idx] = len(keyword_matches)

        # Method A: SECTION keyword on non-TOC pages
        if len(keyword_matches) < 5:  # Not a TOC page
            for sec_num in keyword_matches:
                sec = sec_num.strip()
                if sec not in section_pages:
                    section_pages[sec] = page_idx

        # Method B: Page header format "XX XXXX - 1" (page 1 = section start)
        for m in page_header.finditer(text):
            sec = m.group(1).strip()
            if sec not in section_pages:
                section_pages[sec] = page_idx

    doc.close()

    # Build ordered list
    toc_dict = {sec: title for sec, title, _ in toc_entries}
    ordered = sorted(section_pages.items(), key=lambda x: x[1])

    sections = []
    for i, (sec, start_page) in enumerate(ordered):
        end_page = ordered[i + 1][1] - 1 if i + 1 < len(ordered) else total_pages - 1
        page_count = end_page - start_page + 1
        if page_count <= 0:
            continue
        sections.append({
            "number": sec,
            "title": toc_dict.get(sec, sec),
            "start_page": start_page,
            "end_page": end_page,
            "page_count": page_count,
        })

    return sections


class SectionsFileError(ValueError):
    """A sections file that cannot be split from; the message says why."""


SECTIONS_COLUMNS = ["section", "title", "first", "last"]


def read_sections_file(sections_path, pdf_path, page_count):
    """Read a sections file (format in the module docstring) into split sections.

    Returns sections with 0-based start_page/end_page, sorted by first page.
    Raises SectionsFileError naming the first problem found.
    """
    lines = Path(sections_path).read_text(encoding="utf-8-sig").splitlines()
    headers, rows = {}, []
    for n, raw in enumerate(lines, start=1):
        line = raw.rstrip("\r")
        if not line.strip():
            continue
        if line.startswith("#"):
            key, _, value = line[1:].partition(":")
            headers[key.strip().lower()] = value.strip()
            continue
        rows.append((n, line.split("\t")))

    if "pages" not in headers:
        raise SectionsFileError("the page base is missing: add a '# pages: 1-based' line")
    if headers["pages"].lower() != "1-based":
        raise SectionsFileError(f"the page base is '{headers['pages']}': only 1-based pages are accepted, and none are converted")
    manual = headers.get("manual", "")
    if not manual:
        raise SectionsFileError("the manual is not named: add a '# manual: <path to the PDF>' line")
    named, given = Path(manual), Path(pdf_path)
    same = named.resolve() == given.resolve() if named.exists() else named.name == given.name
    if not same:
        raise SectionsFileError(f"the file is for '{manual}', not '{pdf_path}'")

    if rows and [c.strip().lower() for c in rows[0][1]] == SECTIONS_COLUMNS:
        rows = rows[1:]
    if not rows:
        raise SectionsFileError("no sections are listed")

    sections, seen = [], {}
    for n, fields in rows:
        if len(fields) != 4:
            raise SectionsFileError(f"line {n} has {len(fields)} tab-separated fields, not 4 (section, title, first, last)")
        number, title, first, last = (f.strip() for f in fields)
        if not number or not title:
            raise SectionsFileError(f"line {n}: the section number and title must both be given")
        try:
            first, last = int(first), int(last)
        except ValueError:
            raise SectionsFileError(f"line {n} ({number}): pages must be whole numbers, got '{fields[2].strip()}' and '{fields[3].strip()}'")
        if first > last:
            raise SectionsFileError(f"line {n} ({number}): runs backwards, page {first} to page {last}")
        if first < 1 or last > page_count:
            raise SectionsFileError(f"line {n} ({number}): pages {first}-{last} fall outside the manual's {page_count} pages")
        if number in seen:
            raise SectionsFileError(f"line {n}: section {number} is already listed on line {seen[number]}")
        seen[number] = n
        sections.append({"number": number, "title": title, "start_page": first - 1, "end_page": last - 1,
                         "page_count": last - first + 1})

    sections.sort(key=lambda s: s["start_page"])
    for prev, cur in zip(sections, sections[1:]):
        if cur["start_page"] <= prev["end_page"]:
            raise SectionsFileError(
                f"sections {prev['number']} (pages {prev['start_page'] + 1}-{prev['end_page'] + 1}) and "
                f"{cur['number']} (pages {cur['start_page'] + 1}-{cur['end_page'] + 1}) overlap")
    return sections


def split_pdf(pdf_path, sections, output_dir, clean=True):
    """Split the PDF into individual section files."""
    doc = fitz.open(str(pdf_path))
    output_dir = Path(output_dir)
    output_dir.mkdir(parents=True, exist_ok=True)

    created_files = []

    for sec in sections:
        # Sanitize filename
        safe_title = re.sub(r'[<>:"/\\|?*]', "", sec["title"])
        safe_title = safe_title.strip()[:80]  # Limit length
        filename = f"{sec['number']} - {safe_title}.pdf"
        out_path = output_dir / filename

        # Extract pages
        save_pages(doc, sec["start_page"], sec["end_page"], out_path, clean=clean)

        created_files.append({
            "number": sec["number"],
            "title": sec["title"],
            "filename": filename,
            "pages": sec["page_count"],
        })

    doc.close()
    return created_files


def write_index(created_files, output_dir, source_pdf):
    """Write/merge a YAML index of split spec sections.

    Merges new entries into an existing index if one exists, keyed by
    section number. Supports multi-volume manuals where each volume is
    split separately — all sections end up in a single combined index.
    """
    import yaml

    index_path = Path(output_dir) / "spec_index.yaml"

    # Load existing index if present
    existing = {}
    if index_path.exists():
        try:
            with open(index_path) as f:
                existing = yaml.safe_load(f) or {}
        except (yaml.YAMLError, OSError):
            existing = {}

    # Merge sources list (track all source PDFs)
    sources = existing.get("sources", [])
    # Migrate legacy single-source format
    if not sources and "source" in existing:
        sources.append(existing["source"])
    source_str = str(source_pdf)
    if source_str not in sources:
        sources.append(source_str)

    # Merge sections by section number (new entries win on conflict)
    existing_sections = {s["number"]: s for s in existing.get("sections", [])}
    for entry in created_files:
        existing_sections[entry["number"]] = entry

    merged_sections = sorted(existing_sections.values(), key=lambda s: s["number"])

    index = {
        "sources": sources,
        "total_sections": len(merged_sections),
        "sections": merged_sections,
    }

    with open(index_path, "w") as f:
        yaml.dump(index, f, default_flow_style=False, sort_keys=False)

    return index_path


def extract_title_from_page(pdf_path, page_idx):
    """Extract the section title from the SECTION header line on a given page."""
    doc = fitz.open(str(pdf_path))
    if page_idx >= len(doc):
        doc.close()
        return ""

    text = doc[page_idx].get_text()
    doc.close()

    # Look for: SECTION XX XX XX - TITLE or SECTION XX XX XX\nTITLE
    m = re.search(
        r"SECTION\s+\d{2}\s+\d{2}\s*\d{2}(?:\.\d+)?\s*[-\u2013\u2014]\s*(.+)",
        text,
    )
    if m:
        title = m.group(1).strip()
        # Clean up: take first line only, remove page numbers
        title = title.split("\n")[0].strip()
        if len(title) > 3:
            return title

    # Try two-line format: SECTION number on one line, title on next
    lines = [l.strip() for l in text.split("\n") if l.strip()]
    for i, line in enumerate(lines):
        if re.match(r"SECTION\s+\d{2}\s+\d{2}", line) and i + 1 < len(lines):
            candidate = lines[i + 1].strip()
            if (candidate and len(candidate) > 3
                    and candidate[0].isalpha()
                    and not candidate.startswith("PART")
                    and not candidate.startswith("SECTION")):
                return candidate

    return ""


def find_sections_from_text(pdf_path):
    """Section boundaries from SECTION headings, titles from the ToC or the heading; exits if none."""
    # Step 1: Find section boundaries by scanning ALL pages (primary method)
    # This works regardless of TOC location or format
    print("\n[Step 1] Scanning for SECTION headers across all pages...")
    sections = find_section_pages(pdf_path, [])
    print(f"  Sections found: {len(sections)}")

    if not sections:
        print("  ERROR: No SECTION headers found in PDF. Cannot split. "
              "Find the boundaries another way and pass them with --sections-file.")
        sys.exit(1)

    # Step 2: Enrich with TOC titles (optional, best-effort)
    print("\n[Step 2] Looking for Table of Contents to enrich titles...")
    toc = parse_toc(pdf_path)  # scans all pages — ToC may be deep in document
    print(f"  TOC entries found: {len(toc)}")
    toc_dict = {sec: title for sec, title, _ in toc}

    enriched = 0
    for sec in sections:
        if sec["number"] in toc_dict:
            sec["title"] = toc_dict[sec["number"]]
            enriched += 1
    print(f"  Enriched from TOC: {enriched}/{len(sections)}")

    # Step 3: For sections still without titles, extract from page text
    missing_titles = [s for s in sections if not s.get("title") or s["title"] == s["number"]]
    if missing_titles:
        print(f"\n[Step 3] Extracting titles from page text for {len(missing_titles)} sections...")
        for sec in missing_titles:
            title = extract_title_from_page(pdf_path, sec["start_page"])
            if title:
                sec["title"] = title
        still_missing = sum(1 for s in sections if not s.get("title") or s["title"] == s["number"])
        print(f"  Still missing titles: {still_missing}")
    return sections


def main():
    parser = argparse.ArgumentParser(description="Split a bound project manual into individual spec section PDFs")
    parser.add_argument("pdf", help="Path to the bound project manual PDF")
    parser.add_argument("--output-dir", default=None, help="Output directory (default: specs/ next to PDF)")
    parser.add_argument("--dry-run", action="store_true", help="Show what would be split without writing files")
    parser.add_argument("--no-clean", action="store_true", help="Skip removing unused resources from each page (larger files)")
    parser.add_argument("--sections-file", default=None,
                        help="Split exactly the sections this tab-separated file lists (format in the module docstring)")
    args = parser.parse_args()

    pdf_path = Path(args.pdf)
    if not pdf_path.exists():
        print(f"ERROR: PDF not found: {pdf_path}")
        sys.exit(1)

    output_dir = args.output_dir or str(pdf_path.parent / "specs")

    print(f"Source: {pdf_path.name}")
    print(f"Output: {output_dir}")

    if args.sections_file:
        with fitz.open(str(pdf_path)) as doc:
            page_count = len(doc)
        try:
            sections = read_sections_file(args.sections_file, pdf_path, page_count)
        except (SectionsFileError, OSError) as e:
            print(f"ERROR: sections file refused, nothing written: {e}")
            sys.exit(2)
        print(f"\n[Step 1] Sections from {args.sections_file}: {len(sections)} (pages read: none)")
    else:
        sections = find_sections_from_text(pdf_path)

    # Show preview
    for sec in sections[:5]:
        title = sec.get("title", sec["number"])[:50]
        print(f"    {sec['number']} - {title} (pp. {sec['start_page']+1}-{sec['end_page']+1}, {sec['page_count']} pages)")
    if len(sections) > 5:
        print(f"    ... ({len(sections) - 5} more)")

    if args.dry_run:
        print(f"\n[Dry run] Would create {len(sections)} files in {output_dir}/")
        for sec in sections:
            title = sec.get("title", sec["number"])
            safe_title = re.sub(r'[<>:"/\\|?*]', "", title).strip()[:80]
            print(f"  {sec['number']} - {safe_title}.pdf ({sec['page_count']} pages)")
        return

    # Step 4: Split
    print(f"\n[Step 4] Splitting PDF into {len(sections)} section files...")
    created = split_pdf(pdf_path, sections, output_dir, clean=not args.no_clean)
    print(f"  Created: {len(created)} files")

    # Step 5: Write index
    print("\n[Step 5] Writing spec index...")
    try:
        index_path = write_index(created, output_dir, pdf_path)
        print(f"  Index: {index_path}")
    except ImportError:
        print("  (PyYAML not installed - index skipped)")

    # Summary
    total_pages = sum(c["pages"] for c in created)
    print(f"\n{'='*50}")
    print(f"COMPLETE: {len(created)} spec sections extracted ({total_pages} total pages)")
    print(f"Output: {output_dir}/")
    print(f"{'='*50}")


if __name__ == "__main__":
    main()
canvas-designanthropics/skillsCreate beautiful visual art in .png and .pdf documents using design philosophy. You should use this skill when the user asks to create a poster, piece of art, design, or other static piece. Create original visual designs, never copying existing artists' work to avoid copyright violations.Stars: 197kUpdated Claude CodeCodexVerifiedInstall
docxanthropics/skillsUse this skill whenever the user wants to create, read, edit, or manipulate Word documents (.docx files) or Word templates (.dotx files). Triggers include: any mention of 'Word doc', 'word document', '.docx', '.dotx', or requests to produce professional documents with formatting like tables of contents, headings, page numbers, or letterheads. Also use when extracting or reorganizing content from .docx or .dotx files, inserting or replacing images in documents, performing find-and-replace in Word files, working with tracked changes or comments, or converting content into a polished Word document. If the user asks for a 'report', 'memo', 'letter', 'template', or similar deliverable as a Word or .docx file, use this skill. Do NOT use for PDFs, spreadsheets, Google Docs, or general coding tasks unrelated to document generation.Stars: 197kUpdated Claude CodeCodexVerifiedInstall
pdfanthropics/skillsUse this skill whenever the user wants to do anything with PDF files. This includes reading or extracting text/tables from PDFs, combining or merging multiple PDFs into one, splitting PDFs apart, rotating pages, adding watermarks, creating new PDFs, filling PDF forms, encrypting/decrypting PDFs, extracting images, and OCR on scanned PDFs to make them searchable. If the user mentions a .pdf file or asks to produce one, use this skill.Stars: 197kUpdated Claude CodeCodexVerifiedInstall
pptxanthropics/skillsUse this skill any time a .pptx or .potx file is involved in any way — as input, output, or both. This includes: creating slide decks, pitch decks, or presentations; reading, parsing, or extracting text from any .pptx or .potx file (even if the extracted content will be used elsewhere, like in an email or summary); editing, modifying, or updating existing presentations; combining or splitting slide files; working with templates (.potx), layouts, speaker notes, or comments. Trigger whenever the user mentions "deck," "slides," "presentation," or references a .pptx or .potx filename, regardless of what they plan to do with the content afterward. If a .pptx or .potx file needs to be opened, created, or touched, use this skill.Stars: 197kUpdated Claude CodeCodexVerifiedInstall