Skip to content

carapace-sh/carapace-bin / skills/pandoc

pandoc

Verified

Use when working with pandoc — the universal document converter. Covers CLI options, input/output formats, extensions, defaults files, templates, filters (JSON, Lua, citeproc), citations/CSL, PDF generation, slide shows, EPUBs, chunked HTML, Jupyter notebooks, pandoc Markdown syntax, and the Haskell AST/reader-writer architecture. Triggers on: "pandoc", "pandoc Markdown", "pandoc template", "pandoc filter", "pandoc lua filter", "pandoc defaults", "pandoc citeproc", "pandoc PDF", "pandoc epub", "pandoc slide", "pandoc AST", "PandocMonad", "pandoc-types", "Text.Pandoc", "--from", "--to", "--standalone", "--citeproc", "--lua-filter", "--defaults", "--template", "--pdf-engine".

FileKindSize
SKILL.mdskill md7.3 KB
references/architecture.mdmarkdown11.9 KB
references/citations.mdmarkdown5.2 KB
references/cli-options.mdmarkdown9.8 KB
references/defaults.mdmarkdown5.6 KB
references/filters.mdmarkdown7.0 KB
references/formats.mdmarkdown8.6 KB
references/markdown.mdmarkdown11.0 KB
references/pdf.mdmarkdown5.5 KB
references/specialized-output.mdmarkdown8.1 KB
references/templates.mdmarkdown10.5 KB

references/filters.md

View on GitHub
# Filters

Pandoc's filter system: JSON filters, Lua filters, citeproc, the filter pipeline, and running pandoc as a Lua interpreter.

> **Source of truth**: <https://pandoc.org/MANUAL.html> and <https://pandoc.org/filters.html>. For the internal AST and Walk typeclass, see [architecture.md](architecture.md).

## Overview

Filters sit between the reader and the writer, transforming the AST:

```
INPUT --reader--> AST --filter--> AST --writer--> OUTPUT
```

Filters are applied sequentially in the order specified on the command line. Multiple filters can be chained:

```bash
pandoc --lua-filter=filter1.lua --lua-filter=filter2.lua --filter=json_filter.py input.md -o output.html
```

## Filter Types

| Filter Type | CLI Flag | Implementation | Data Flow |
|-------------|----------|-----------------|-----------|
| **JSON Filter** | `--filter PROGRAM` | External executable | AST serialized to JSON on stdin, deserialized from stdout |
| **Lua Filter** | `--lua-filter SCRIPT` | Embedded Lua interpreter (`pandoc-lua-engine`) | Direct AST manipulation, no serialization overhead |
| **Citeproc** | `--citeproc` | Built-in Haskell module | Internal AST transformation via `processCitations` |

### JSON Filters

JSON filters are external programs that read the Pandoc AST as JSON from stdin and write a transformed AST as JSON to stdout. Any programming language can be used.

```bash
pandoc --filter pandoc-citeproc input.md -o output.html
pandoc --filter ./my-filter.py input.md -o output.html
```

The program receives JSON on stdin and must output valid Pandoc JSON on stdout. The filter runs once per invocation, processing the entire document at once.

**Example Python filter** (requires `pandocfilters` package):

```python
#!/usr/bin/env python3
from pandocfilters import toJSONFilter, Str, Emph

def caps(key, value, format, meta):
    if key == 'Str':
        return Str(value.upper())

if __name__ == "__main__":
    toJSONFilter(caps)
```

### Lua Filters

Lua filters use pandoc's embedded Lua interpreter, avoiding serialization overhead. They define functions that match AST element names.

```bash
pandoc --lua-filter=behead.lua input.md -o output.html
```

**Example Lua filter** (converts headers to bold text):

```lua
function Header(el)
    return pandoc.Para{pandoc.Strong(el.content)}
end
```

Lua filters have direct access to the pandoc AST through the `pandoc` Lua module. Key functions include:

| Function | Trigger |
|----------|---------|
| `Pandoc(doc)` | Called once on the full document |
| `Header(el)` | Called on each header element |
| `Para(el)` | Called on each paragraph |
| `CodeBlock(el)` | Called on each code block |
| `Link(el)` | Called on each link |
| `Image(el)` | Called on each image |
| `Math(el)` | Called on each math element |
| `Div(el)` | Called on each Div |
| `Span(el)` | Called on each Span |
| `Table(el)` | Called on each table |
| `Cite(el)` | Called on each citation |
| `Meta(meta)` | Called on metadata |

Each function receives the AST element and returns either a modified element, a list of elements, or `nil` (no change). Functions are applied in a specific order: `Pandoc` first, then per-element traversal (bottom-up), then `Meta`.

The `pandoc` Lua module provides constructors for all AST types (e.g., `pandoc.Header`, `pandoc.Para`, `pandoc.Str`, `pandoc.Emph`) and utility functions like `pandoc.walk` for recursive traversal.

### Citeproc

The `--citeproc` flag activates the built-in citation processor. It transforms citation elements (`Cite`) into rendered citations and adds a bibliography section. See [citations.md](citations.md) for details.

```bash
pandoc --citeproc --bibliography=refs.bib --csl=chicago.csl input.md -o output.html
```

## Filter Pipeline

Internally, filters are orchestrated by `Text.Pandoc.Filter.applyFilters`. The pipeline applies each filter sequentially using a monadic `foldM` over the filter list:

```
AST₀ → filter₁ → AST₁ → filter₂ → AST₂ → ... → ASTₙ → writer
```

After all external/Lua filters run, pandoc applies **internal transforms**:

| Transform | Effect |
|-----------|--------|
| `headerShift` | Applies `--shift-heading-level-by` |
| `eastAsianLineBreakFilter` | Handles East Asian line breaks |
| `filterToBeamer` | Adjusts for Beamer slides |
| `flatten` | Flattens nested constructs |

## Filter Ordering

The order of `--filter`, `--lua-filter`, and `--citeproc` flags determines the execution order. In defaults files, the `filters` list order determines execution order.

```bash
# citeproc runs after the Lua filter
pandoc --lua-filter=normalize.lua --citeproc input.md -o output.pdf
```

## Running Pandoc as a Lua Interpreter

Pandoc can be invoked as a standalone Lua interpreter:

```bash
pandoc-lua script.lua
pandoc lua script.lua
```

In this mode, the following are available:

| Global | Description |
|--------|-------------|
| `pandoc.*` | All pandoc Lua modules (AST constructors, utils, types) |
| `re` | Regular expression library |
| `lpeg` | LPeg parsing library |
| `PANDOC_VERSION` | Pandoc version string |
| `PANDOC_STATE` | Pandoc runtime state |
| `PANDOC_API_VERSION` | API version |

This mode is useful for running standalone scripts that manipulate pandoc documents without doing a full conversion.

## Custom Readers and Writers (Lua)

### Custom Lua Readers

A Lua script defining a `Reader(string)` function can be used as a custom reader. The function receives the raw input text and must return a Pandoc AST.

```lua
function Reader(input)
    local blocks = {}
    for line in input:lines() do
        table.insert(blocks, pandoc.Para{pandoc.Str(line)})
    end
    return pandoc.Pandoc(blocks)
end
```

Custom readers are sought in the `custom` subdirectory of the data directory. The `lpeg` parsing library is available by default.

### Custom Lua Writers

A custom Lua writer is a Lua script defining rendering functions for each AST element type. There is no default template for custom writers; one must be provided via `--template`.

Custom writers define functions like `Str(s)`, `Para(s)`, `Header(level, s, attr)`, etc., each returning a string fragment.

## Security Considerations

1. **Filters can perform arbitrary file operations**: JSON filters are external programs with full system access. Lua filters have access to the file system via `io` and `os` modules.
2. **`--sandbox`** limits file system and network access for Lua filters.
3. **Use `PandocPure` monad** in Haskell library mode for full isolation (no I/O at all).
4. **Parsers can exhibit pathological performance** on untrusted input. Use `commonmark` reader for untrusted Markdown. Use timeouts.
5. **HTML output is not guaranteed safe** (XSS). Sanitize before serving.

For the full security discussion, see [architecture.md](architecture.md).

## Related References

- [architecture.md](architecture.md) — AST types, Walk typeclass, PandocMonad, filter modules
- [citations.md](citations.md) — Citeproc and citation processing
- [cli-options.md](cli-options.md) — `--filter`, `--lua-filter`, `--citeproc` flags
- [defaults.md](defaults.md) — Specifying filters in defaults files
deprecation-and-migrationaddyosmani/agent-skillsManages deprecation and migration. Use when removing old systems, APIs, or features. Use when migrating users from one implementation to another. Use when migrating a database schema in production, such as renaming or dropping a column without downtime (expand/contract). Use when deciding whether to maintain or sunset existing code.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
performance-optimizationaddyosmani/agent-skillsOptimizes application performance across frontend, backend, queries, and databases. Use when performance requirements exist, when you suspect performance regressions, when Core Web Vitals or load times need improvement, when N+1 query patterns need fixing, or when profiling reveals bottlenecks.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
agent-cost-reportthedotmack/claude-memBelievable agent cost report for any period, default the last 7 full days PT, not counting today. Measured tokens from Claude Code transcripts priced at OpenRouter list prices (ESTIMATED), measured provider spend when a sanctioned source exists, note-taker cost separate, Timing-style HTML/PDF plus report.json, line-items.csv, evidence.json.Stars: 118kUpdated Claude CodeCodexVerifiedInstall
agent-cost-reportthedotmack/claude-memBelievable agent cost report for any period, default the last 7 full days PT, not counting today. Measured tokens from Claude Code transcripts priced at OpenRouter list prices (ESTIMATED), measured provider spend when a sanctioned source exists, note-taker cost separate, Timing-style HTML/PDF plus report.json, line-items.csv, evidence.json.Stars: 118kUpdated Claude CodeCodexVerifiedInstall