Skip to content

jackson2w/claude-code-skills / caddy-cloudflare-wildcard-proxy

caddy-cloudflare-wildcard-proxy

Verified

This skill should be used when standing up Caddy as a reverse proxy for internal/homelab domains with a wildcard TLS certificate via Cloudflare DNS-01, when a Caddyfile needs one certificate to cover many internal hostnames, when routing Caddy to a backend that's deliberately bound to loopback only and reachable via an existing Tailscale Serve endpoint, when a Caddy instance itself needs to be loopback-only and a request returns an empty 200 response or `ss -tlnp` shows it listening on `*:port` despite the site address looking like `127.0.0.1:port`, or when debugging a fresh unprivileged LXC where tailscaled fails with "/dev/net/tun does not exist" or MagicDNS doesn't register into systemd-resolved. Trigger phrases include "caddy wildcard cert", "caddy dns-01 cloudflare", "caddy custom build dns plugin", "reverse proxy to tailscale serve backend", "loopback-bound service reverse proxy", "tun device does not exist unprivileged lxc", "tailscale magicdns not registering systemd-resolved", "split-horizon internal domain caddy", "caddy empty 200 response", "caddy content-length 0", "caddy site address host matcher not bind", "caddy listening on wildcard despite 127.0.0.1", "tailscale serve empty response from caddy backend".

Caddy + Cloudflare DNS-01 wildcard proxy for internal domains

Pattern for fronting several internal/homelab services with friendly hostnames (service.example.dev) under one real, trusted wildcard certificate, without any public DNS record — Cloudflare is used purely as the ACME DNS-01 challenge provider.

Custom Caddy build for DNS plugins

Stock Caddy packages (apt, official binary releases) do not include DNS provider modules — DNS-01 wildcard certs need caddy-dns/cloudflare (or the equivalent for another provider), which isn't in the default build. Rather than requiring a Go toolchain + xcaddy on the target host, fetch a prebuilt custom binary from Caddy's official build server:

https://caddyserver.com/api/download?os=linux&arch=amd64&p=github.com%2Fcaddy-dns%2Fcloudflare

Verified live: returns HTTP 200, content-disposition: attachment; filename="caddy_linux_amd64_custom". Swap p= for a different DNS provider module path, or chain multiple &p= params for more than one plugin. Use ansible.builtin.get_url (or plain curl) straight to /usr/local/bin/caddy, mode 0755 — no build step needed on the target.

get_url with force: true re-downloads on every playbook run, and the build server can return a different binary (a newer plugin, or a new Caddy release) with no version change on your side. --check always reports that task as changed. When a run is meant to change only the Caddyfile, compare caddy version with the latest release and use --start-at-task "Deploy Caddyfile", then confirm the binary's checksum didn't move. Make upgrades a deliberate run.

Upgrading a pinned custom build (done 2026-11-15, v2.11.7 → v2.11.9)

The homelab pins caddy_version + caddy_sha256 in caddy-install.yml. The build server compiles on request, so two downloads of the same version can hash differently. Don't let the playbook download:

  1. Read the release notes' Breaking changes (2.11.8 tightened config validation).
  2. On the Caddy host, download ...&version=vX.Y.Z to a side path; check caddy version, caddy list-modules | grep cloudflare, sha256sum.
  3. set -a; . <EnvironmentFile>; set +a; <new> validate --config /etc/caddy/Caddyfile --adapter caddyfile.
  4. Keep the old binary (/usr/local/bin/caddy.vOLD), install the new one, set the pin to the hash you actually installed, run the playbook (checksum matches → no download), re-run for changed=0.
  5. Verify: openssl x509 -ext subjectAltName still shows only DNS:*.<zone>; real requests to several vhosts (ssl_verify_result=0); dig @1.1.1.1 empty.
  6. Prove DNS-01 without touching production: run the new binary as a second instance with admin off, its own storage file_system <tmp>, acme_ca = Let's Encrypt staging, odd http_port/https_port, one throwaway name, tls { dns cloudflare {env.CF_API_TOKEN} }; wait for certificate obtained successfully (challenge_type dns-01), then kill it and delete the tmp dir.

Never run Caddy with --environ when a secret arrives through the environment. It prints every variable at startup into the journal: the homelab's ExecStart=caddy run --environ had written CF_API_TOKEN 37 times (and so into every PBS backup of the CT). Removed 2026-11-15; check with journalctl -u caddy -o cat | grep -c CF_API_TOKEN= (count only).

Rotating the Cloudflare DNS-01 token, with proof before the old one goes (2026-11-16)

Done for Caddy on caddy after the old token sat in the journal (from caddy run --environ).

  • Token: an Account API token with ONE policy: the single zone, DNS Write + Zone Read, 1-year expiry, no IP filter. The home WAN IP changes and the 5G failover has another, so an IP filter breaks renewals quietly. One policy avoids Cloudflare's multi-policy edit bug.

  • Prove it before Caddy uses it (all with curl --config - so the token stays off argv):

    • GET /zones?name=<zone> finds 1 zone;
    • GET /accounts/<account.id>/tokens/verify returns active with the expiry;
    • POST and then DELETE a _acme-challenge.tokentest.<zone> TXT record;
    • another zone in the account is invisible.
  • Deploy with the binary pinned, so only the env file changes and Caddy restarts. Check without printing: the sha256 of the Infisical value equals that of CF_API_TOKEN in /proc/<MainPID>/environ.

  • End-to-end proof that leaves the live Caddy alone: run a throwaway caddy run with this global block:

    • admin off
    • storage file_system <tmpdir>
    • acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
    • http_port 18080
    • https_port 18443
    • auto_https disable_redirects

    Give it one site stg-x.<zone>:18443 with tls { dns cloudflare {env.CF_API_TOKEN} }. "certificate obtained successfully" appears in about 30 s. Kill it and remove the tmpdir.

  • Identify the old token before anyone deletes it: feed its value (from wherever it leaked or was stored) to /user/tokens/verify and to the account verify. That tells you whether it's a user token (My Profile) or an account token, plus its id. After Will deletes it, the same call must return "Invalid API Token". If it's still active, the wrong row was deleted.

One wildcard cert covering many hostnames

Put every internal hostname in a single site block keyed on the apex + wildcard, and route by Host header inside it with @matcher/handle pairs — not one site block per hostname (that issues a separate cert per name):

{
    email you@example.com
}

example.dev, *.example.dev {
    tls {
        dns cloudflare {env.CF_API_TOKEN}
    }

    @service1 host service1.example.dev
    handle @service1 {
        reverse_proxy 192.168.1.10:8080
    }

    @service2 host service2.example.dev
    handle @service2 {
        reverse_proxy https://192.168.1.20:8443 {
            transport http {
                tls_insecure_skip_verify
            }
        }
    }

    handle {
        respond "Not found" 404
    }
}

Confirm it actually issued one cert, not N: openssl x509 -noout -ext subjectAltName against any hostname should show a single DNS:*.example.dev SAN. New internal hostnames later are a @matcher/handle pair plus one DNS record — zero new cert work, since the wildcard already covers them.

tls_insecure_skip_verify on a specific backend's transport http block is the right tool when that backend serves a self-signed cert (e.g. Proxmox VE/PBS web UIs on 8006/8007) — the Caddy edge still presents the real trusted cert to clients, so this is a contained backend-leg tradeoff, not a user-facing warning. Confirm which backends actually need it by checking with ss -tlnp on the real host — don't assume; some are already directly reachable, others aren't.

Routing to a backend that's deliberately loopback-bound

A common pattern in a Tailscale-first homelab: a service binds 127.0.0.1 only on its own host, reachable exclusively through an existing tailscale serve mapping — done deliberately to minimize exposure. Don't widen that bind address just to make it reachable from the new Caddy host; that changes an already-deployed service's security posture for no real gain.

Instead:

  1. Join the Caddy host to the tailnet as a plain client (tailscale up --hostname=<name>, no subnet routes, no exit node — needs one-time interactive browser auth at the URL it prints).
  2. Point reverse_proxy at the backend's existing Serve URL instead of its raw port:
    reverse_proxy https://backend.tailnet-name.ts.net {
        header_up Host backend.tailnet-name.ts.net
    }
    
  3. Verify the backend's actual Serve config first with tailscale serve status on that host — don't assume a port or path; read the real mapping.

The header_up Host override matters: Caddy forwards the original inbound Host header (service.example.dev) by default, but some backends validate Host against an allowlist (e.g. a Next.js app's ALLOWED_HOSTS-style env var checking for the tailnet hostname) and will reject anything else. Override it to whatever the backend actually expects. Tailscale-issued certs are already publicly trusted (Let's Encrypt via Tailscale), so no tls_insecure_skip_verify needed for this path.

The Host override breaks CSRF origin checks. The browser still sends Origin: https://service.example.dev, which no longer matches the Host the backend sees. Grafana rejects every logged-in write with 403 origin not allowed; fix it with csrf_trusted_origins = service.example.dev under [security] before the switch. A plain login test won't show this, because Grafana skips the check when there is no session cookie. Test with a dummy one: curl -X POST -H 'Origin: https://service.example.dev' -H 'Cookie: grafana_session=x' https://backend.ts.net/api/dashboards/db should give 401, not 403, and a foreign Origin should still give 403 (found 2026-10-25 moving Grafana off a plain-HTTP LAN upstream).

Split-horizon internal DNS

No public A/CNAME record needs to exist. Cloudflare's role is solely the DNS-01 challenge — a token scoped to Zone:DNS:Edit on the one zone is enough; Caddy creates/deletes its own _acme-challenge TXT records automatically. Resolve the hostnames internally instead (e.g. Pi-hole Local DNS Records, dns.hosts) pointing each name at the Caddy host's LAN IP.

Pi-hole's TOML-based Local DNS Records do not support wildcard/glob entries — each new subdomain needs one explicit line. The wildcard/automatic part is the certificate, not the DNS record.

Verify split-horizon actually holds by querying a public resolver directly and confirming it returns nothing: dig +short <hostname> @1.1.1.1.

A loopback-only site address is a Host matcher, not a socket bind restriction

When a small app-fronting Caddy instance itself needs to be loopback-only (e.g. it sits behind Tailscale Serve on the same host, matching the "loopback-bound backend" pattern above but for Caddy itself, not the app behind it), writing the site address as http://127.0.0.1:8000 { } does not restrict the actual socket bind — Caddy still listens on *:8000 (verify with ss -tlnp), and 127.0.0.1 in the site address instead becomes a Host-header matcher. Confirmed live 2026-08-21: a request whose Host header didn't literally match fell through to an empty automatic-HTTPS redirect handler — 200 OK, content-length: 0, no error anywhere — which broke both a direct curl to the Tailscale IP and, more surprisingly, Tailscale Serve's own reverse-proxy hop (it doesn't send a Host header matching 127.0.0.1:8000/localhost:8000 either). This masquerades as a Tailscale Serve or DNS problem; it's neither.

Fix: use a bare port as the site address plus an explicit bind directive, which actually restricts the socket:

:8000 {
    bind 127.0.0.1
    reverse_proxy unix//run/php/php-app.sock
}

Verify both directions after the fix, not just the happy path: ss -tlnp shows 127.0.0.1:8000 (never *:8000 or 0.0.0.0:8000), a direct connection to the host's Tailscale IP on that port is refused (curl -m5 http://<tailscale-ip>:8000/ should time out or connection-refuse), and the real Tailscale Serve URL still works.

Fresh unprivileged LXC + Tailscale gotchas

  • TUN passthrough: tailscaled needs /dev/net/tun explicitly passed through on an unprivileged LXC. Append to the host-side /etc/pve/lxc/<vmid>.conf (or equivalent for other LXC hypervisors):

    lxc.cgroup2.devices.allow: c 10:200 rwm
    lxc.mount.entry: /dev/net/tun dev/net/tun none bind,create=file
    

    Then a full stop/start cycle of the container itself — restarting tailscaled from inside isn't enough. Symptom without the fix: journalctl -u tailscaled shows CreateTUN("tailscale0") failed; /dev/net/tun does not exist.

  • MagicDNS not registering into systemd-resolved on first pass: after tailscale up + tailscale set --accept-dns=true + a delayed tailscaled restart (common in a DNS-routing playbook that avoids severing the SSH session mid-run), resolvectl status can still show the tailscale0 link with no DNS scope at all — just LLMNR, no *.ts.net split. A direct, synchronous systemctl restart tailscaled right after (safe once the playbook run itself has finished) fixes it. Verify with resolvectl status tailscale0: correct output shows Current Scopes: DNS, Current DNS Server: 100.100.100.100, and DNS Domain: <tailnet-name>.ts.net ~. — compare against a known-good host if unsure.

  • Locale: fresh Debian LXC templates have no locale generated, which breaks Ansible (could not initialize the preferred locale) before any of the above even runs — generate en_US.UTF-8 first.

reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 466kUpdated Claude CodeCodexVerifiedInstall
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 466kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 319kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 319kUpdated Claude CodeCodexVerifiedInstall