Skip to content

yacosta738/yacosta738.github.io / .agents/skills/cloudflare

cloudflare

Flagged

Comprehensive Cloudflare platform skill covering Workers, Pages, storage (KV, D1, R2), AI (Workers AI, Vectorize, Agents SDK), networking (Tunnel, Spectrum), security (WAF, DDoS), and infrastructure-as-code (Terraform, Pulumi). Use for any Cloudflare development task. Biases towards retrieval from Cloudflare docs over pre-trained knowledge.

FileKindSize
SKILL.mdskill md8.0 KB
references/agents-sdk/README.mdmarkdown2.7 KB
references/agents-sdk/api.mdmarkdown5.7 KB
references/agents-sdk/configuration.mdmarkdown3.5 KB
references/agents-sdk/gotchas.mdmarkdown5.0 KB
references/agents-sdk/patterns.mdmarkdown5.5 KB
references/ai-gateway/README.mdmarkdown6.2 KB
references/ai-gateway/configuration.mdmarkdown2.9 KB
references/ai-gateway/dynamic-routing.mdmarkdown1.8 KB
references/ai-gateway/features.mdmarkdown2.7 KB
references/ai-gateway/sdk-integration.mdmarkdown2.6 KB
references/ai-gateway/troubleshooting.mdmarkdown2.4 KB
references/ai-search/README.mdmarkdown4.4 KB
references/ai-search/api.mdmarkdown2.5 KB
references/ai-search/configuration.mdmarkdown1.7 KB
references/ai-search/gotchas.mdmarkdown2.1 KB
references/ai-search/patterns.mdmarkdown2.0 KB
references/analytics-engine/README.mdmarkdown3.3 KB
references/analytics-engine/api.mdmarkdown3.1 KB
references/analytics-engine/configuration.mdmarkdown2.3 KB
references/analytics-engine/gotchas.mdmarkdown2.0 KB
references/analytics-engine/patterns.mdmarkdown2.3 KB
references/api-shield/README.mdmarkdown1.8 KB
references/api-shield/api.mdmarkdown3.8 KB
references/api-shield/configuration.mdmarkdown4.9 KB
references/api-shield/gotchas.mdmarkdown4.8 KB
references/api-shield/patterns.mdmarkdown5.7 KB
references/api/README.mdmarkdown2.4 KB
references/api/api.mdmarkdown4.2 KB
references/api/configuration.mdmarkdown3.7 KB
references/api/gotchas.mdmarkdown5.3 KB
references/api/patterns.mdmarkdown4.5 KB
references/argo-smart-routing/README.mdmarkdown4.0 KB
references/argo-smart-routing/api.mdmarkdown6.4 KB
references/argo-smart-routing/configuration.mdmarkdown4.4 KB
references/argo-smart-routing/gotchas.mdmarkdown4.2 KB
references/argo-smart-routing/patterns.mdmarkdown3.0 KB
references/bindings/README.mdmarkdown3.9 KB
references/bindings/api.mdmarkdown5.1 KB
references/bindings/configuration.mdmarkdown4.4 KB
references/bindings/gotchas.mdmarkdown6.0 KB
references/bindings/patterns.mdmarkdown4.9 KB
references/bot-management/README.mdmarkdown3.7 KB
references/bot-management/api.mdmarkdown5.7 KB
references/bot-management/configuration.mdmarkdown5.6 KB
references/bot-management/gotchas.mdmarkdown5.5 KB
references/bot-management/patterns.mdmarkdown4.9 KB
references/browser-rendering/README.mdmarkdown3.0 KB
references/browser-rendering/api.mdmarkdown2.9 KB
references/browser-rendering/configuration.mdmarkdown1.8 KB
references/browser-rendering/gotchas.mdmarkdown2.5 KB
references/browser-rendering/patterns.mdmarkdown2.6 KB
references/c3/README.mdmarkdown3.3 KB
references/c3/api.mdmarkdown2.2 KB
references/c3/configuration.mdmarkdown1.6 KB
references/c3/gotchas.mdmarkdown2.3 KB
references/c3/patterns.mdmarkdown2.2 KB
references/cache-reserve/README.mdmarkdown5.5 KB
references/cache-reserve/api.mdmarkdown5.9 KB
references/cache-reserve/configuration.mdmarkdown3.6 KB
references/cache-reserve/gotchas.mdmarkdown5.7 KB
references/cache-reserve/patterns.mdmarkdown6.1 KB
references/containers/README.mdmarkdown3.4 KB
references/containers/api.mdmarkdown4.2 KB
references/containers/configuration.mdmarkdown5.3 KB
references/containers/gotchas.mdmarkdown4.4 KB
references/containers/patterns.mdmarkdown4.8 KB
references/cron-triggers/README.mdmarkdown2.9 KB
references/cron-triggers/api.mdmarkdown5.7 KB
references/cron-triggers/configuration.mdmarkdown4.2 KB
references/cron-triggers/gotchas.mdmarkdown6.2 KB
references/cron-triggers/patterns.mdmarkdown6.8 KB
references/d1/README.mdmarkdown4.6 KB
references/d1/api.mdmarkdown6.6 KB
references/d1/configuration.mdmarkdown5.5 KB
references/d1/gotchas.mdmarkdown4.2 KB
references/d1/patterns.mdmarkdown7.1 KB
references/ddos/README.mdmarkdown2.0 KB
references/ddos/api.mdmarkdown4.1 KB
references/ddos/configuration.mdmarkdown2.9 KB
references/ddos/gotchas.mdmarkdown3.8 KB
references/ddos/patterns.mdmarkdown5.5 KB
references/do-storage/README.mdmarkdown2.7 KB
references/do-storage/api.mdmarkdown3.5 KB
references/do-storage/configuration.mdmarkdown2.2 KB
references/do-storage/gotchas.mdmarkdown5.1 KB
references/do-storage/patterns.mdmarkdown6.0 KB
references/do-storage/testing.mdmarkdown4.8 KB
references/durable-objects/README.mdmarkdown6.5 KB
references/durable-objects/api.mdmarkdown6.0 KB
references/durable-objects/configuration.mdmarkdown4.2 KB
references/durable-objects/gotchas.mdmarkdown7.1 KB
references/durable-objects/patterns.mdmarkdown6.5 KB
references/email-routing/README.mdmarkdown3.4 KB
references/email-routing/api.mdmarkdown5.2 KB
references/email-routing/configuration.mdmarkdown3.5 KB
references/email-routing/gotchas.mdmarkdown4.0 KB
references/email-routing/patterns.mdmarkdown5.4 KB
references/email-workers/README.mdmarkdown5.3 KB
references/email-workers/api.mdmarkdown5.9 KB
references/email-workers/configuration.mdmarkdown2.7 KB
references/email-workers/gotchas.mdmarkdown3.0 KB
references/email-workers/patterns.mdmarkdown2.7 KB
references/graphql-api/README.mdmarkdown7.6 KB
references/graphql-api/api.mdmarkdown5.6 KB
references/graphql-api/configuration.mdmarkdown4.6 KB
references/graphql-api/gotchas.mdmarkdown4.7 KB
references/graphql-api/patterns.mdmarkdown6.4 KB
references/hyperdrive/README.mdmarkdown2.8 KB
references/hyperdrive/api.mdmarkdown3.8 KB
references/hyperdrive/configuration.mdmarkdown3.8 KB
references/hyperdrive/gotchas.mdmarkdown3.8 KB
references/hyperdrive/patterns.mdmarkdown5.8 KB
references/images/README.mdmarkdown2.8 KB
references/images/api.mdmarkdown2.5 KB
references/images/configuration.mdmarkdown4.0 KB
references/images/gotchas.mdmarkdown2.4 KB
references/images/patterns.mdmarkdown3.5 KB
references/kv/README.mdmarkdown2.8 KB
references/kv/api.mdmarkdown4.2 KB
references/kv/configuration.mdmarkdown3.0 KB
references/kv/gotchas.mdmarkdown4.3 KB
references/kv/patterns.mdmarkdown4.6 KB
references/miniflare/README.mdmarkdown2.9 KB
references/miniflare/api.mdmarkdown4.5 KB
references/miniflare/configuration.mdmarkdown3.4 KB
references/miniflare/gotchas.mdmarkdown3.6 KB
references/miniflare/patterns.mdmarkdown4.2 KB
references/network-interconnect/README.mdmarkdown3.2 KB
references/network-interconnect/api.mdmarkdown5.5 KB
references/network-interconnect/configuration.mdmarkdown4.2 KB
references/network-interconnect/gotchas.mdmarkdown5.4 KB
references/network-interconnect/patterns.mdmarkdown3.9 KB
references/observability/README.mdmarkdown3.9 KB
references/observability/api.mdmarkdown3.4 KB
references/observability/configuration.mdmarkdown3.5 KB
references/observability/gotchas.mdmarkdown3.7 KB
references/observability/patterns.mdmarkdown2.4 KB
references/pages-functions/README.mdmarkdown3.2 KB
references/pages-functions/api.mdmarkdown4.6 KB
references/pages-functions/configuration.mdmarkdown2.9 KB
references/pages-functions/gotchas.mdmarkdown3.8 KB
references/pages-functions/patterns.mdmarkdown4.1 KB
references/pages/README.mdmarkdown2.8 KB
references/pages/api.mdmarkdown5.8 KB
references/pages/configuration.mdmarkdown5.7 KB
references/pages/gotchas.mdmarkdown7.9 KB
references/pages/patterns.mdmarkdown5.9 KB
references/pipelines/README.mdmarkdown3.3 KB
references/pipelines/api.mdmarkdown5.0 KB
references/pipelines/configuration.mdmarkdown2.7 KB
references/pipelines/gotchas.mdmarkdown2.4 KB
references/pipelines/patterns.mdmarkdown2.0 KB
references/pulumi/README.mdmarkdown3.2 KB
references/pulumi/api.mdmarkdown5.6 KB
references/pulumi/configuration.mdmarkdown5.5 KB
references/pulumi/gotchas.mdmarkdown6.4 KB
references/pulumi/patterns.mdmarkdown6.8 KB
references/queues/README.mdmarkdown3.0 KB
references/queues/api.mdmarkdown5.4 KB
references/queues/configuration.mdmarkdown3.7 KB
references/queues/gotchas.mdmarkdown7.1 KB
references/queues/patterns.mdmarkdown5.4 KB
references/r2-data-catalog/README.mdmarkdown6.9 KB
references/r2-data-catalog/api.mdmarkdown5.9 KB
references/r2-data-catalog/configuration.mdmarkdown5.2 KB
references/r2-data-catalog/gotchas.mdmarkdown6.0 KB
references/r2-data-catalog/patterns.mdmarkdown5.7 KB
references/r2-sql/README.mdmarkdown5.1 KB
references/r2-sql/SKILL.md.backupother12.8 KB
references/r2-sql/api.mdmarkdown3.9 KB
references/r2-sql/configuration.mdmarkdown3.7 KB
references/r2-sql/gotchas.mdmarkdown6.1 KB
references/r2-sql/patterns.mdmarkdown6.6 KB
references/r2/README.mdmarkdown2.8 KB
references/r2/api.mdmarkdown5.5 KB
references/r2/configuration.mdmarkdown3.6 KB
references/r2/gotchas.mdmarkdown4.9 KB
references/r2/patterns.mdmarkdown5.7 KB
references/realtime-sfu/README.mdmarkdown3.0 KB
references/realtime-sfu/api.mdmarkdown3.7 KB
references/realtime-sfu/configuration.mdmarkdown3.4 KB
references/realtime-sfu/gotchas.mdmarkdown4.8 KB
references/realtime-sfu/patterns.mdmarkdown5.3 KB
references/realtimekit/README.mdmarkdown4.0 KB
references/realtimekit/api.mdmarkdown8.2 KB
references/realtimekit/configuration.mdmarkdown5.1 KB
references/realtimekit/gotchas.mdmarkdown6.7 KB
references/realtimekit/patterns.mdmarkdown7.5 KB
references/sandbox/README.mdmarkdown3.0 KB
references/sandbox/api.mdmarkdown4.9 KB
references/sandbox/configuration.mdmarkdown3.3 KB
references/sandbox/gotchas.mdmarkdown5.3 KB
references/sandbox/patterns.mdmarkdown5.1 KB
references/secrets-store/README.mdmarkdown2.2 KB
references/secrets-store/api.mdmarkdown4.3 KB
references/secrets-store/configuration.mdmarkdown3.8 KB
references/secrets-store/gotchas.mdmarkdown3.7 KB
references/secrets-store/patterns.mdmarkdown5.5 KB
references/smart-placement/README.mdmarkdown4.9 KB

references/r2-data-catalog/README.md

View on GitHub
# Cloudflare R2 Data Catalog Skill Reference

Expert guidance for Cloudflare R2 Data Catalog - Apache Iceberg catalog built into R2 buckets.

## Reading Order

**New to R2 Data Catalog?** Start here:
1. Read "What is R2 Data Catalog?" and "When to Use" below
2. [configuration.md](configuration.md) - Enable catalog, create tokens
3. [patterns.md](patterns.md) - PyIceberg setup and common patterns
4. [api.md](api.md) - REST API reference as needed
5. [gotchas.md](gotchas.md) - Troubleshooting when issues arise

**Quick reference?** Jump to:
- [Enable catalog on bucket](configuration.md#enable-catalog-on-bucket)
- [PyIceberg connection pattern](patterns.md#pyiceberg-connection-pattern)
- [Permission errors](gotchas.md#permission-errors)

## What is R2 Data Catalog?

R2 Data Catalog is a **managed Apache Iceberg REST catalog** built directly into R2 buckets. It provides:

- **Apache Iceberg tables** - ACID transactions, schema evolution, time-travel queries
- **Zero-egress costs** - Query from any cloud/region without data transfer fees
- **Standard REST API** - Works with Spark, PyIceberg, Snowflake, Trino, DuckDB
- **No infrastructure** - Fully managed, no catalog servers to run
- **Public beta** - Available to all R2 subscribers, no extra cost beyond R2 storage

### What is Apache Iceberg?

Open table format for analytics datasets in object storage. Features:
- **ACID transactions** - Safe concurrent reads/writes
- **Metadata optimization** - Fast queries without full scans
- **Schema evolution** - Add/rename/delete columns without rewrites
- **Time-travel** - Query historical snapshots
- **Partitioning** - Organize data for efficient queries

## When to Use

**Use R2 Data Catalog for:**
- **Log analytics** - Store and query application/system logs
- **Data lakes/warehouses** - Analytical datasets queried by multiple engines
- **BI pipelines** - Aggregate data for dashboards and reports
- **Multi-cloud analytics** - Share data across clouds without egress fees
- **Time-series data** - Event streams, metrics, sensor data

**Don't use for:**
- **Transactional workloads** - Use D1 or external database instead
- **Sub-second latency** - Iceberg optimized for batch/analytical queries
- **Small datasets (<1GB)** - Setup overhead not worth it
- **Unstructured data** - Store files directly in R2, not as Iceberg tables

## Architecture

```
┌─────────────────────────────────────────────────┐
│  Query Engines                                  │
│  (PyIceberg, Spark, Trino, Snowflake, DuckDB)  │
└────────────────┬────────────────────────────────┘
                 │
                 │ REST API (OAuth2 token)
                 ▼
┌─────────────────────────────────────────────────┐
│  R2 Data Catalog (Managed Iceberg REST Catalog)│
│  • Namespace/table metadata                     │
│  • Transaction coordination                     │
│  • Snapshot management                          │
└────────────────┬────────────────────────────────┘
                 │
                 │ Vended credentials
                 ▼
┌─────────────────────────────────────────────────┐
│  R2 Bucket Storage                              │
│  • Parquet data files                           │
│  • Metadata files                               │
│  • Manifest files                               │
└─────────────────────────────────────────────────┘
```

**Key concepts:**
- **Catalog URI** - REST endpoint for catalog operations (e.g., `https://<account-id>.r2.cloudflarestorage.com/iceberg/<bucket>`)
- **Warehouse** - Logical grouping of tables (typically same as bucket name)
- **Namespace** - Schema/database containing tables (e.g., `logs`, `analytics`)
- **Table** - Iceberg table with schema, data files, snapshots
- **Vended credentials** - Temporary S3 credentials catalog provides for data access

## Limits

| Resource | Limit | Notes |
|----------|-------|-------|
| Namespaces per catalog | No hard limit | Organize tables logically |
| Tables per namespace | <10,000 recommended | Performance degrades beyond this |
| Files per table | <100,000 recommended | Run compaction regularly |
| Snapshots per table | Configurable retention | Expire >7 days old |
| Partitions per table | 100-1,000 optimal | Too many = slow metadata ops |
| Table size | Same as R2 bucket | 10GB-10TB+ common |
| API rate limits | Standard R2 API limits | Shared with R2 storage operations |
| Target file size | 128-512 MB | After compaction |

## Current Status

**Public Beta** (as of Jan 2026)
- Available to all R2 subscribers
- No extra cost beyond standard R2 storage/operations
- Production-ready, but breaking changes possible
- Supports: namespaces, tables, snapshots, compaction, time-travel, table maintenance

## Decision Tree: Is R2 Data Catalog Right For You?

```
Start → Need analytics on object storage data?
         │
         ├─ No → Use R2 directly for object storage
         │
         └─ Yes → Dataset >1GB with structured schema?
                  │
                  ├─ No → Too small, use R2 + ad-hoc queries
                  │
                  └─ Yes → Need ACID transactions or schema evolution?
                           │
                           ├─ No → Consider simpler solutions (Parquet on R2)
                           │
                           └─ Yes → Need multi-cloud/multi-tool access?
                                    │
                                    ├─ No → D1 or external DB may be simpler
                                    │
                                    └─ Yes → ✅ Use R2 Data Catalog
```

**Quick check:** If you answer "yes" to all:
- Dataset >1GB and growing
- Structured/tabular data (logs, events, metrics)
- Multiple query tools or cloud environments
- Need versioning, schema changes, or concurrent access

→ R2 Data Catalog is a good fit.

## In This Reference

- **[configuration.md](configuration.md)** - Enable catalog, create API tokens, connect clients
- **[api.md](api.md)** - REST endpoints, operations, maintenance
- **[patterns.md](patterns.md)** - PyIceberg examples, common use cases
- **[gotchas.md](gotchas.md)** - Troubleshooting, best practices, limitations

## See Also

- [Cloudflare R2 Data Catalog Docs](https://developers.cloudflare.com/r2/data-catalog/)
- [Apache Iceberg Docs](https://iceberg.apache.org/)
- [PyIceberg Docs](https://py.iceberg.apache.org/)
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall