yacosta738/yacosta738.github.io / .agents/skills/cloudflare
cloudflare
FlaggedComprehensive Cloudflare platform skill covering Workers, Pages, storage (KV, D1, R2), AI (Workers AI, Vectorize, Agents SDK), networking (Tunnel, Spectrum), security (WAF, DDoS), and infrastructure-as-code (Terraform, Pulumi). Use for any Cloudflare development task. Biases towards retrieval from Cloudflare docs over pre-trained knowledge.
references/pipelines/configuration.md
View on GitHub# Pipelines Configuration
## Worker Binding
```jsonc
// wrangler.jsonc
{
"pipelines": [
{ "pipeline": "<STREAM_ID>", "binding": "STREAM" }
]
}
```
Get stream ID: `npx wrangler pipelines streams list`
## Schema (Structured Streams)
```json
{
"fields": [
{ "name": "user_id", "type": "string", "required": true },
{ "name": "event_type", "type": "string", "required": true },
{ "name": "amount", "type": "float64", "required": false },
{ "name": "timestamp", "type": "timestamp", "required": true }
]
}
```
**Types:** `string`, `int32`, `int64`, `float32`, `float64`, `bool`, `timestamp`, `json`, `binary`, `list`, `struct`
## Stream Setup
```bash
# With schema
npx wrangler pipelines streams create my-stream --schema-file schema.json
# Unstructured (no validation)
npx wrangler pipelines streams create my-stream
# List/get/delete
npx wrangler pipelines streams list
npx wrangler pipelines streams get <ID>
npx wrangler pipelines streams delete <ID>
```
## Sink Configuration
**R2 Data Catalog (Iceberg):**
```bash
npx wrangler pipelines sinks create my-sink \
--type r2-data-catalog \
--bucket my-bucket --namespace default --table events \
--catalog-token $TOKEN \
--compression zstd --roll-interval 60
```
**R2 Raw (Parquet):**
```bash
npx wrangler pipelines sinks create my-sink \
--type r2 --bucket my-bucket --format parquet \
--path analytics/events \
--partitioning "year=%Y/month=%m/day=%d" \
--access-key-id $KEY --secret-access-key $SECRET
```
| Option | Values | Guidance |
|--------|--------|----------|
| `--compression` | `zstd`, `snappy`, `gzip` | `zstd` best ratio, `snappy` fastest |
| `--roll-interval` | Seconds | Low latency: 10-60, Query perf: 300 |
| `--roll-size` | MB | Larger = better compression |
## Pipeline Creation
```bash
npx wrangler pipelines create my-pipeline \
--sql "INSERT INTO my_sink SELECT * FROM my_stream WHERE event_type = 'purchase'"
```
**⚠️ Pipelines are immutable** - cannot modify SQL. Must delete/recreate.
## Credentials
| Type | Permission | Get From |
|------|------------|----------|
| Catalog token | R2 Admin Read & Write | Dashboard → R2 → API tokens |
| R2 credentials | Object Read & Write | `wrangler r2 bucket create` output |
| HTTP ingest token | Workers Pipeline Send | Dashboard → Workers → API tokens |
## Complete Example
```bash
npx wrangler r2 bucket create my-bucket
npx wrangler r2 bucket catalog enable my-bucket
npx wrangler pipelines streams create my-stream --schema-file schema.json
npx wrangler pipelines sinks create my-sink --type r2-data-catalog --bucket my-bucket ...
npx wrangler pipelines create my-pipeline --sql "INSERT INTO my_sink SELECT * FROM my_stream"
npx wrangler deploy
```
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |