Skip to content

pugyy/dw-harness / plugins/dw-harness/skills/dw-etl

dw-etl

Verified

Step 3/8 数仓 ETL 开发:生成 MaxCompute 建表 DDL、INSERT SQL, 并由 hook 做写后规范反馈。适用于契约定稿后开始写表和任务 SQL。

Step 3: ETL 开发

执行流程

  1. 按契约生成 ddl_[表名].sql(字段/类型/注释与契约一致)
  2. 生成 insert_[表名].sql(OVERWRITE + 显式 PARTITION)
  3. 写文件后 PostToolUse hook 自动给出规范反馈(写后反馈,非拦截)
  4. 自查一把确定性校验:
dw lint ddl_[表名].sql insert_[表名].sql

模板

INSERT OVERWRITE TABLE [表] PARTITION (partition_dt = '[yyyyMMdd]')
SELECT [显式字段列表]
FROM [上游]
WHERE partition_dt = '[yyyyMMdd]';

强制规范(hook 复核)

  • 禁 SELECT * / t.*(COUNT(*) 例外)
  • INSERT 必须带 PARTITION;金额 DECIMAL(20,4) 禁 DOUBLE
  • 注释三段式:业务含义 | 数据来源 | 计算逻辑
  • 金额单位继承契约声明,不做隐式换算

交付物

ddl_[表名].sql、insert_[表名].sql(如需同步:ddl_sr_[表名].sql 留给 Step 6)

deprecation-and-migrationaddyosmani/agent-skillsManages deprecation and migration. Use when removing old systems, APIs, or features. Use when migrating users from one implementation to another. Use when migrating a database schema in production, such as renaming or dropping a column without downtime (expand/contract). Use when deciding whether to maintain or sunset existing code.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
performance-optimizationaddyosmani/agent-skillsOptimizes application performance across frontend, backend, queries, and databases. Use when performance requirements exist, when you suspect performance regressions, when Core Web Vitals or load times need improvement, when N+1 query patterns need fixing, or when profiling reveals bottlenecks.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
agent-cost-reportthedotmack/claude-memBelievable agent cost report for any period, default the last 7 full days PT, not counting today. Measured tokens from Claude Code transcripts priced at OpenRouter list prices (ESTIMATED), measured provider spend when a sanctioned source exists, note-taker cost separate, Timing-style HTML/PDF plus report.json, line-items.csv, evidence.json.Stars: 118kUpdated Claude CodeCodexVerifiedInstall
agent-cost-reportthedotmack/claude-memBelievable agent cost report for any period, default the last 7 full days PT, not counting today. Measured tokens from Claude Code transcripts priced at OpenRouter list prices (ESTIMATED), measured provider spend when a sanctioned source exists, note-taker cost separate, Timing-style HTML/PDF plus report.json, line-items.csv, evidence.json.Stars: 118kUpdated Claude CodeCodexVerifiedInstall