Skip to content

cianfhoghlaim/cianchosaint / .agents/skills/dagster-asset-sync

dagster-asset-sync

Verified

Layer 6 of the knowledge-sync-loop: validate the ~833 Dagster assets in the 5-layer defs/ tree + ingest them into the dagster_assets Cognee cluster. Use when the user asks 'how do I validate Dagster assets', 'how do I keep the Dagster asset graph in sync', 'what does sync:dagster do', 'how do I write a Dagster asset health check', 'how do I find broken assets', or 'where is the Dagster asset dashboard'. Per the 2026-08-15-retroactive-pre-v7-cleanup-v1 change. Triggers: 'sync:dagster', 'sync_dagster_assets_to_cognee.py', 'dagster_sync_health', 'notebook 25', 'stedding/sync-reports/dagster-{date}.md', 'orchestration/defs/2_materials/...'.

Dagster Asset Sync (Layer 6)

The Layer 6 of the knowledge-sync-loop architecture. Validates the 5-layer Dagster asset graph + ingests it into the dagster_assets Cognee cluster + emits metadata via the dagster_sync_health Dagster asset.

Why Layer 6 exists

The 5-layer pull-based sync architecture (from 2026-08-15-knowledge-sync-loop-v1) covered:

LayerWhat it syncs
1. pathspre-v7 path drift in source files
2. cccthe codebase semantic index
3. cogneeopenspec changes + specs + skills
4. skillsagent SKILL.md metadata + references
5. mcp14 MCP server health

The biggest gap: the Dagster asset graph (~833 assets across the 5-layer orchestration/defs/ tree) wasn't validated by any of the 5 layers. The 2026-08-15-retroactive-pre-v7-cleanup-v1 change closes this gap by adding Layer 6.

What sync:dagster does

mise run sync:dagster
# → bash scripts/sync/dagster.sh
# → stedding/sync-reports/dagster-{date}.md

The script:

  1. Walks the 5-layer orchestration/defs/ tree: 1_ingestion/, 2_materials/, 3_model_lifecycle/, 4_asset_generation/, 5_agent_ops/
  2. Counts @asset, @asset_check, @sensor decorators per layer
  3. Counts YAML defs (the Cianfhoghlaim Component YAML files)
  4. Counts the Cianfhoghlaim Component classes at orchestration/components/
  5. Writes a per-group report to stedding/sync-reports/dagster-{date}.md
  6. Returns 0 if the total asset count is > 0 (suspicious if 0)

The new Dagster asset: dagster_sync_health

Lives at orchestration/defs/sync_assets.py. Emits:

MetadataDescription
asset_countTotal @asset decorators across the 5 layers
sensor_countTotal @sensor decorators across the 5 layers
group_count5 (the canonical 5-layer convention)
broken_asset_count0 if healthy (placeholder for future expansion)

Plus a dagster_sync_alert companion asset that fires when broken_asset_count > 0 + a dagster_assets_sensor that fires on every orchestration/defs/ file change.

The new Cognee cluster: dagster_assets

Ingested by scripts/sync_dagster_assets_to_cognee.py. The script:

  1. Walks orchestration/defs/ + parses each .py file via ast
  2. Extracts @asset, @asset_check, @sensor decorator usage
  3. Renders a per-layer markdown summary
  4. Adds the summary to the dagster_assets Cognee cluster

After sync, the Cognee graph has 8 typed clusters (was 7):

  • cognee_docs, openspec_changes, openspec_specs, agent_skills
  • (5 existing) + dagster_assets (new)

The consolidated marimo dashboard: sync_health.py

The Dagster tab in notebooks/sync_health.py reads the latest stedding/sync-reports/dagster-{date}.md and surfaces:

  • The per-layer breakdown table
  • The total asset count vs the expected baseline (~833)
  • The health status (OK if total_assets > 0)
  • A full-report expander

Run via:

uv run marimo edit notebooks/sync_health.py

The new CCC concept guide: dagster-asset-graph

The 21st concept guide in .cocoindex_code/guides.yml. Maps queries about "Dagster asset graph", "5-layer defs/", "Cianfhoghlaim Components" to the canonical files (the 5 layer folders + the 5 Cianfhoghlaim Component classes).

Quick routing

If you want to...Do this
Validate the Dagster asset graphmise run sync:dagster
Re-ingest into Cogneemise run sync:dagster && uv run python scripts/sync_dagster_assets_to_cognee.py
See the asset health in Dagster UIOpen dagster_sync_health asset at orchestration/defs/sync_assets.py
See the asset sync dashboarduv run marimo edit notebooks/sync_health.py (Dagster tab)
Read the latest reportcat stedding/sync-reports/dagster-{date}.md
Run all 6 sync layersmise run sync:all

Cross-references

  • openspec/changes/2026-08-15-retroactive-pre-v7-cleanup-v1/ (this change)
  • openspec/specs/retrospective-cleanup/spec.md (the capability spec)
  • scripts/sync/dagster.sh (the Layer 6 sync script)
  • scripts/sync_dagster_assets_to_cognee.py (the Cognee ingest)
  • orchestration/defs/sync_assets.py (the dagster_sync_health asset)
  • notebooks/sync_health.py (Dagster tab) (the dashboard)
  • .cocoindex_code/guides.yml (the 21st CCC concept guide)
  • .agents/skills/knowledge-sync-loop/SKILL.md (the parent sync loop skill)
  • .agents/skills/dagster/SKILL.md (the core Dagster skill)
  • openspec/specs/dagster-5-layer-component-architecture/spec.md (the 5-layer convention spec)
deprecation-and-migrationaddyosmani/agent-skillsManages deprecation and migration. Use when removing old systems, APIs, or features. Use when migrating users from one implementation to another. Use when migrating a database schema in production, such as renaming or dropping a column without downtime (expand/contract). Use when deciding whether to maintain or sunset existing code.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
performance-optimizationaddyosmani/agent-skillsOptimizes application performance across frontend, backend, queries, and databases. Use when performance requirements exist, when you suspect performance regressions, when Core Web Vitals or load times need improvement, when N+1 query patterns need fixing, or when profiling reveals bottlenecks.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
agent-cost-reportthedotmack/claude-memBelievable agent cost report for any period, default the last 7 full days PT, not counting today. Measured tokens from Claude Code transcripts priced at OpenRouter list prices (ESTIMATED), measured provider spend when a sanctioned source exists, note-taker cost separate, Timing-style HTML/PDF plus report.json, line-items.csv, evidence.json.Stars: 118kUpdated Claude CodeCodexVerifiedInstall
agent-cost-reportthedotmack/claude-memBelievable agent cost report for any period, default the last 7 full days PT, not counting today. Measured tokens from Claude Code transcripts priced at OpenRouter list prices (ESTIMATED), measured provider spend when a sanctioned source exists, note-taker cost separate, Timing-style HTML/PDF plus report.json, line-items.csv, evidence.json.Stars: 118kUpdated Claude CodeCodexVerifiedInstall