Disentinel/grafema / .claude/skills/rfdb-v2-clear-ephemeral-trap
rfdb-v2-clear-ephemeral-trap
VerifiedFix RFDB V2 graph data silently not persisting to disk after analysis. Use when: (1) rfdb-server reports "0 nodes, 0 edges" on restart despite successful analysis, (2) segment directories exist but are empty (no .bin files), (3) manifest_index.json shows total_nodes: 0 despite analysis logging 70k+ nodes, (4) Docker builds produce empty graph databases, (5) --clear flag used before analyze command. Root cause: GraphEngineV2::clear() replaces the store with MultiShardStore::ephemeral() which has path: None, causing all flush operations to write to in-memory buffers only.
RFDB V2 Clear Makes Engine Ephemeral
STATUS UPDATE (2026-06-11, branch feat/datalog — W8 Part 2)
Both flavors of this trap are FIXED on feat/datalog. GraphEngineV2::clear() /
clear_durable() now performs a REAL durable clear: it truncates the on-disk database
(deletes segments/ + gc/ + manifest authority, recreates a fresh empty manifest and
shard skeleton) and resets all engine caches (D2 materialize pins, planner-stats, W9
shared indexes, derive pin sidecars). The engine stays DISK-BACKED after clear — no
ephemeral swap, so post-clear analysis persists (the 2026-02 data-loss flavor) and a
clear+restart does NOT resurrect the old graph (the 2026-06 placebo flavor, gaps.md).
Wire Clear surfaces truncation errors to the client. Tests:
w8_durable_clear_truncates_disk_and_survives_reopen, w8_durable_clear_drops_pin_sidecars
(engine_v2.rs). Manual fallback remains rm -rf <db>.rfdb (e.g. after a crash inside the
tiny manifest-reset window). On binaries WITHOUT this fix, everything below still applies.
Problem
After running grafema analyze --clear, the analysis completes successfully and reports
tens of thousands of nodes/edges, but on restart the database is empty (0 nodes, 0 edges).
The data was never persisted to disk despite multiple flush operations succeeding without
error.
Context / Trigger Conditions
grafema analyze --clearreports successful analysis with node/edge counts- On restart, rfdb-server logs "Default database: 0 nodes, 0 edges"
manifest_index.jsonshows"total_nodes": 0, "total_edges": 0- Segment directories (
segments/00/,segments/01/, etc.) exist but are empty - No error messages anywhere in the logs
- The
tracing::info!("Flushing...")message fromcollect_and_write_data()never appears (because CommitBatch already flushed, but to ephemeral storage)
Root Cause
GraphEngineV2::clear() (engine_v2.rs:572-578) replaces the store and manifest with
ephemeral versions:
fn clear(&mut self) {
self.store = MultiShardStore::ephemeral(DEFAULT_SHARD_COUNT); // path: None!
self.manifest = ManifestStore::ephemeral();
// ...
}
MultiShardStore::ephemeral() creates shards with path: None. When Shard::flush_with_ids()
runs (shard.rs:710), it checks if let Some(path) = &self.path — ephemeral shards take the
else branch which writes to an in-memory Cursor<Vec<u8>> instead of disk files.
The data lives in memory segments and is queryable (nodeCount/edgeCount return correct values), but nothing is written to disk. When the process exits, all data is lost.
Why It's Hard to Detect
- No errors: All operations succeed — add_nodes, flush, nodeCount all work correctly
- Counts are correct: The server reports accurate node/edge counts from in-memory segments
- CommitBatch masks the issue: Each CommitBatch internally calls flush(), which succeeds
(to memory). The subsequent explicit
flush()from the CLI finds an empty delta_log and returns early — so thetracing::info!("Flushing...")message never appears - Segments directory structure exists: The directories are created during initial DB creation before clear() is called
Solution
Don't use --clear when you need data to persist. For fresh builds (Docker, CI):
# BAD: --clear makes V2 engine ephemeral, data lost
RUN grafema analyze /build --clear
# GOOD: No --clear needed for fresh build (no existing DB)
RUN grafema analyze /build
If you truly need to clear and re-analyze an existing database, the proper approach is to
delete the database directory before starting the server, rather than using --clear:
rm -rf .grafema/graph.rfdb
grafema analyze .
Verification
After analysis completes, check that segments contain actual data:
# Should show .bin files with non-zero size
ls -la .grafema/graph.rfdb/segments/00/
# Should show total_nodes > 0
cat .grafema/graph.rfdb/manifest_index.json
Notes
- This is a V2-specific issue. V1 engine's
clear()resets in-place without making shards ephemeral. - The
--clearflag sends aClearcommand to the server which callsengine.clear(). create_ephemeral()is designed for test databases that don't need persistence. Using it inclear()is an architectural shortcut that creates this trap.- Related MEMORY note: "Ephemeral databases skip flush" — this is the V2 manifestation of the same concept documented for V1.
Second flavor discovered 2026-06-09: clear+restart = PLACEBO (stale data SURVIVES)
The Feb-2026 flavor above is data LOSS (clear, then analyze into the ephemeral engine, no
restart). The CLI's actual --clear flow (analyzeAction.ts:340-346) is
backend.clear() → shutdownServer() → connect() — the shutdown DISCARDS the ephemeral
state and the fresh auto-started server reloads the untouched on-disk DB. Net effect: --clear
clears nothing; the manifest version just keeps counting (observed 660 → 1001 across a
"cleared" run).
Detection signature: after --clear + full reanalysis, edges from a PRIOR run survive with an
older _generation in their metadata (e.g. 15 531 DERIVED_FROM edges at gen 36 alongside the
fresh gen-37 DERIVES_FROM — 15 014 (src,dst) pairs carrying BOTH). Forensics: edge metadata
_generation/_source (stamped by orchestrator gc.rs) maps every edge to the run that wrote
it; gen-tracker.json holds the run counter. Segment-file mtimes are NOT evidence of write
time — compaction rewrites old data into new files.
Workaround stands: rm -rf .grafema/graph.rfdb (+ optionally gen-tracker.json) before the
server starts. Real fix: GraphEngineV2::clear() must clear the persistent store. Gap recorded
in _ai/gaps.md (2026-06-09).
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |