Disentinel/grafema / .claude/skills/rfdb-stale-node-cleanup
rfdb-stale-node-cleanup
VerifiedRecover from stale RFDB nodes that survive `commit_batch` cleanup despite having their `file` field listed in `changed_files`. Use when: (1) re-running an enricher/orchestrator step but old data with the same file field stays in the graph, (2) `commit_batch(changed_files=[X], ...)` reports success but a query later still returns nodes with `file = X`, (3) you re-pointed structural nodes (e.g., DIRECTORY/FILE) from a synthetic file path to real file paths and the old synthetic-path nodes won't go away, (4) graph-stream / find_by_type returns a mix of "stale" and "new" nodes for the same logical entity. Root cause: `handle_commit_batch` calls `engine.find_by_attr({file: X})` to enumerate nodes-to-delete, but that query path returns only a PARTIAL set after compaction (L1 segments and the file→nodes index can desync). Workaround: enumerate stale nodes via the TS rfdb client `queryNodes({file: X})` (different code path that finds all of them) and call `deleteNode(id)` per node, then re-commit the intended state. Related: rfdb-manifest-l1-carryforward (same L1 family).
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 8.0 KB |
SKILL.md
View on GitHub---
name: rfdb-stale-node-cleanup
description: |
Recover from stale RFDB nodes that survive `commit_batch` cleanup despite
having their `file` field listed in `changed_files`. Use when:
(1) re-running an enricher/orchestrator step but old data with the same file
field stays in the graph, (2) `commit_batch(changed_files=[X], ...)` reports
success but a query later still returns nodes with `file = X`, (3) you
re-pointed structural nodes (e.g., DIRECTORY/FILE) from a synthetic file
path to real file paths and the old synthetic-path nodes won't go away,
(4) graph-stream / find_by_type returns a mix of "stale" and "new" nodes
for the same logical entity. Root cause: `handle_commit_batch` calls
`engine.find_by_attr({file: X})` to enumerate nodes-to-delete, but that
query path returns only a PARTIAL set after compaction (L1 segments and
the file→nodes index can desync). Workaround: enumerate stale nodes via
the TS rfdb client `queryNodes({file: X})` (different code path that finds
all of them) and call `deleteNode(id)` per node, then re-commit the
intended state. Related: rfdb-manifest-l1-carryforward (same L1 family).
author: Claude Code
version: 1.0.0
date: 2026-04-07
---
# RFDB Stale Node Cleanup After Compaction
## Problem
`commit_batch(changed_files=[X], nodes=[...], ...)` is supposed to atomically
replace all nodes with `file = X` by tombstoning the old set and inserting
the new one. After compaction (RFD-20 L1 segments), this is unreliable: only
a partial subset of the old nodes gets tombstoned and the rest survive
indefinitely, contaminating queries.
This is especially visible when:
- An orchestrator step is rewritten to emit nodes with new `file` fields
(e.g., switching DIRECTORY/FILE structural nodes from a synthetic
`__grafema_virtual/...` path to real directory paths). The new commit
uses upsert by semantic_id, but the OLD nodes at the synthetic path
are not all tombstoned.
- Multiple commit_batch invocations target the same legacy file path —
each invocation tombstones a few more nodes but never gets all of them.
## Context / Trigger Conditions
- After running a commit pipeline, `find_by_type(NODE_TYPE)` returns more
rows than the latest commit produced.
- Graph-stream output mixes nodes with the new `file` value and a "stale"
group with the old `file` value, both for entities you considered
rewritten.
- `changed_files` contained the legacy path, the orchestrator log says
`commitBatch` succeeded with `files=N` matching expectation.
- Splitting nodes by file shows orphans only at the legacy/synthetic path
while the rest of the graph looks correct.
- Re-running the same `commit_batch` reduces the orphan count but never
clears it to zero.
- `manifest_index.json` has a recent compaction snapshot
(`l1_node_segments` populated) older than the failing commits.
## Diagnostic Path
1. Get a per-`file` count of the affected node type:
```bash
curl -s "http://localhost:3333/api/graph-stream?nodeTypes=TYPE&maxNodes=5000" \
| python3 -c "
import json, sys
counts = {}
for line in sys.stdin:
try:
d = json.loads(line)
if d.get('type') == 'node':
counts[d.get('file','')] = counts.get(d.get('file',''), 0) + 1
except: pass
for k,v in sorted(counts.items(), key=lambda kv: -kv[1])[:20]: print(v, k)
"
```
2. Look for entries clustered under the legacy/synthetic file path that
should have been replaced.
3. Confirm via `/api/node/<id>` that one such node has the legacy `file`
and the metadata you remember from the old code path.
4. Confirm a corresponding "new" node exists (with the same name but new
`file`) — proves the rewrite ran but cleanup didn't.
## Solution
### Direct cleanup via TS rfdb client (most reliable)
The TS client's `queryNodes({file: X})` uses a different storage code path
than `handle_commit_batch`'s `find_by_attr` and reliably enumerates all
matching nodes, including those promoted to L1 segments.
```js
// /tmp/cleanup-stale.mjs
import { RFDBClient } from '/path/to/grafema/packages/rfdb/dist/client.js';
const client = new RFDBClient(
'/path/to/.grafema/rfdb.sock', // socket path is positional arg, not options object
'cleanup',
);
await client.connect();
const stale = [];
for await (const n of client.queryNodes({ file: '__grafema_virtual/legacy-path' })) {
stale.push(n.id);
}
console.log('Found stale nodes:', stale.length);
for (const id of stale) {
try { await client.deleteNode(id); }
catch (e) { console.error('delete failed', id, e.message); }
}
await client._send('flush', {}).catch(e => console.error('flush:', e.message));
await client.close();
```
Then re-run the commit step that produces the desired state. The graph
will now be clean.
### Caveats
- **`queryNodes({file: X})` may match more than literal equality.** It
can return any node where `file` matches the pattern under the storage
layer's filter rules (substring or exact, depending on backend version).
Use the **most specific** path you can to avoid wiping live data.
- **Verify the count before deleting.** If you expect ~300 stale and the
query returns the entire graph, abort. Print first few IDs and double-
check their type/file via `/api/node/<id>`.
- After deleting, re-commit the intended state immediately. The graph is
in a partially-empty state until you do.
### Why not just keep calling commit_batch?
`handle_commit_batch` (rfdb_server.rs) does:
```rust
for file in &changed_files {
let old_ids = engine.find_by_attr(&AttrQuery { file: Some(file.clone()), .. });
for id in &old_ids { engine.delete_node(*id); }
}
```
`find_by_attr` for a `file` filter consults the in-memory `file_to_node_ids`
index, which is rebuilt from segments at startup. After compaction, some
nodes for a given file path are in L1 with index entries that can be
incomplete or stale relative to what's still on disk. Subsequent
`find_by_attr` calls return only the index-known IDs, so other historical
copies of the same node are never enumerated for tombstoning.
The TS client's `queryNodes` route through a different store method that
walks segments directly instead of trusting the file index, so it sees
the missed entries.
## Verification
After cleanup + recommit:
```bash
# Total count for the type matches what you committed
curl -s "http://localhost:3333/api/graph-stream?nodeTypes=DIRECTORY,FILE&maxNodes=2000" \
| tail -2
# → {"edgeCount":721,"elapsed":...,"nodeCount":722,"type":"done"}
# No nodes left at the legacy path
curl -s "http://localhost:3333/api/graph-stream?nodeTypes=DIRECTORY,FILE&maxNodes=2000" \
| python3 -c "
import json, sys
n = sum(1 for line in sys.stdin
for d in [json.loads(line)] if d.get('type')=='node'
and d.get('file','').startswith('__grafema_virtual'))
print('stale survivors:', n)
"
# → stale survivors: 0
```
## Notes
- **This is a workaround, not a fix.** The root fix is to make
`find_by_attr({file:X})` consistent with `queryNodes({file:X})` after
compaction — both should walk every storage layer (write_buffer + L0 +
L1) and respect tombstones.
- After running this workaround, keep an eye out for the same symptom in
future analyzes; until the root fix lands, every step that re-points a
node's `file` field will need this cleanup.
- This bug is in the same family as `rfdb-manifest-l1-carryforward`
(compaction interactions). Both surface as "data exists on disk but
isn't queryable correctly."
- Don't try `delete_nodes` via REST/HTTP — only the unix-socket protocol
exposes `deleteNode`. Use the TS client.
- Always **flush** after delete via `client._send('flush', {})` so
subsequent queries see the tombstones.
## Related Files
- `packages/rfdb-server/src/bin/rfdb_server.rs:1996` —
`handle_commit_batch` (the buggy delete path)
- `packages/rfdb-server/src/bin/rfdb_server.rs:1242` — `Request::DeleteNode`
- `packages/rfdb/dist/base-client.js:86` — `deleteNode` TS client API
- `packages/rfdb/dist/base-client.js:254` — `queryNodes` TS client API
- `.claude/skills/rfdb-manifest-l1-carryforward/SKILL.md` — sibling bug
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| receiving-code-reviewobra/superpowersUse when receiving code review feedback, before implementing suggestions, especially if feedback seems unclear or technically questionable - requires technical rigor and verification, not performative agreement or blind implementation | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| requesting-code-reviewobra/superpowersUse when completing tasks, implementing major features, or before merging to verify work meets requirements | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| using-git-worktreesobra/superpowersUse when starting feature work that needs isolation from current workspace or before executing implementation plans - ensures an isolated workspace exists via native tools or git worktree fallback | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| verification-before-completionobra/superpowersUse when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |