Skip to content

donaldfilimon/abi / .agents/skills/wdbx-bench

wdbx-bench

Verified

Build the abi CLI and benchmark the WDBX vector store (in-process insert/search timing), optionally running the abi-wdbx unit-test suite. Use when asked to benchmark WDBX, measure insert/search latency, profile the vector store, or check benchmark output after a storage change.

FileKindSize
SKILL.mdskill md3.2 KB
bench.shexecutablescript1.8 KB
---
name: wdbx-bench
description: Build the abi CLI and benchmark the WDBX vector store (in-process insert/search timing), optionally running the abi-wdbx unit-test suite. Use when asked to benchmark WDBX, measure insert/search latency, profile the vector store, or check benchmark output after a storage change.
---

> **WDBX moved out of this repository on 2026-08-22.** It now lives in the
> sibling repo `../wdbx` together with `abi-compute`,
> `abi-foundation`, `abi-core`, and `abi-telemetry`; `abi` consumes them by
> relative path. Source paths below therefore read `../wdbx/crates/...`. Run
> WDBX-only tests from that repo (`./tools/cargo.sh test --locked
> --manifest-path ../wdbx/Cargo.toml --workspace < /dev/null`, run from the abi
> checkout), and `abi`'s gate (`./tools/check.sh`) from here.
>
> Under the Abbey System Constitution
> (`docs/superpowers/specs/2026-08-22-abbey-system-constitution.md`) WDBX is the
> **provenance-aware episodic substrate**, not a vector store. Most of the
> evidence half of its specification is unimplemented; the measured gap list is
> in `docs/superpowers/specs/2026-08-22-wdbx-conformance-gap-analysis.md`. Do not
> describe an episodic capability as Current on the strength of the vector-store
> features that do exist.
>
> Note for this skill specifically: `bench.sh --suite` runs
> `./tools/cargo.sh test -p abi-wdbx --lib < /dev/null` **from the abi repo**, and
> that still works after the move — abi-wdbx remains in abi's resolve graph as a
> path dependency, so `-p` selection succeeds (verified 2026-08-22: 284 tests,
> exit 0). It is not a workspace member any more, so `--workspace` from abi will
> not include it.

# wdbx-bench — benchmark the WDBX vector store

Driver: **`.agents/skills/wdbx-bench/bench.sh`** (paths relative to repo root).
CLI/timing check — no GUI; evidence is the `RESULT:` line + the latency table.

These are **local, in-memory** measurements — not a published throughput claim
(the CLI says so itself). Numbers vary by machine and are noisy at low counts.

## Prerequisites
- Stable Rust 1.99.1 via `./tools/cargo.sh` (never bare Homebrew `cargo`).

## Run (agent path)
```bash
.agents/skills/wdbx-bench/bench.sh 50        # build CLI, run `abi wdbx benchmark 50`
.agents/skills/wdbx-bench/bench.sh 50 --suite  # also run `./tools/cargo.sh test -p abi-wdbx --lib < /dev/null`
```
It builds the CLI, runs `abi wdbx benchmark <count>`, and asserts the markers
`benchmark (local, in-memory`, `inserts:`, `searches:`. Prints
`RESULT: PASS — WDBX benchmark ran.` (exit 0) or `RESULT: FAIL — N check(s) failed.`

## Gotchas
- **Not a throughput claim.** Per-op time includes acceleration-kernel dispatch;
  the GPU path is the vectorized CPU fallback unless native kernels are linked
  (`abi backends` shows `accelerated=false` on this machine).
- Low counts are high-variance — use ≥50 for a stable-ish p50; the suite
  (`--suite`) runs `./tools/cargo.sh test -p abi-wdbx --lib < /dev/null`.
- `bench.sh <non-number>` → usage, exit 2.

## Troubleshooting
| Symptom | Fix |
|---|---|
| `build` FAIL | Run `./tools/check.sh` to see the real error. |
| missing `inserts:`/`searches:` marker | CLI grammar drifted — check the `crates/abi-cli` WDBX handler and `../wdbx/crates/abi-wdbx`. |
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall