plurigrid/asi / ies/music-topos/.claude/skills/compositional-acset-comparison
compositional-acset-comparison
VerifiedCompare data structures (DuckDB, LanceDB) via ACSets with persistent homology coverage analysis and geometric morphism translation.
| File | Kind | Size |
|---|---|---|
| ColoringFunctor.jl | other | 14.3 KB |
| ComparisonUtils.jl | other | 15.8 KB |
| DuckDBACSet.jl | other | 9.5 KB |
| GeometricMorphism.jl | other | 18.1 KB |
| GhristCoverage.jl | other | 15.5 KB |
| IrreversibleMorphisms.jl | other | 16.7 KB |
| LanceDBACSet.jl | other | 15.4 KB |
| SKILL.md | skill md | 16.6 KB |
| SideBySideComparison.jl | other | 29.0 KB |
GeometricMorphism.jl
View on GitHub# GeometricMorphism.jl - Presheaf Topos Translations Between Schemas
# Gay.jl Seed: 4000000
#
# A geometric morphism f: PSh(C) → PSh(D) between presheaf topoi consists of:
# f*: PSh(D) → PSh(C) (inverse image - pullback)
# f_*: PSh(C) → PSh(D) (direct image - pushforward)
#
# These satisfy: f* ⊣ f_* (adjunction)
#
# For DuckDB ↔ LanceDB: concepts that translate losslessly vs lossy
using ACSets, Catlab
include("DuckDBACSet.jl")
include("LanceDBACSet.jl")
# ═══════════════════════════════════════════════════════════════════════
# PRESHEAF CATEGORY STRUCTURE
# ═══════════════════════════════════════════════════════════════════════
"""
PSh(C) = [C^op, Set] = Presheaves on C
A presheaf F: C^op → Set assigns:
- To each object c ∈ C: a set F(c)
- To each morphism f: c → d: a function F(f): F(d) → F(c)
(contravariant - arrows reverse)
For schemas:
- F(Table) = set of all tables
- F(Column) = set of all columns
- F(table_morphism) = function mapping columns to their tables
"""
struct PresheafCategory
schema::Any
objects::Vector{Symbol}
morphisms::Vector{Symbol}
end
function presheaf_category(schema)
PresheafCategory(
schema,
collect(obs(schema)),
[nameof(h) for h in homs(schema)]
)
end
# ═══════════════════════════════════════════════════════════════════════
# GEOMETRIC MORPHISM DEFINITION
# ═══════════════════════════════════════════════════════════════════════
"""
Geometric Morphism f: E → F between topoi consists of:
1. Inverse image f*: F → E (preserves finite limits, all colimits)
- Pulls concepts FROM target TO source
- "What does this F-concept look like in E?"
2. Direct image f_*: E → F (right adjoint to f*)
- Pushes concepts FROM source TO target
- "How does this E-concept appear in F?"
For DuckDB → LanceDB:
- f* pulls LanceDB concepts to DuckDB
- f_* pushes DuckDB concepts to LanceDB
"""
struct GeometricMorphism
source::PresheafCategory # E = PSh(SchDuckDB)
target::PresheafCategory # F = PSh(SchLanceDB)
inverse_image::Dict{Symbol, Union{Symbol, Nothing}} # f*
direct_image::Dict{Symbol, Union{Symbol, Nothing}} # f_*
translation_type::Dict{Symbol, Symbol} # :lossless, :lossy, :none
end
# ═══════════════════════════════════════════════════════════════════════
# CONCEPT TRANSLATION MAPPINGS
# ═══════════════════════════════════════════════════════════════════════
"""
Inverse image f*: PSh(LanceDB) → PSh(DuckDB)
"Pull LanceDB concepts back to DuckDB"
Returns: What DuckDB object represents this LanceDB concept?
"""
const INVERSE_IMAGE_MAP = Dict{Symbol, Union{Symbol, Nothing}}(
# Storage hierarchy
:Database => nothing, # No DuckDB analog (DuckDB is single-DB)
:Table => :Table, # ✓ Direct correspondence
:Manifest => nothing, # ✗ No versioning in core DuckDB
:Fragment => :RowGroup, # ~ Approximate (both are row partitions)
# Column types
:Column => :Column, # ✓ Direct correspondence
:VectorColumn => :Column, # ~ Lossy (loses vector semantics)
:ScalarColumn => :Column, # ✓ Direct correspondence
# Vector indexing
:VectorIndex => nothing, # ✗ Requires vss extension
:Partition => nothing, # ✗ No IVF in DuckDB
:Centroid => nothing, # ✗ No centroids in DuckDB
# Embedding
:EmbeddingFunction => nothing, # ✗ No embedding registry
# Versioning
:SDKVersion => nothing, # ✗ Different versioning model
:FileFormatVersion => nothing, # ✗ Different format
:TableFormatFeature => nothing, # ✗ No feature flags
)
"""
Direct image f_*: PSh(DuckDB) → PSh(LanceDB)
"Push DuckDB concepts forward to LanceDB"
Returns: What LanceDB object represents this DuckDB concept?
"""
const DIRECT_IMAGE_MAP = Dict{Symbol, Union{Symbol, Nothing}}(
# Storage hierarchy
:Table => :Table, # ✓ Direct correspondence
:RowGroup => :Fragment, # ~ Approximate (Fragment is more general)
:Column => :Column, # ✓ Direct correspondence
:Segment => nothing, # ✗ No segment concept (Lance uses columnar files)
# Type system
:LogicalType => nothing, # ~ Embedded in Column dtype
:PhysicalType => nothing, # ~ Arrow handles this
# Execution
:Vector => nothing, # ✗ Arrow RecordBatch instead
:DataChunk => nothing, # ✗ Different execution model
# State
:SegmentState => nothing, # ✗ Lance is immutable (no TRANSIENT)
:CompressionAlgo => nothing,# ~ Lance handles compression internally
)
"""
Translation type classification
"""
function classify_translation(source_obj::Symbol, target_obj::Union{Symbol, Nothing})
if target_obj === nothing
:none # No translation exists (dead zone)
elseif source_obj == target_obj
:lossless # Direct correspondence
else
:lossy # Approximate translation
end
end
# ═══════════════════════════════════════════════════════════════════════
# BUILD GEOMETRIC MORPHISM
# ═══════════════════════════════════════════════════════════════════════
"""
Construct geometric morphism f: PSh(DuckDB) → PSh(LanceDB)
"""
function build_geometric_morphism()
source = presheaf_category(SchDuckDB)
target = presheaf_category(SchLanceDB)
# Build translation type map
translation_type = Dict{Symbol, Symbol}()
# Classify inverse image translations
for (lance_obj, duck_obj) in INVERSE_IMAGE_MAP
translation_type[lance_obj] = classify_translation(lance_obj, duck_obj)
end
# Classify direct image translations
for (duck_obj, lance_obj) in DIRECT_IMAGE_MAP
key = Symbol("direct_", duck_obj)
translation_type[key] = classify_translation(duck_obj, lance_obj)
end
GeometricMorphism(
source,
target,
INVERSE_IMAGE_MAP,
DIRECT_IMAGE_MAP,
translation_type
)
end
# ═══════════════════════════════════════════════════════════════════════
# ESSENTIAL IMAGE COMPUTATION
# ═══════════════════════════════════════════════════════════════════════
"""
Essential Image of f: The objects in E that have "essential" representation in F
For geometric morphism f: E → F:
- Essential image = objects x ∈ E such that f_*(x) is "substantive" in F
- These are the concepts that translate meaningfully
DuckDB concepts with LanceDB analogs:
"""
function compute_essential_image(gm::GeometricMorphism)
essential = Symbol[]
partial = Symbol[]
none = Symbol[]
for (duck_obj, lance_obj) in DIRECT_IMAGE_MAP
if lance_obj !== nothing && duck_obj == lance_obj
push!(essential, duck_obj)
elseif lance_obj !== nothing
push!(partial, duck_obj)
else
push!(none, duck_obj)
end
end
Dict(
:essential => essential, # Lossless translation
:partial => partial, # Lossy translation
:none => none, # No translation (dead zone)
:coverage => length(essential) / length(DIRECT_IMAGE_MAP)
)
end
"""
Compute which LanceDB concepts have DuckDB analogs
"""
function compute_inverse_essential_image(gm::GeometricMorphism)
essential = Symbol[]
partial = Symbol[]
none = Symbol[]
for (lance_obj, duck_obj) in INVERSE_IMAGE_MAP
if duck_obj !== nothing && lance_obj == duck_obj
push!(essential, lance_obj)
elseif duck_obj !== nothing
push!(partial, lance_obj)
else
push!(none, lance_obj)
end
end
Dict(
:essential => essential,
:partial => partial,
:none => none,
:coverage => length(essential) / length(INVERSE_IMAGE_MAP)
)
end
# ═══════════════════════════════════════════════════════════════════════
# AM RADIO COVERAGE INTERPRETATION
# ═══════════════════════════════════════════════════════════════════════
"""
AM Radio Coverage = Geometric Morphism Translation
Radio Station → Schema Object
Coverage Zone → Set of instances F(object)
Signal Overlap → Translatable concepts
Dead Zone → Objects with no translation
Essential Image = Coverage overlap between stations
- Station A covers zone Z_A
- Station B covers zone Z_B
- Essential overlap = Z_A ∩ Z_B (both can serve)
- Dead zone = Z_A △ Z_B (only one can serve)
"""
const AM_RADIO_INTERPRETATION = """
AM Radio Coverage Model for Schema Translation
═══════════════════════════════════════════════
COVERAGE ZONES:
DuckDB Station (550 AM): Table, Column, RowGroup, Segment
LanceDB Station (880 AM): Database, Table, Manifest, Fragment, VectorColumn
SIGNAL OVERLAP (Essential Image):
✓ Table ↔ Table Full signal overlap, same semantics
✓ Column ↔ Column Full signal overlap, same semantics
WEAK SIGNAL (Lossy Translation):
~ RowGroup ↔ Fragment Partial overlap, different granularity
~ VectorColumn → Column Partial, loses vector index semantics
DEAD ZONES (No Translation):
✗ Segment → ??? DuckDB-only (compressed storage blocks)
✗ ??? ← Manifest LanceDB-only (version chains)
✗ ??? ← VectorIndex LanceDB-only (IVF partitions)
✗ ??? ← Centroid LanceDB-only (partition centroids)
IRREVERSIBLE MORPHISMS = Station Blackouts:
🔴 parent_manifest: Only broadcasts forward in time
🔴 source_column: Lossy compression, original unrecoverable
These are FUNDAMENTAL COVERAGE LIMITS.
No amount of transmitter power can overcome them.
"""
# ═══════════════════════════════════════════════════════════════════════
# COLOR CODING
# ═══════════════════════════════════════════════════════════════════════
const TRANSLATION_COLORS = Dict(
:lossless => "#2BEE64", # Green - full coverage
:lossy => "#EED52B", # Yellow - weak signal
:none => "#EE2B2B", # Red - dead zone
:irreversible => "#CD24A6", # Magenta - fundamental limit
)
"""
Color-code the geometric morphism analysis
"""
function color_geometric_morphism(gm::GeometricMorphism)
colored = Dict{Symbol, String}()
for (obj, trans_type) in gm.translation_type
colored[obj] = TRANSLATION_COLORS[trans_type]
end
colored
end
# ═══════════════════════════════════════════════════════════════════════
# ADJUNCTION VERIFICATION
# ═══════════════════════════════════════════════════════════════════════
"""
Verify adjunction f* ⊣ f_*
For geometric morphisms: Hom_E(f*(Y), X) ≅ Hom_F(Y, f_*(X))
In terms of translations:
- If LanceDB concept Y pulls back to DuckDB X via f*
- And DuckDB X pushes forward to LanceDB Z via f_*
- Then Y should be "close to" Z
Adjunction holds when round-trip is coherent.
"""
function verify_adjunction(gm::GeometricMorphism)
coherent_pairs = []
broken_pairs = []
for (lance_obj, duck_obj) in INVERSE_IMAGE_MAP
if duck_obj === nothing
continue
end
# Check round-trip: LanceDB → DuckDB → LanceDB
round_trip = get(DIRECT_IMAGE_MAP, duck_obj, nothing)
if round_trip == lance_obj
push!(coherent_pairs, (lance_obj, duck_obj, round_trip))
else
push!(broken_pairs, (lance_obj, duck_obj, round_trip))
end
end
Dict(
:coherent => coherent_pairs,
:broken => broken_pairs,
:adjunction_holds => length(broken_pairs) == 0,
:coherence_ratio => length(coherent_pairs) / (length(coherent_pairs) + length(broken_pairs))
)
end
# ═══════════════════════════════════════════════════════════════════════
# HOMOICONIC CONNECTION
# ═══════════════════════════════════════════════════════════════════════
"""
Homoiconic Insight: Algorithm ↔ Data Boundary
In self-hosted Lisps:
- Code is data (S-expressions)
- Data is code (eval)
- The distinction dissolves
For schema comparison:
- Schema IS an algorithm (categorical structure)
- Data IS a schema (instances are functors)
- Geometric morphism = algorithm transformation
Phase-scoped evaluation:
- RED phase: Generate schema objects
- GREEN phase: Coordinate translations
- BLUE phase: Validate consistency
Leaving phases open avoids entanglement.
"""
const HOMOICONIC_CONNECTION = """
Homoiconic Presheaf Interpretation
═══════════════════════════════════
In a self-hosted Lisp:
(define schema '(Table → RowGroup → Column → Segment))
(define instance (eval schema data))
The schema IS the algorithm for structuring data.
The instance IS the data structured by the algorithm.
Geometric morphism f: PSh(C) → PSh(D) is then:
(define f (lambda (algorithm) (transform algorithm C→D)))
Where:
- f* = pull algorithm from D to C (inverse transformation)
- f_* = push algorithm from C to D (forward transformation)
PHASE-SCOPED EVALUATION:
- Phase 1 (RED): Define schema objects
- Phase 2 (GREEN): Define morphisms (translations)
- Phase 3 (BLUE): Verify consistency (adjunction)
Leave phases open to avoid premature commitment.
Close only when translation is validated.
"""
# ═══════════════════════════════════════════════════════════════════════
# SUMMARY GENERATOR
# ═══════════════════════════════════════════════════════════════════════
"""
Generate summary string with computed metrics
"""
function generate_summary(essential, inverse_essential, adjunction)
duck_coverage = round(essential[:coverage] * 100, digits=1)
lance_coverage = round(inverse_essential[:coverage] * 100, digits=1)
adj_status = adjunction[:adjunction_holds] ? "HOLDS" : "BROKEN"
coherent_count = length(adjunction[:coherent])
broken_count = length(adjunction[:broken])
"""
GEOMETRIC MORPHISM SUMMARY: DuckDB ↔ LanceDB
═════════════════════════════════════════════
Essential Coverage:
DuckDB → LanceDB: $(duck_coverage)%
LanceDB → DuckDB: $(lance_coverage)%
Adjunction Status: $(adj_status)
Coherent pairs: $(coherent_count)
Broken pairs: $(broken_count)
Key Insights:
1. Both schemas share Table/Column core (strong overlap)
2. LanceDB adds versioning + vectors (unique coverage)
3. DuckDB adds execution model (unique coverage)
4. Irreversible morphisms create fundamental limits
"""
end
# ═══════════════════════════════════════════════════════════════════════
# RUN ANALYSIS
# ═══════════════════════════════════════════════════════════════════════
function run_geometric_morphism_analysis()
gm = build_geometric_morphism()
essential = compute_essential_image(gm)
inverse_essential = compute_inverse_essential_image(gm)
colors = color_geometric_morphism(gm)
adjunction = verify_adjunction(gm)
Dict(
:geometric_morphism => gm,
:essential_image => essential,
:inverse_essential_image => inverse_essential,
:colors => colors,
:adjunction => adjunction,
:am_radio => AM_RADIO_INTERPRETATION,
:homoiconic => HOMOICONIC_CONNECTION,
:summary => generate_summary(essential, inverse_essential, adjunction)
)
end
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |