topprismdata/cultivating-ml-agent / skills/examples/onnx-minimal-network-design
onnx-minimal-network-design
VerifiedUse when: (1) Competing in ONNX-based competitions (NeuroGolf, model compression), (2) Need to design minimal neural networks as ONNX graphs, (3) Want to replace verbose agent-generated circuits with compact hand-designed nets. Key principles: single-node graphs have zero memory cost (output is free), Conv is a linear classifier (cannot separate nonlinear rules), Gather implements color permutations (10 params), use LogisticRegression to solve Conv weights for local 3×3 rules. Sparse initializers are rejected by ONNX strict checker for Conv nodes. Validated on NeuroGolf 2026 (7228 baseline, independent toolkit).
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 3.7 KB |
SKILL.md
View on GitHub---
name: onnx-minimal-network-design
description: |
Use when: (1) Competing in ONNX-based competitions (NeuroGolf, model compression),
(2) Need to design minimal neural networks as ONNX graphs, (3) Want to replace
verbose agent-generated circuits with compact hand-designed nets. Key principles:
single-node graphs have zero memory cost (output is free), Conv is a linear
classifier (cannot separate nonlinear rules), Gather implements color
permutations (10 params), use LogisticRegression to solve Conv weights for
local 3×3 rules. Sparse initializers are rejected by ONNX strict checker for
Conv nodes. Validated on NeuroGolf 2026 (7228 baseline, independent toolkit).
---
# ONNX Minimal Network Design
## Problem
Neural network golf competitions require the **smallest possible ONNX network**
that correctly implements a transformation. Score = `max(1, 25 - ln(memory + params))`.
Smaller networks → exponentially higher scores. The key insight: **memory
dominates cost** (intermediate tensor bytes), not parameters.
## Solution
### Core Scoring Formula
```
memory = Σ(bytes of all intermediate tensors) # input/output are FREE
params = Σ(initializer element counts)
score = max(1, 25 - ln(memory + params))
```
### Design Principle 1: Single-Node = Zero Memory
A single-node graph (1 Conv/Gather/Transpose) has **zero intermediate memory**
(output tensor is free). Only params count.
**Implication**: A 1-node Conv with 900 params (cost=900, score=18.2) beats a
4-node net with 200 params + 5000 memory (cost=5200, score=16.3).
### Design Principle 2: Conv is a Linear Classifier
A Conv node maps 3×3 one-hot neighborhoods → output channels. This is a
**linear classifier** on 90-dim features (9 positions × 10 colors).
To find Conv weights for a local rule:
1. Collect all (3×3 neighborhood, output color) pairs from training data
2. Verify the rule is deterministic (same neighborhood → same output)
3. Train LogisticRegression (multi-class, C=1e4, solver='newton-cg') on the pairs
4. Extract coef_ as Conv W[10,10,3,3], intercept_ as B[10]
5. Verify ONNX runtime output matches ground truth on ALL examples
**Limitation**: If LogReg accuracy < 100%, the rule is NOT linearly separable.
Need nonlinear ops (Cast/Equal/Where) — a single Conv cannot implement it.
### Design Principle 3: Gather for Color Permutations
A pixel-wise color map (e.g., swap color 3↔4) is a 1-node Gather:
- idx = [0,5,6,4,3,1,2,7,9,8] (10-element lookup table)
- Gather(input, idx, axis=1) maps each one-hot vector to the permuted one
- Cost: 10 params, 0 memory → score ≈ 22.7
### Design Principle 4: Sparse Initializers REJECTED
Converting dense sparse weights to `sparse_initializer` reduces params but
**onnx.checker.check_model(full_check=True) rejects sparse Conv weights**:
```
W typestr: T, has unsupported type: sparse_tensor(float)
```
Only use sparse for ops that explicitly support it (not Conv/MatMul in practice).
## Independent Toolkit Architecture
```python
# Grid encoder: ARC grid [[c,...]] → one-hot [1,10,30,30] float32
grid_to_onehot(grid, size=30)
# ONNX builders:
design_gather_colormap(color_map) # 1-node Gather, 10 params
design_transpose() # 1-node Transpose, 0 params
design_border(color=8) # 1-node Conv border detection
# Verifier: onnxruntime on ALL train + arc-gen examples
verify(model_bytes, task_json)
# Scorer: compute params + memory → points
score_model(model_bytes)
```
## Evidence (NeuroGolf 2026)
- task171 border: independently designed 1-Conv (19/19 correct, matches existing bundle)
- task222: LogReg solver achieved 99.95% (not 100% — rule is nonlinear)
- 87 tasks had sparse-able weights but sparse rejected by strict checker
- 400-task scan: all simple rules already 1-node in community bundle
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| brainstormingobra/superpowersYou MUST use this before any creative work - creating features, building components, adding functionality, modifying behavior, or planning anything new, in software or out of it (a talk, a business, a renovation). | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| brand-guidelinesanthropics/skillsApplies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. Use it when brand colors or style guidelines, visual formatting, or company design standards apply. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| frontend-designanthropics/skillsGuidance for distinctive, intentional visual design when building new UI or reshaping an existing one. Helps with aesthetic direction, typography, and making choices that don't read as templated defaults. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| mcp-builderanthropics/skillsGuide for creating high-quality MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools. Use when building MCP servers to integrate external APIs or services, whether in Python (FastMCP) or Node/TypeScript (MCP SDK). | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |