Skip to content

topprismdata/cultivating-ml-agent / skills/examples/onnx-minimal-network-design

onnx-minimal-network-design

Verified

Use when: (1) Competing in ONNX-based competitions (NeuroGolf, model compression), (2) Need to design minimal neural networks as ONNX graphs, (3) Want to replace verbose agent-generated circuits with compact hand-designed nets. Key principles: single-node graphs have zero memory cost (output is free), Conv is a linear classifier (cannot separate nonlinear rules), Gather implements color permutations (10 params), use LogisticRegression to solve Conv weights for local 3×3 rules. Sparse initializers are rejected by ONNX strict checker for Conv nodes. Validated on NeuroGolf 2026 (7228 baseline, independent toolkit).

FileKindSize
SKILL.mdskill md3.7 KB
---
name: onnx-minimal-network-design
description: |
  Use when: (1) Competing in ONNX-based competitions (NeuroGolf, model compression),
  (2) Need to design minimal neural networks as ONNX graphs, (3) Want to replace
  verbose agent-generated circuits with compact hand-designed nets. Key principles:
  single-node graphs have zero memory cost (output is free), Conv is a linear
  classifier (cannot separate nonlinear rules), Gather implements color
  permutations (10 params), use LogisticRegression to solve Conv weights for
  local 3×3 rules. Sparse initializers are rejected by ONNX strict checker for
  Conv nodes. Validated on NeuroGolf 2026 (7228 baseline, independent toolkit).
---

# ONNX Minimal Network Design

## Problem
Neural network golf competitions require the **smallest possible ONNX network**
that correctly implements a transformation. Score = `max(1, 25 - ln(memory + params))`.

Smaller networks → exponentially higher scores. The key insight: **memory
dominates cost** (intermediate tensor bytes), not parameters.

## Solution

### Core Scoring Formula
```
memory = Σ(bytes of all intermediate tensors)  # input/output are FREE
params = Σ(initializer element counts)
score  = max(1, 25 - ln(memory + params))
```

### Design Principle 1: Single-Node = Zero Memory

A single-node graph (1 Conv/Gather/Transpose) has **zero intermediate memory**
(output tensor is free). Only params count.

**Implication**: A 1-node Conv with 900 params (cost=900, score=18.2) beats a
4-node net with 200 params + 5000 memory (cost=5200, score=16.3).

### Design Principle 2: Conv is a Linear Classifier

A Conv node maps 3×3 one-hot neighborhoods → output channels. This is a
**linear classifier** on 90-dim features (9 positions × 10 colors).

To find Conv weights for a local rule:
1. Collect all (3×3 neighborhood, output color) pairs from training data
2. Verify the rule is deterministic (same neighborhood → same output)
3. Train LogisticRegression (multi-class, C=1e4, solver='newton-cg') on the pairs
4. Extract coef_ as Conv W[10,10,3,3], intercept_ as B[10]
5. Verify ONNX runtime output matches ground truth on ALL examples

**Limitation**: If LogReg accuracy < 100%, the rule is NOT linearly separable.
Need nonlinear ops (Cast/Equal/Where) — a single Conv cannot implement it.

### Design Principle 3: Gather for Color Permutations

A pixel-wise color map (e.g., swap color 3↔4) is a 1-node Gather:
- idx = [0,5,6,4,3,1,2,7,9,8] (10-element lookup table)
- Gather(input, idx, axis=1) maps each one-hot vector to the permuted one
- Cost: 10 params, 0 memory → score ≈ 22.7

### Design Principle 4: Sparse Initializers REJECTED

Converting dense sparse weights to `sparse_initializer` reduces params but
**onnx.checker.check_model(full_check=True) rejects sparse Conv weights**:
```
W typestr: T, has unsupported type: sparse_tensor(float)
```

Only use sparse for ops that explicitly support it (not Conv/MatMul in practice).

## Independent Toolkit Architecture

```python
# Grid encoder: ARC grid [[c,...]] → one-hot [1,10,30,30] float32
grid_to_onehot(grid, size=30)

# ONNX builders:
design_gather_colormap(color_map)  # 1-node Gather, 10 params
design_transpose()                  # 1-node Transpose, 0 params  
design_border(color=8)             # 1-node Conv border detection

# Verifier: onnxruntime on ALL train + arc-gen examples
verify(model_bytes, task_json)

# Scorer: compute params + memory → points
score_model(model_bytes)
```

## Evidence (NeuroGolf 2026)
- task171 border: independently designed 1-Conv (19/19 correct, matches existing bundle)
- task222: LogReg solver achieved 99.95% (not 100% — rule is nonlinear)
- 87 tasks had sparse-able weights but sparse rejected by strict checker
- 400-task scan: all simple rules already 1-node in community bundle
brainstormingobra/superpowersYou MUST use this before any creative work - creating features, building components, adding functionality, modifying behavior, or planning anything new, in software or out of it (a talk, a business, a renovation).Stars: 322kUpdated Claude CodeCodexVerifiedInstall
brand-guidelinesanthropics/skillsApplies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. Use it when brand colors or style guidelines, visual formatting, or company design standards apply.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
frontend-designanthropics/skillsGuidance for distinctive, intentional visual design when building new UI or reshaping an existing one. Helps with aesthetic direction, typography, and making choices that don't read as templated defaults.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
mcp-builderanthropics/skillsGuide for creating high-quality MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools. Use when building MCP servers to integrate external APIs or services, whether in Python (FastMCP) or Node/TypeScript (MCP SDK).Stars: 198kUpdated Claude CodeCodexVerifiedInstall