Skip to content

matlab/matlab-agentic-toolkit / skills-catalog/ai-and-statistics/matlab-deploy-embedded-ai

matlab-deploy-embedded-ai

Verified

Deploy AI models to embedded hardware using MathWorks tools (MATLAB, Simulink, Embedded Coder). Covers two workflow patterns: (1) MathWorks-native or imported models rebuilt as dlnetwork for lean hardware, (2) direct C/C++ code generation from PyTorch and LiteRT models. Both patterns support all targets (Cortex-M/A/R, x86, GPU). Trigger when: user wants to deploy AI to embedded targets; generate C/CUDA from neural networks; compress AI models for MCU; integrate AI in Simulink for system-level simulation; import PyTorch/ONNX/TensorFlow models for embedded deployment; optimize AI for resource-constrained hardware; or use loadPyTorchExportedProgram, loadLiteRTModel, importNetworkFromPyTorch, importNetworkFromONNX, importNetworkFromTensorFlow, importNetworkFromKeras, dlquantizer, exportNetworkToSimulink, or Embedded Coder with AI models.

references/pattern1/compression.md

View on GitHub
# Model Compression and Quantization

Patterns for compressing AI models for deployment on resource-constrained embedded hardware.
Compression reduces flash/RAM footprint for flash-constrained targets. Compression
requirements differ by technique:

- **Quantization (`dlquantizer`):** requires a **native dlnetwork** with built-in layers —
  imported networks with custom autogenerated layers are not supported.
- **Projection (`compressNetworkUsingProjection`):** works on imported networks **as long
  as the learnables live in built-in layers** (e.g., `fullyConnectedLayer`, `lstmLayer`,
  `convolution2dLayer`). The more common blocker is **unformatted dlarray inputs**:
  imported networks frequently end up with input formats labelled `"UU"` or `"UUU"`,
  which the projection PCA step cannot interpret. Workarounds typically involve wrapping the
  imported network so the input is presented with a proper format (`"CB"`, `"CBT"`,
  `"SSCB"`, etc.) before calibration.
- **Pruning (`compressNetworkUsingTaylorPruning`):** also expects a native dlnetwork.

For maximum compatibility, rebuilding as a native dlnetwork remains the safest path.

**Choose the compression path before applying techniques.** The right combination of
pruning, projection, and quantization depends on the user's hardware target, deployment
goal (flash vs SRAM vs latency vs accuracy), and retraining tolerance. Load
[`compression-decision.md`](compression-decision.md) at the start of Phase 5 and walk
the user through the question flow before applying any technique. Compression is not
mandatory and is not always optimal — for example, on ARM Cortex-M with a latency-bound
LSTM model, the float32 path with CMSIS-DSP matrix-multiply replacement outperforms a
quantized path because CMSIS-NN provides no INT8 kernels for recurrent layers.

## Compression Paths by Model Type

| Model Type              | Compression Toolbox                            | Primary Techniques         |
|--------------------------|------------------------------------------------|----------------------------|
| `dlnetwork`             | Deep Learning Toolbox Model Compression Library | Quantization, Pruning, Projection |
| `fitcnet` / `fitrnet`  | Fixed-Point Designer                           | Fixed-point conversion     |

**CRITICAL:** Do NOT implement pruning, projection, or quantization in custom MATLAB
code. Always use the official MathWorks functions listed below. These functions are
part of the Deep Learning Toolbox Model Compression Library and are designed for
correctness, code generation compatibility, and integration with the deployment toolchain.

## Official Compression Functions Reference

| Technique    | Key Functions                                                              |
|--------------|---------------------------------------------------------------------------|
| Pruning      | `compressNetworkUsingTaylorPruning`                                       |
| Projection   | `compressNetworkUsingProjection` (calls `neuronPCA` internally; precompute separately only for multi-goal sweeps), `ProjectedLayer`, `lstmProjectedLayer`, `gruProjectedLayer` |
| Quantization | `prepareNetwork`, `dlquantizer`, `dlquantizationOptions`, `calibrate`, `quantize`, `validate`, `quantizationDetails`, `estimateNetworkMetrics` |

## Compression Techniques Overview

The table below summarizes the **mechanism** of each technique. **Flash, SRAM, and
accuracy outcomes are model-dependent** — especially for structural compression
(pruning and projection), where the same reduction goal can produce very different
results across architectures and datasets. Always measure on the actual model with
held-out data before committing to a path.

| Technique | Function | Flash impact | SRAM (activation memory) impact | Accuracy impact | Retraining |
|-----------|----------|--------------|-------------------------------|-----------------|------------|
| **Pruning** (structural) | `compressNetworkUsingTaylorPruning` | Reduces — removes filters, so weight matrices shrink | **Reduces** — fewer activations to store between layers | Model-dependent; iterative scoring + fine-tuning aims to hold accuracy within the user's budget | Required (between iterations) |
| **Projection** (structural) | `compressNetworkUsingProjection` | Reduces — low-rank factorization shrinks weight matrices | Does **not** reduce | Model-dependent; do a trade-off analysis | Typically required; modest goals can be tried without retraining as a trade-off pass |
| **INT8 quantization** (non-structural) | `dlquantizer` (`prepareNetwork`, `calibrate`, `quantize`) | Up to ~4× from the FP32 → INT8 weight storage drop, before per-layer overhead | Activation buffers also drop to INT8 (up to ~4× depending on shapes) | Model-dependent; usually small for CNN/FC, depends on output dynamic range and calibration set | Not required (post-training) |
| **Combined** (pruning and/or projection + quantization) | All of the above, in order | Multiplicative across techniques | Multiplicative (skip projection if SRAM is the goal) | Model-dependent; ~77% combined flash savings has been reported on representative CNN + FC examples | Required for the structural stages |

A separate manual INT8 (per-matrix min-max scaling) fallback is documented in
[`compression-quantization.md`](compression-quantization.md) for cases where
`dlquantizer` is not viable; it is a storage-only optimization that does not produce
integer arithmetic in generated code.

**No single recommended default.** Pick the technique based on the goal collected via
[`compression-decision.md`](compression-decision.md):

- **Flash-bound + retraining OK** → combined **pruning + projection + quantization**
  (each technique attacks a different source of flash; combining them is multiplicative)
- **Flash-bound + no retraining** → quantization, optionally with a conservative
  projection trade-off pass on overparameterized models (try the default
  `ExplainedVarianceGoal=0.95` first, or sweep `LearnablesReductionGoal` and
  measure accuracy)
- **SRAM-bound** → **pruning + quantization** (skip projection — it does not reduce
  activation memory)
- **Latency-bound** → hardware-dependent. Structural compression (pruning, projection)
  is also worth trying — fewer MACs typically means lower latency, and on MCU targets
  the structural savings can dominate kernel-replacement gains. On Cortex-M with
  recurrent layers, skip quantization for the LSTM/GRU portion (CMSIS-NN has no
  INT8 recurrent kernels; float32 + CMSIS-DSP is faster).
- **Accuracy-first** → try compression with accuracy gating before falling back to
  the float32 baseline. `compressNetworkUsingTaylorPruning` exposes a
  `ValidationThreshold` NVP that stops pruning as soon as the validation metric
  drops below the threshold; quantization with proper calibration often preserves
  accuracy on overparameterized models.

## Compression Order (Combined Pipeline)

For maximum compression, apply techniques in this order:

1. **Prune first** -- Remove unnecessary weights (structured weight removal)
2. **Project second** -- Reduce layer dimensions (`compressNetworkUsingProjection`)
3. **Quantize last** -- Convert to lower precision (INT8)

### When to Include Projection

`compressNetworkUsingProjection` supports `convolution1dLayer`
(since R2024b), `convolution2dLayer`, `fullyConnectedLayer`, `lstmLayer`, and
`gruLayer`. Include projection when:

- The user's goal is **flash reduction** (projection shrinks weight matrices), AND
- The user can tolerate **retraining / fine-tuning** (highly recommended after projection), AND
- The model contains at least one supported layer type, AND
- No projectable layer shares learnable parameters via weight tying (unsupported).

**Skip projection when:**

- The user's goal is purely latency with no flash constraint
- The user's goal is SRAM reduction only (projection does not shrink activation memory)
- Retraining is not acceptable
- The model has no supported layer types

### Reporting Compression Results

After each compression step, summarize the outcome (accuracy delta vs. the
uncompressed baseline, flash/SRAM savings) so the user can make a go/no-go
decision before proceeding. No fixed template is required.

**Anti-pattern: do not hardcode numeric values in print statements.** Always
read the values from MATLAB variables computed in the same run. If a script is
re-executed with different inputs or a different model, hardcoded numbers in
`fprintf`/`disp` strings will silently lie about the new results.

```matlab
% BAD — value is baked into the format string:
fprintf("Quantization MAE: 1.2e-3 (within budget)\n");

% GOOD — value is computed and printed from a variable:
mae = mean(abs(yBase(:) - yQuant(:)));
verdict = "FAIL";
if mae < budget
    verdict = "PASS";
end
fprintf("Quantization MAE: %.3e (budget: %.1e) — %s\n", mae, budget, verdict);
```

---

## Detailed Technique References

Each technique has its own reference file with full code examples and guidance:

- **[Pruning](compression-pruning.md)** — `compressNetworkUsingTaylorPruning` workflow
- **[Projection](compression-projection.md)** — `compressNetworkUsingProjection`, sequence models, codegen preparation
- **[Quantization](compression-quantization.md)** — `dlquantizer` pipeline, INT8 suitability check, manual fallback, combined pipeline
- **[Classical ML (fitcnet/fitrnet)](compression-classical-ml.md)** — Fixed-Point Designer with `generateLearnerDataTypeFcn`

---

## dlnetwork Compression: Deep Learning Toolbox Model Compression Library

This library is a **support package**, not a toolbox. It is NOT detected by
`detect_matlab_toolboxes`. To check availability, use:

```matlab
pkgs = matlabshared.supportpkg.getInstalled;
hasCompLib = false;
if ~isempty(pkgs)
    hasCompLib = any(contains({pkgs.Name}, "Model Compression"));
end
```

If not installed, **do not install on the user's behalf**. Direct the user to the
MATLAB Add-On Explorer to download it, then wait for confirmation before proceeding.

---

## Compression Validation

After any compression step, always validate:

```matlab
% Compare original vs. compressed accuracy using testnet
accOrig = testnet(net, XTest, TTest, "accuracy");
accComp = testnet(compressedNet, XTest, TTest, "accuracy");
fprintf("Original accuracy:   %.4f\n", accOrig);
fprintf("Compressed accuracy: %.4f\n", accComp);
fprintf("Accuracy drop:       %.4f\n", accOrig - accComp);

% Model size comparison using analyzeNetwork (works for all compression types)
naOrig = analyzeNetwork(net, Plots='none');
naComp = analyzeNetwork(compressedNet, Plots='none');
fprintf("Original learnables:   %d\n", naOrig.TotalLearnables);
fprintf("Compressed learnables: %d\n", naComp.TotalLearnables);
fprintf("Reduction: %.1f%%\n", (1 - naComp.TotalLearnables/naOrig.TotalLearnables) * 100);
```

---

## CRITICAL Rules Summary

1. **prepareNetwork(quantObj) before calibrate()** -- `prepareNetwork` takes the
   `dlquantizer` object (not a `dlnetwork`) and mutates it in place. It handles
   unfused batch normalization, certain LSTM state configurations, and unpacks
   projected layers into their underlying built-in layers. Without it, calibration
   may fail with fi() type errors on networks that need fusion.

2. **Projected-layer codegen** -- `lstmProjectedLayer` and `gruProjectedLayer`
   are supported by `coder.loadDeepLearningNetwork` for generic C/C++ codegen
   (no MKL-DNN / ARM Compute acceleration). A `ProjectedLayer` wrapper is
   supported only when its contents are stateless (conv/FC, or LSTM/GRU in
   stateful-I/O mode). When in doubt, call `unpackProjectedLayers()` before
   saving for the direct MATLAB Coder path — that produces the most
   codegen-friendly form. **Not required for the Simulink path in R2026a:**
   `exportNetworkToSimulink` accepts projected networks directly.

3. **quantize() output is deployable via Simulink, NOT via direct MATLAB Coder** --
   `coder.loadDeepLearningNetwork` does not accept the output of `quantize()`. For
   embedded deployment of a quantized network, use `exportNetworkToSimulink(qNet)`
   followed by `slbuild`. For the direct MATLAB Coder path, use the uncompressed,
   pruned, or unpacked projected float32 network.

4. **Use ExecutionEnvironment="MATLAB"** -- For target-agnostic C/C++ code generation, always
   set `ExecutionEnvironment` to `"MATLAB"` when creating the dlquantizer.

5. **Compression Library is a support package** -- It is NOT detected by
   `detect_matlab_toolboxes`. Use `matlabshared.supportpkg.getInstalled` to check.

---

## Workflow Pattern 1 Compression Targets

For lean hardware deployment (Cortex-M):

| Metric             | Target                  | Notes                              |
|--------------------|-------------------------|------------------------------------|
| Model size         | < 500 KB (ideally < 100 KB) | Flash/ROM constraint          |
| RAM usage          | < 64 KB                 | SRAM constraint for inference      |
| Inference time     | < 10 ms (application-dependent) | Real-time constraint        |
| Data type          | int8 or fixed-point     | No FPU on some targets             |
| Accuracy drop      | < 2% relative           | Application-dependent tolerance    |


Copyright 2026 The MathWorks, Inc.
writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 319kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 196kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 118kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall