matlab/matlab-agentic-toolkit / skills-catalog/ai-and-statistics/matlab-deploy-embedded-ai
matlab-deploy-embedded-ai
VerifiedDeploy AI models to embedded hardware using MathWorks tools (MATLAB, Simulink, Embedded Coder). Covers two workflow patterns: (1) MathWorks-native or imported models rebuilt as dlnetwork for lean hardware, (2) direct C/C++ code generation from PyTorch and LiteRT models. Both patterns support all targets (Cortex-M/A/R, x86, GPU). Trigger when: user wants to deploy AI to embedded targets; generate C/CUDA from neural networks; compress AI models for MCU; integrate AI in Simulink for system-level simulation; import PyTorch/ONNX/TensorFlow models for embedded deployment; optimize AI for resource-constrained hardware; or use loadPyTorchExportedProgram, loadLiteRTModel, importNetworkFromPyTorch, importNetworkFromONNX, importNetworkFromTensorFlow, importNetworkFromKeras, dlquantizer, exportNetworkToSimulink, or Embedded Coder with AI models.
references/pattern1/native-rebuild-preparation.md
View on GitHub# Weight Extraction and Native Rebuild Preparation
Extract weights from imported networks and prepare for the native-rebuild path. The
imported network provides weights and architecture information for constructing a
native `dlnetwork` suitable for codegen, quantization, and Simulink export.
## Import Step
Use the `/matlab-import-external-ai-model` skill for the import itself (PyTorch,
ONNX, TensorFlow/Keras). That skill handles the full import pipeline: preparing
the model in Python, choosing import arguments, troubleshooting, and numeric
validation against the source framework.
Once you have an imported `dlnetwork`, return here if the network contains layers that
do not support code generation (verify by attempting `codegen`). ONNX autogenerated
layers support codegen since R2025a and may not need a rebuild. PyTorch autogenerated
layers may not support codegen in R2026a — rebuild is typically required.
## Expanding NetworkLayer Containers
By default, `importNetworkFromPyTorch` produces nested `NetworkLayer` containers.
`NetworkLayer` is supported by `predict` and `exportNetworkToSimulink`, but
**you must call `expandLayers` before code generation** — `NetworkLayer` containers
are not supported by the codegen path. Always expand before attempting `codegen`:
- Required for code generation (MATLAB Coder / Embedded Coder)
- Also needed to inspect individual layer properties (LSTM OutputMode, etc.)
- Also needed to access weights by index from `net.Learnables`
- Also needed to compare layer counts between imported and native networks
Without `expandLayers`, the Learnables table references container layer names (e.g.,
`"SOC_LSTM:lstm"`) rather than individual layer names, and you cannot see the full
architecture via `net.Layers`.
```matlab
net = expandLayers(net);
```
## Inspecting Imported Weights
After import, examine the Learnables table to understand the weight structure:
```matlab
disp(net.Learnables)
% Columns: Layer | Parameter | Value
% Each row is one learnable tensor (weights, biases, etc.)
```
Example output for a 2-layer LSTM + FC head:
```
Layer Parameter Value
____________ ________________ _________________
'lstm_1' 'InputWeights' {256x5 single}
'lstm_1' 'RecurrentWeights' {256x64 single}
'lstm_1' 'Bias' {256x1 single}
'lstm_2' 'InputWeights' {256x64 single}
'lstm_2' 'RecurrentWeights' {256x64 single}
'lstm_2' 'Bias' {256x1 single}
'fc_1' 'Weights' {64x64 single}
'fc_1' 'Bias' {64x1 single}
'fc_2' 'Weights' {1x64 single}
'fc_2' 'Bias' {1x1 single}
```
## Weight Dimension Conventions
PyTorch and MATLAB use different dimension orderings. You must handle this when transferring.
### Fully Connected Layers
| Framework | Weight Shape | Convention |
|-----------|-------------|-----------|
| PyTorch | `[output, input]` | Row-major |
| MATLAB | `[output, input]` | Same -- usually no transpose needed |
When importing from ONNX, FC weights may need transposing:
```matlab
% ONNX MatMul stores [input, output]; MATLAB FC expects [output, input]
matlabW = onnxW';
```
Always verify by comparing `predict()` outputs after transfer.
### Convolutional Layers
| Framework | Weight Shape | Convention |
|-----------|-------------|-----------|
| PyTorch | `[out_channels, in_channels, H, W]` | OIHW |
| MATLAB | `[H, W, in_channels, out_channels]` | HWIO |
**Transpose needed:**
```matlab
% PyTorch [O,I,H,W] -> MATLAB [H,W,I,O]
matlabWeights = permute(pytorchWeights, [3, 4, 2, 1]);
```
### LSTM Layers
Both PyTorch and MATLAB LSTM layers use the same concatenated gate layout (4*hidden_size rows),
so weights usually transfer directly without reshaping. However, verify gate ordering:
| Framework | Gate Order |
|-----------|-----------|
| PyTorch | i (input), f (forget), g (cell), o (output) |
| MATLAB | Same order |
### Self-Attention (Transformers) -- QKV Splitting
ONNX-exported attention typically stores Q, K, V as a single concatenated matrix:
```matlab
% ONNX: QKV combined [inputDim, 3*embedDim]
% Split into Q, K, V:
qWeights = combinedWeights(:, 1:embedDim);
kWeights = combinedWeights(:, embedDim+1:2*embedDim);
vWeights = combinedWeights(:, 2*embedDim+1:3*embedDim);
```
## Identifying Custom Layers to Replace
After import, scan for non-standard layers that will need replacement for downstream
integration (code generation, Simulink export, quantization):
```matlab
for i = 1:numel(net.Layers)
L = net.Layers(i);
cls = class(L);
if startsWith(cls, "nnet.onnx.") || startsWith(cls, "nnet.pytorch.") || ...
startsWith(cls, "nnet.tensorflow.") || contains(cls, "PlaceholderLayer")
fprintf('CUSTOM: %3d %-35s [%s]\n', i, L.Name, cls);
else
fprintf(' OK: %3d %-35s [%s]\n', i, L.Name, cls);
end
end
```
Common custom layers by architecture:
| Architecture | Custom Layer Name Pattern | What It Does | Native Replacement |
|-------------|-------------------------|-------------|-------------------|
| Any with LSTM | `*_select_*` | h_n[-1] selection | `lstmLayer(..., 'OutputMode', 'last')` |
| Any with argmax | `*_max_to_*` | Argmax | Move to entry-point function |
| CNN (ShuffleNet, etc.) | `*_view_view_*` | Channel shuffle | `ChannelShuffleLayer` |
| CNN | `*_chunk_*` | Channel split | `ChannelSplitLayer` |
| CNN | `*_mean_*` | Global avg pool | `globalAveragePooling2dLayer` + `flattenLayer` |
| Transformer | `*_view_view_*` | Head reshape | `ChannelShuffleLayer` or native `selfAttentionLayer` |
## Weight Transfer Methods
### Method A: Learnables Table Matching (General)
Works when layer names in native and imported networks partially match:
```matlab
importLearn = netImport.Learnables;
for i = 1:height(netNative.Learnables)
lyr = netNative.Learnables.Layer{i};
prm = netNative.Learnables.Parameter{i};
idx = find(contains(importLearn.Layer, lyr) & strcmp(importLearn.Parameter, prm));
if ~isempty(idx)
netNative.Learnables.Value{i} = importLearn.Value{idx(1)};
else
warning('No match for %s/%s', lyr, prm);
end
end
```
### Method B: Explicit Index-Based Assignment (When Names Don't Match)
```matlab
% Extract specific weights by position from imported network
lstm1_IW = netImport.Learnables.Value{1}; % InputWeights
lstm1_RW = netImport.Learnables.Value{2}; % RecurrentWeights
lstm1_B = netImport.Learnables.Value{3}; % Bias
% Assign directly to native network's Learnables table by index
netNative.Learnables.Value{1} = lstm1_IW;
netNative.Learnables.Value{2} = lstm1_RW;
netNative.Learnables.Value{3} = lstm1_B;
```
**NOTE:** There is no public API to set individual learnables by name. Always assign
via the `net.Learnables.Value{idx}` table directly.
### Method C: Constructor-Time Weight Assignment
Set weights directly when constructing layers:
```matlab
fc1 = fullyConnectedLayer(50, 'Name', 'fc1', ...
'Weights', extractedWeights, 'Bias', extractedBias);
```
## Post-Import Validation (Codegen Compatibility)
After importing any external model, validate for code generation:
```matlab
% Analyze network architecture
analyzeNetwork(net)
% Check for placeholder, autogenerated, or unsupported layers
numLayers = numel(net.Layers);
isUnsupported = false(numLayers, 1);
for i = 1:numLayers
layerClass = class(net.Layers(i));
if contains(layerClass, "PlaceholderLayer") || ...
contains(layerClass, "nnet.pytorch") || ...
contains(layerClass, "nnet.tensorflow") || ...
contains(layerClass, "autogenerated")
warning("Layer '%s' (%s) may not support code generation.", ...
net.Layers(i).Name, layerClass);
isUnsupported(i) = true;
end
end
unsupportedLayers = find(isUnsupported);
if ~isempty(unsupportedLayers)
fprintf("Found %d layers that may not support code generation.\n", ...
numel(unsupportedLayers));
fprintf("See 'Handling Unsupported Custom Layers' section.\n");
end
% Test inference with sample data
sampleInput = dlarray(randn(1, numFeatures, "single"), "BC");
output = predict(net, sampleInput);
```
## Post-Transfer Verification (Numerical Equivalency Testing)
Always verify the rebuilt network produces identical outputs. This is a critical gate
before proceeding to compression.
### Validate Against the Original Framework
Compare the rebuilt native network's predictions against the original framework's
outputs on a set of test inputs. This end-to-end comparison is the primary
validation — it confirms that weights transferred correctly and architecture
reconstruction is faithful.
**Direct weight comparison** (checking individual weight tensors match between
imported and native networks) is useful as a debugging tool when end-to-end
validation fails, but is not a substitute for output-level verification.
### CRITICAL: dlarray Shape Convention for Sequence Models
For sequence models with format `"CBT"` (Channels x Batch x Time):
- A **single sequence** of length T with C features MUST be shaped `[C, 1, T]`
- **NEVER** pass shape `[C, T]` with format `"CBT"` — MATLAB will interpret T as the
batch dimension and set sequence length to 1, causing MinLength errors or wrong results
```matlab
% WRONG: [5 x 10] with "CBT" -> C=5, B=10, T=1
testInput = dlarray(seq', "CBT"); % BUG!
% CORRECT: [5 x 1 x 10] with "CBT" -> C=5, B=1, T=10
testInput = dlarray(reshape(seq', [numFeatures, 1, seqLen]), "CBT");
```
### CRITICAL: torch.export Models Have Fixed Dimensions
`.pt2` files created with `torch.export.export()` have **fixed input shapes** by default.
The sequence length is baked into the exported program unless `dynamic_shapes` was
explicitly specified during export. Always check the model's expected dimensions before
generating test data:
```python
ep = torch.export.load("model.pt2")
# Check input specs for fixed dimensions
print(ep.graph_signature.input_specs)
```
### Step 1: Propose Test Count
Before running tests, propose to the user how many tests to run and why:
- **Minimum:** 50 test inputs for simple models (MLP with few classes)
- **Recommended:** 100 test inputs for most models
- **Thorough:** 200+ for complex models (transformers, multi-output regression)
**Rationale for test count:**
- Cover all output classes (classification) or output range (regression)
- Include edge cases and boundary inputs
- Statistical confidence: 100 inputs gives ~95% confidence of catching 3%+ error rates
Present your proposal: "I propose running N numerical equivalency tests because [reason].
Does this work for you, or would you prefer a different number?"
### Step 2: Collect Reference Data from Original Model
For PyTorch models, generate reference test vectors using the bundled Python:
```matlab
% Generate reference data from PyTorch model
% Create a script: generate_reference_vectors.py
% Run via the configured project Python or MATLAB support-package bundled Python.
% See references/shared/environment-setup.md for discovery guidance.
```
For ONNX models, use the MATLAB ONNX import directly with known test inputs.
### Step 3: Run Equivalency Tests
```matlab
testInput = dlarray(single(randn(inputShape)), formatString);
yImport = predict(netImport, testInput);
yNative = predict(netNative, testInput);
maxErr = max(abs(extractdata(yImport(:)) - extractdata(yNative(:))));
cosine = dot(extractdata(yImport(:)), extractdata(yNative(:))) / ...
(norm(extractdata(yImport(:))) * norm(extractdata(yNative(:))));
fprintf('Max error: %.2e, Cosine similarity: %.10f\n', maxErr, cosine);
assert(maxErr < 1e-3, 'Weight transfer verification failed');
```
Run on the agreed number of test inputs (not just one). Numerical equivalence should
be < 1e-5 for identical architectures, < 1e-3 when replacement layers change operation
order slightly.
### Step 4: Report Results
After running all tests, present a summary table:
```
Numerical Equivalency Report: Original Model vs. MATLAB Rebuilt Model
================================================================
Tests run: N
Mean Absolute Error: X.XXe-Y
Max Absolute Error: X.XXe-Y
Cosine Similarity: 0.XXXXXXXXXX (avg)
Pass/Fail: PASS (all errors < threshold)
```
### Step 5: Load in Deep Network Designer
After successful verification, load the rebuilt model in Deep Network Designer:
```matlab
deepNetworkDesigner(netNative)
```
Announce: "The rebuilt model is now open in Deep Network Designer. Please inspect
the architecture and let me know when you're ready to proceed."
## Handling Unsupported Custom Layers (ONNX and TensorFlow)
If the imported network contains custom autogenerated layers that are not supported
for code generation, rebuild the network natively using `dlnetwork` and transfer
the learned weights.
### Step 1: Identify unsupported layers
```matlab
% List all layers and their classes
for i = 1:numel(net.Layers)
fprintf("%3d: %-30s %s\n", i, net.Layers(i).Name, class(net.Layers(i)));
end
```
### Step 2: Rebuild the network with native MATLAB layers
Map each imported layer to its native MATLAB equivalent:
```matlab
% Example: rebuild a simple network natively
layers = [
featureInputLayer(numFeatures)
fullyConnectedLayer(64)
reluLayer
fullyConnectedLayer(32)
reluLayer
fullyConnectedLayer(numClasses)
softmaxLayer
];
nativeNet = dlnetwork(layers);
```
### Step 3: Extract and transfer weights from the imported network
```matlab
% Get learnables from the imported network
importedLearnables = net.Learnables;
disp(importedLearnables);
% If the layer structure matches, copy all learnables at once
nativeNet.Learnables = net.Learnables;
% If layer names differ, copy weights layer by layer
% Example: copy weights from imported "dense_1" to native "fc_1"
importedWeights = net.Learnables(net.Learnables.Layer == "dense_1", :);
nativeIdx = find(nativeNet.Learnables.Layer == "fc_1");
nativeNet.Learnables.Value{nativeIdx(1)} = importedWeights.Value{1}; % Weights
nativeNet.Learnables.Value{nativeIdx(2)} = importedWeights.Value{2}; % Bias
```
### Step 4: Validate the rebuilt network
```matlab
% Compare outputs between imported and native networks
sampleInput = dlarray(randn(1, numFeatures, "single"), "BC");
importedOutput = predict(net, sampleInput);
nativeOutput = predict(nativeNet, sampleInput);
maxDiff = max(abs(extractdata(importedOutput) - extractdata(nativeOutput)), [], "all");
fprintf("Max output difference: %.6e\n", maxDiff);
if maxDiff < 1e-5
fprintf("Native network matches imported network.\n");
else
warning("Significant difference detected. Review weight mapping.");
end
```
## Important Notes for Workflow Pattern 1
- Imported models for lean hardware should be **small** (< 500 KB target)
- Large imported models will need compression in Phase 5
- Verify that the imported network uses layers supported by code generation
- Run `analyzeNetwork` to check code generation compatibility
Copyright 2026 The MathWorks, Inc.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 197k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |