matlab/agent-skills-playground / demos/embedded-ai-deployment/skills/embedded-ai-deployment
embedded-ai-deployment
VerifiedDeploy AI models to embedded hardware using MathWorks tools (MATLAB, Simulink, Embedded Coder). Covers two workflow patterns: (1) MathWorks-native or 3P-imported models rebuilt as dlnetwork for lean hardware (Cortex-M, DSP), (2) direct C/C++ code generation from PyTorch and LiteRT models for high-performance hardware (Cortex-A, x86, GPU). Trigger when: user wants to deploy AI to embedded targets; generate C/CUDA from neural networks; compress AI models for MCU/DSP; integrate AI in Simulink for system-level simulation; import PyTorch/ONNX/TensorFlow models for embedded deployment; optimize AI for resource-constrained hardware; or use loadPyTorchExportedProgram, importNetworkFromPyTorch, dlquantizer, exportNetworkToSimulink, or Embedded Coder with AI models.
references/pattern1/import-weight-extraction.md
View on GitHub# Model Import and Weight Extraction
The imported network is used **only as a weight source**. The network object itself is never
used for codegen, quantization, or Simulink export.
## Critical Rule
**NEVER** use legacy import functions. Always use the modern replacements.
| Legacy (BANNED) | Modern Replacement |
|-----------------------------|-------------------------------------|
| `importONNXNetwork` | `importNetworkFromONNX` |
| `importONNXLayers` | `importNetworkFromONNX` |
| `importTensorFlowNetwork` | `importNetworkFromTensorFlow` |
| `importTensorFlowLayers` | `importNetworkFromTensorFlow` |
| `importKerasNetwork` | `importNetworkFromTensorFlow` |
| `importKerasLayers` | `importNetworkFromTensorFlow` |
The modern functions produce `dlnetwork` objects directly, ensuring compatibility
with the rest of the Workflow Pattern 1 toolchain.
## Two Import Paths
### Path A: importNetworkFromPyTorch (from .pt / .pt2)
The model must be saved as a TorchScript trace (`.pt` file) or via `torch.export` (`.pt2`).
```matlab
% Basic import from TorchScript trace
net = importNetworkFromPyTorch("model.pt");
% With explicit input size
net = importNetworkFromPyTorch("model.pt", ...
InputSize=[1 3 224 224]);
% For CNN/ViT models, specify the PyTorch input size (NCHW):
net = importNetworkFromPyTorch('model.pt2', PyTorchInputSizes=[1 3 224 224]);
% Expand nested NetworkLayer containers into flat layers:
net = expandLayers(net);
```
**IMPORTANT:** By default, `importNetworkFromPyTorch` produces nested `NetworkLayer`
containers. You **MUST** call `expandLayers(net)` before:
- Inspecting individual layer properties (LSTM OutputMode, etc.)
- Accessing weights by index from `net.Learnables`
- Comparing layer counts between imported and native networks
Without `expandLayers`, the Learnables table references container layer names (e.g.,
`"SOC_LSTM:lstm"`) rather than individual layer names, and you cannot see the full
architecture via `net.Layers`.
**Alternative:** Use `PackageName` to generate custom layer classes instead of
`NetworkLayer` containers at import time:
```matlab
net = importNetworkFromPyTorch("model.pt", PackageName="myModel");
```
This creates a `+myModel/` package with individual custom layer `.m` files. These
custom layers may be preferable for code generation because they expose explicit
`predict()` methods that MATLAB Coder can analyze, rather than opaque `NetworkLayer`
wrappers. However, the native rebuild approach (extracting weights into standard
MATLAB layers) remains the most reliable path for embedded codegen.
**When to use:** When you have a `.pt` or `.pt2` file.
**Requires:** Deep Learning Toolbox Converter for PyTorch Models support package.
#### TorchScript Trace Requirement for PyTorch (.pt file)
Export from Python:
```python
import torch
model.eval()
example_input = torch.randn(1, num_features)
traced = torch.jit.trace(model, example_input)
traced.save("model.pt")
```
**IMPORTANT:** Do NOT run Python for users -- provide the snippet and wait for them to
execute it in their own Python environment.
### Path B: importNetworkFromONNX (from .onnx)
```matlab
% Basic import
net = importNetworkFromONNX("model.onnx");
% With input data format specification
net = importNetworkFromONNX("model.onnx", ...
InputDataFormats="BCSS");
% With target network type
net = importNetworkFromONNX("model.onnx", ...
TargetNetwork="dlnetwork");
```
**When to use:** When you have an `.onnx` file, or when `importNetworkFromPyTorch` fails.
**Requires:** Deep Learning Toolbox Converter for ONNX Model Format support package.
#### Common ONNX Issues
- **Unsupported operators:** Check `analyzeNetwork(net)` for placeholder layers.
Replace them with custom MATLAB implementations.
- **Dynamic shapes:** Specify fixed input sizes if the ONNX model uses dynamic axes.
- **Opset version:** MATLAB supports ONNX opset versions up to a specific level
per release. Check release notes for your MATLAB version.
## PyTorch Import Failure Fallback
If `importNetworkFromPyTorch` fails or produces custom autogenerated layers that
are **NOT supported for code generation** (check with `analyzeNetwork`):
1. **Do NOT attempt to fix the layers manually in MATLAB as a first step.**
2. Advise the user to export the network to ONNX format from Python instead.
3. Provide this Python code snippet for the user to run **in their own Python
environment** -- do NOT run this code for them:
```python
import torch
import torch.onnx
# Load the model
model = torch.load("model.pt")
model.eval()
# Create dummy input matching model's expected input shape
dummy_input = torch.randn(1, num_features) # Adjust shape to match your model
# Export to ONNX
torch.onnx.export(
model,
dummy_input,
"model.onnx",
opset_version=13,
input_names=["input"],
output_names=["output"],
dynamic_axes=None # Use fixed shapes for embedded deployment
)
print("Exported to model.onnx")
```
4. Tell the user: "Please run this Python code in your Python environment and
provide the resulting `.onnx` file. I cannot run Python code for you."
5. Once the user returns with the `.onnx` file, proceed with `importNetworkFromONNX`.
6. If the ONNX import also produces unsupported layers, follow the
"Handling Unsupported Custom Layers" section below.
## Importing from TensorFlow
```matlab
% Import a SavedModel directory
net = importNetworkFromTensorFlow("saved_model_dir");
% Import with specific output layer
net = importNetworkFromTensorFlow("saved_model_dir", ...
OutputLayerType="classification");
```
### Supported TensorFlow Formats
- **SavedModel format** (directory): Recommended
- **HDF5 format** (`.h5`): Also supported via SavedModel conversion
### Prerequisites for TensorFlow Import
- The model should be saved using `tf.saved_model.save()` or `model.save()` in
SavedModel format
- Export from Python:
```python
import tensorflow as tf
model.save("saved_model_dir")
```
**Requires:** Deep Learning Toolbox Converter for TensorFlow Models support package.
## Inspecting Imported Weights
After import, examine the Learnables table to understand the weight structure:
```matlab
disp(net.Learnables)
% Columns: Layer | Parameter | Value
% Each row is one learnable tensor (weights, biases, etc.)
```
Example output for a 2-layer LSTM + FC head:
```
Layer Parameter Value
____________ ________________ _________________
'lstm_1' 'InputWeights' {256x5 single}
'lstm_1' 'RecurrentWeights' {256x64 single}
'lstm_1' 'Bias' {256x1 single}
'lstm_2' 'InputWeights' {256x64 single}
'lstm_2' 'RecurrentWeights' {256x64 single}
'lstm_2' 'Bias' {256x1 single}
'fc_1' 'Weights' {64x64 single}
'fc_1' 'Bias' {64x1 single}
'fc_2' 'Weights' {1x64 single}
'fc_2' 'Bias' {1x1 single}
```
## Weight Dimension Conventions
PyTorch and MATLAB use different dimension orderings. You must handle this when transferring.
### Fully Connected Layers
| Framework | Weight Shape | Convention |
|-----------|-------------|-----------|
| PyTorch | `[output, input]` | Row-major |
| MATLAB | `[output, input]` | Same -- usually no transpose needed |
When importing from ONNX, FC weights may need transposing:
```matlab
% ONNX MatMul stores [input, output]; MATLAB FC expects [output, input]
matlabW = onnxW';
```
Always verify by comparing `predict()` outputs after transfer.
### Convolutional Layers
| Framework | Weight Shape | Convention |
|-----------|-------------|-----------|
| PyTorch | `[out_channels, in_channels, H, W]` | OIHW |
| MATLAB | `[H, W, in_channels, out_channels]` | HWIO |
**Transpose needed:**
```matlab
% PyTorch [O,I,H,W] -> MATLAB [H,W,I,O]
matlabWeights = permute(pytorchWeights, [3, 4, 2, 1]);
```
### LSTM Layers
Both PyTorch and MATLAB LSTM layers use the same concatenated gate layout (4*hidden_size rows),
so weights usually transfer directly without reshaping. However, verify gate ordering:
| Framework | Gate Order |
|-----------|-----------|
| PyTorch | i (input), f (forget), g (cell), o (output) |
| MATLAB | Same order |
### Self-Attention (Transformers) -- QKV Splitting
ONNX-exported attention typically stores Q, K, V as a single concatenated matrix:
```matlab
% ONNX: QKV combined [inputDim, 3*embedDim]
% Split into Q, K, V:
qWeights = combinedWeights(:, 1:embedDim);
kWeights = combinedWeights(:, embedDim+1:2*embedDim);
vWeights = combinedWeights(:, 2*embedDim+1:3*embedDim);
```
## Identifying Custom Layers to Replace
After import, scan for non-standard layers that will need replacement:
```matlab
for i = 1:numel(net.Layers)
L = net.Layers(i);
cls = class(L);
if contains(cls, 'Autogenerated') || contains(cls, '+')
fprintf('CUSTOM: %3d %-35s [%s]\n', i, L.Name, cls);
else
fprintf(' OK: %3d %-35s [%s]\n', i, L.Name, cls);
end
end
```
Common custom layers by architecture:
| Architecture | Custom Layer Name Pattern | What It Does | Native Replacement |
|-------------|-------------------------|-------------|-------------------|
| Any with LSTM | `*_select_*` | h_n[-1] selection | `lstmLayer(..., 'OutputMode', 'last')` |
| Any with argmax | `*_max_to_*` | Argmax | Move to entry-point function |
| CNN (ShuffleNet, etc.) | `*_view_view_*` | Channel shuffle | `ChannelShuffleLayer` |
| CNN | `*_chunk_*` | Channel split | `ChannelSplitLayer` |
| CNN | `*_mean_*` | Global avg pool | `globalAveragePooling2dLayer` + `flattenLayer` |
| Transformer | `*_view_view_*` | Head reshape | `ChannelShuffleLayer` or native `selfAttentionLayer` |
## Weight Transfer Methods
### Method A: Learnables Table Matching (General)
Works when layer names in native and imported networks partially match:
```matlab
importLearn = netImport.Learnables;
for i = 1:height(netNative.Learnables)
lyr = netNative.Learnables.Layer{i};
prm = netNative.Learnables.Parameter{i};
idx = find(contains(importLearn.Layer, lyr) & strcmp(importLearn.Parameter, prm));
if ~isempty(idx)
netNative.Learnables.Value{i} = importLearn.Value{idx(1)};
else
warning('No match for %s/%s', lyr, prm);
end
end
```
### Method B: Explicit Index-Based Assignment (When Names Don't Match)
```matlab
% Extract specific weights by position from imported network
lstm1_IW = netImport.Learnables.Value{1}; % InputWeights
lstm1_RW = netImport.Learnables.Value{2}; % RecurrentWeights
lstm1_B = netImport.Learnables.Value{3}; % Bias
% Assign directly to native network's Learnables table by index
netNative.Learnables.Value{1} = lstm1_IW;
netNative.Learnables.Value{2} = lstm1_RW;
netNative.Learnables.Value{3} = lstm1_B;
```
**NOTE:** The 4-argument form `setLearnables(net, layerName, paramName, value)` does NOT
exist. Always assign via the Learnables table directly.
### Method C: Constructor-Time Weight Assignment
Set weights directly when constructing layers:
```matlab
fc1 = fullyConnectedLayer(50, 'Name', 'fc1', ...
'Weights', extractedWeights, 'Bias', extractedBias);
```
## Post-Import Validation with analyzeNetwork
After importing any external model, always validate:
```matlab
% Analyze network architecture
analyzeNetwork(net)
% Check for placeholder, autogenerated, or unsupported layers
unsupportedLayers = [];
for i = 1:numel(net.Layers)
layerClass = class(net.Layers(i));
if contains(layerClass, "PlaceholderLayer") || ...
contains(layerClass, "nnet.onnx") || ...
contains(layerClass, "nnet.pytorch") || ...
contains(layerClass, "nnet.tensorflow") || ...
contains(layerClass, "autogenerated")
warning("Layer '%s' (%s) may not support code generation.", ...
net.Layers(i).Name, layerClass);
unsupportedLayers = [unsupportedLayers; i]; %#ok<AGROW>
end
end
if ~isempty(unsupportedLayers)
fprintf("Found %d layers that may not support code generation.\n", ...
numel(unsupportedLayers));
fprintf("See 'Handling Unsupported Custom Layers' section.\n");
end
% Test inference with sample data
sampleInput = dlarray(randn(1, numFeatures, "single"), "BC");
output = predict(net, sampleInput);
```
## Post-Transfer Verification (Numerical Equivalency Testing)
Always verify the rebuilt network produces identical outputs. This is a critical gate
before proceeding to compression.
### CRITICAL: Do NOT Use the Imported Model as Ground Truth
Custom autogenerated layers (e.g., `*_select_*`, `*_view_*`) may produce **incorrect
outputs** even though they appear to run without errors. For example, a `select` layer
intended to pick the last time step may output the full sequence unchanged.
**Always validate against PyTorch ground truth** (via bundled Python), NOT the imported
MATLAB model. The imported model is a weight source only — never trust its inference
outputs as reference.
### CRITICAL: dlarray Shape Convention for Sequence Models
For sequence models with format `"CBT"` (Channels × Batch × Time):
- A **single sequence** of length T with C features MUST be shaped `[C, 1, T]`
- **NEVER** pass shape `[C, T]` with format `"CBT"` — MATLAB will interpret T as the
batch dimension and set sequence length to 1, causing MinLength errors or wrong results
```matlab
% WRONG: [5 x 10] with "CBT" → C=5, B=10, T=1
testInput = dlarray(seq', "CBT"); % BUG!
% CORRECT: [5 x 1 x 10] with "CBT" → C=5, B=1, T=10
testInput = dlarray(reshape(seq', [numFeatures, 1, seqLen]), "CBT");
```
### CRITICAL: torch.export Models Have Fixed Dimensions
`.pt2` files created with `torch.export.export()` have **fixed input shapes** by default.
The sequence length is baked into the exported program unless `dynamic_shapes` was
explicitly specified during export. Always check the model's expected dimensions before
generating test data:
```python
ep = torch.export.load("model.pt2")
# Check input specs for fixed dimensions
print(ep.graph_signature.input_specs)
```
### Step 1: Propose Test Count
Before running tests, propose to the user how many tests to run and why:
- **Minimum:** 50 test inputs for simple models (MLP with few classes)
- **Recommended:** 100 test inputs for most models
- **Thorough:** 200+ for complex models (transformers, multi-output regression)
**Rationale for test count:**
- Cover all output classes (classification) or output range (regression)
- Include edge cases and boundary inputs
- Statistical confidence: 100 inputs gives ~95% confidence of catching 3%+ error rates
Present your proposal: "I propose running N numerical equivalency tests because [reason].
Does this work for you, or would you prefer a different number?"
### Step 2: Collect Reference Data from 3P Model
For PyTorch models, generate reference test vectors using the bundled Python:
```matlab
% Generate reference data from PyTorch model
% Create a script: generate_reference_vectors.py
% Run via the configured project Python or MATLAB support-package bundled Python.
% See references/shared/environment-setup.md for discovery guidance.
```
For ONNX models, use the MATLAB ONNX import directly with known test inputs.
### Step 3: Run Equivalency Tests
```matlab
testInput = dlarray(single(randn(inputShape)), formatString);
yImport = predict(netImport, testInput);
yNative = predict(netNative, testInput);
maxErr = max(abs(extractdata(yImport(:)) - extractdata(yNative(:))));
cosine = dot(extractdata(yImport(:)), extractdata(yNative(:))) / ...
(norm(extractdata(yImport(:))) * norm(extractdata(yNative(:))));
fprintf('Max error: %.2e, Cosine similarity: %.10f\n', maxErr, cosine);
assert(maxErr < 1e-3, 'Weight transfer verification failed');
```
Run on the agreed number of test inputs (not just one). Numerical equivalence should
be < 1e-5 for identical architectures, < 1e-3 when replacement layers change operation
order slightly.
### Step 4: Report Results
After running all tests, present a summary table:
```
Numerical Equivalency Report: 3P Model vs. MATLAB Rebuilt Model
================================================================
Tests run: N
Mean Absolute Error: X.XXe-Y
Max Absolute Error: X.XXe-Y
Cosine Similarity: 0.XXXXXXXXXX (avg)
Pass/Fail: PASS (all errors < threshold)
```
### Step 5: Load in Deep Network Designer
After successful verification, load the rebuilt model in Deep Network Designer:
```matlab
deepNetworkDesigner(netNative)
```
Announce: "The rebuilt model is now open in Deep Network Designer. Please inspect
the architecture and let me know when you're ready to proceed."
## Handling Unsupported Custom Layers (ONNX and TensorFlow)
If `importNetworkFromONNX` or `importNetworkFromTensorFlow` produces custom
autogenerated layers that are not supported for code generation, rebuild the
network natively in MATLAB using `dlnetwork` and transfer the learned weights.
### Step 1: Identify unsupported layers
```matlab
% List all layers and their classes
for i = 1:numel(net.Layers)
fprintf("%3d: %-30s %s\n", i, net.Layers(i).Name, class(net.Layers(i)));
end
```
### Step 2: Rebuild the network with native MATLAB layers
Map each imported layer to its native MATLAB equivalent:
```matlab
% Example: rebuild a simple network natively
layers = [
featureInputLayer(numFeatures)
fullyConnectedLayer(64)
reluLayer
fullyConnectedLayer(32)
reluLayer
fullyConnectedLayer(numClasses)
softmaxLayer
];
nativeNet = dlnetwork(layers);
```
### Step 3: Extract and transfer weights from the imported network
```matlab
% Get learnables from the imported network
importedLearnables = net.Learnables;
disp(importedLearnables);
% If the layer structure matches, copy all learnables at once
nativeNet.Learnables = net.Learnables;
% If layer names differ, copy weights layer by layer
% Example: copy weights from imported "dense_1" to native "fc_1"
importedWeights = net.Learnables(net.Learnables.Layer == "dense_1", :);
nativeIdx = find(nativeNet.Learnables.Layer == "fc_1");
nativeNet.Learnables.Value{nativeIdx(1)} = importedWeights.Value{1}; % Weights
nativeNet.Learnables.Value{nativeIdx(2)} = importedWeights.Value{2}; % Bias
```
### Step 4: Validate the rebuilt network
```matlab
% Compare outputs between imported and native networks
sampleInput = dlarray(randn(1, numFeatures, "single"), "BC");
importedOutput = predict(net, sampleInput);
nativeOutput = predict(nativeNet, sampleInput);
maxDiff = max(abs(extractdata(importedOutput) - extractdata(nativeOutput)), [], "all");
fprintf("Max output difference: %.6e\n", maxDiff);
if maxDiff < 1e-5
fprintf("Native network matches imported network.\n");
else
warning("Significant difference detected. Review weight mapping.");
end
```
## Important Notes for Workflow Pattern 1
- Imported models for lean hardware should be **small** (< 500 KB target)
- Large imported models will need compression in Phase 5
- Verify that the imported network uses layers supported by code generation
- Run `analyzeNetwork` to check code generation compatibility
- Each import function requires its corresponding **support package**:
- `importNetworkFromPyTorch` -- Deep Learning Toolbox Converter for PyTorch Models
- `importNetworkFromONNX` -- Deep Learning Toolbox Converter for ONNX Model Format
- `importNetworkFromTensorFlow` -- Deep Learning Toolbox Converter for TensorFlow Models
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |