matlab/matlab-agentic-toolkit / skills-catalog/image-processing-and-computer-vision/matlab-use-visual-inspection
matlab-use-visual-inspection
VerifiedBuild machine vision inspection systems with MATLAB Visual Inspection Toolbox. Covers the full arc from problem selection through deployment: anomaly detection (Student-Teacher, PatchCore, FastFlow, FCDD), object detection (YOLOX), counting (CounTR, shape matching), gauging and measurement (caliper, fitcircle, fitangle), blob analysis (regionprops, calibrated measurement), shape matching for fixturing and localization, synthetic data generation for training augmentation, and deployment to standalone apps, embedded C/C++, NVIDIA GPUs (TensorRT), NPUs (ONNX), and Beckhoff PLCs. Routes to rule-based tools when problems are geometric and deterministic, AI-based tools when defect appearance varies, and hybrid pipelines combining both.
Visual Inspection with MATLAB
Build machine vision inspection systems using Visual Inspection Toolbox. Select the right approach — rule-based, AI-based, or hybrid — based on problem characteristics, then implement, train, and deploy.
When to Use
Defect detection and classification
- Detecting anomalies on manufactured parts (scratches, dents, contamination)
- Training anomaly detectors with good-part images only (one-class learning)
- Detecting and localizing defects with bounding boxes (YOLOX)
- Classifying defect types or severity levels
Measurement and gauging
- Measuring widths, gaps, angles, diameters on parts
- Calibrated measurement in physical units (mm, degrees)
- Vision fixturing — locating parts and repositioning measurement tools
- Blob analysis — area, perimeter, shape metrics of segmented regions
Counting and localization
- Counting identical rigid parts via shape matching
- Few-shot counting with intra-class variation (CounTR)
- Locating parts by edge geometry for downstream inspection
Training data augmentation
- Generating synthetic labeled data via copy-paste augmentation
- Expanding small defect datasets for YOLOX training
Deployment
- Packaging inspection as standalone executables (MATLAB Compiler)
- Generating C/C++ for embedded/edge targets (MATLAB Coder)
- GPU-accelerated inference with TensorRT (GPU Coder)
- ONNX export for NPUs, AI accelerators, smart camera SDKs
- Deploying to Beckhoff TwinCAT industrial PLCs
When NOT to Use
- Camera acquisition, frame grabber configuration — use Image Acquisition Toolbox
- PLC communication, OPC UA, MQTT protocols — use Industrial Communication Toolbox
- General image processing unrelated to inspection (artistic filters, medical imaging)
Must-Follow Rules
Approach Selection
- Rule-based for geometric, deterministic problems — when defects are dimensional deviations or parts have known geometry, use caliper/fitcircle/matchshape. No training data needed
- AI-based for variable appearance — when defect appearance can't be enumerated, use anomaly detection or YOLOX. Requires training data
- Hybrid when parts move — use
matchshapeto fixture (locate + normalize pose), then AI on the cropped/warped region. Anomaly detectors are NOT rotation/scale invariant
Anomaly Detection
- Anomaly detectors require consistent pose — CNN backbones are not rotation/scale invariant. Fixture with
matchshape+imwarpto canonical orientation before training and inference - Choose detector by data availability — >100 good images:
studentTeacherAnomalyDetector; <100 images:patchCoreAnomalyDetector; have anomaly labels:fcddAnomalyDetector - Use
classifyfor inference — returns[tf, score, map]for anomaly map visualization
Object Detection (YOLOX)
- Use
AutoResize=falsefor full-resolution inference — prevents downscaling that loses small defects. Pair with tiled training when objects are small relative to image - Synthetic data benefits YOLOX most — copy-paste augmentation via
objectInsertionDatastoreis proven for detection tasks
Measurement
- Vision fixturing pattern —
matchshapelocates part → transform measurement positions to detected pose → measure at repositioned locations - Calibrated measurements require
imageToWorldPlane— converts pixel measurements to physical units using camera calibration images.geotrans.Warperfor production rectification — precompute the mapping once, apply per frame for maximum throughput
Deployment
- Always use
persistent+isemptyguard in codegen entry points — load model once, infer many times coder.loadDeepLearningNetworkfor DL models — notcoder.load(which is for structs/shape models)coder.Constantfor MAT file path — codegen requires compile-time constant paths- Pass detector objects directly to
exportONNXNetwork— never extract.Networkor.Backbonefirst. The object-level export includes pre/post-processing logic - ONNX for NPUs/AI accelerators; GPU Coder + TensorRT for NVIDIA — don't use ONNX for Jetson/Orin
Pipeline Construction
Machine vision inspections compose tools in sequence:
- Acquire — Get the image (Image Acquisition Toolbox)
- Fixture — Locate a stable reference feature (
matchshape) - Reposition — Transform measurement positions to detected pose
- Inspect — Apply measurement or classification at repositioned locations
- Decide — Compare to tolerances → pass/fail
- Communicate — Send results to PLC/SCADA (Industrial Communication Toolbox)
Common Hybrid Patterns
| Pattern | Rule-Based Part | AI Part |
|---|---|---|
| Fixture + classify | matchshape locates part, crops ROI | Anomaly detector on cropped region |
| Fixture + detect | matchshape locates part | YOLOX finds defects relative to fixture |
| Pose normalize + anomaly | matchshape + imwarp to canonical pose | Anomaly detector on normalized image |
| Segment + measure | Thresholding / morphology | regionprops on segmented blobs |
Primary Decision: Rule-Based vs. AI-Based
| Choose Rule-Based When | Choose AI-Based When |
|---|---|
| Defects have predictable, enumerable geometry | Defect appearance varies unpredictably |
| Lighting and positioning are tightly controlled | Background or part appearance varies |
| Need dimensional measurements (mm, degrees) | Need classification without precise measurement |
| Few or no labeled examples available | Have labeled training data (or can generate synthetic) |
| Require explainable, auditable decisions | Accuracy matters more than explainability |
Key Functions
| Function | Purpose | Toolbox | Available From |
|---|---|---|---|
shapemodel | Build edge-geometry model from template | Visual Inspection Toolbox | R2026b |
matchshape | Find model instances in search image | Visual Inspection Toolbox | R2026b |
caliper | Measure edge distances along a profile | Visual Inspection Toolbox | R2026b |
fitangle | Measure angle between edges | Visual Inspection Toolbox | R2026b |
fitcircle | Measure circle center and radius | Visual Inspection Toolbox | R2026b |
studentTeacherAnomalyDetector | One-class anomaly detection (>100 good images) | Visual Inspection Toolbox | R2026b |
patchCoreAnomalyDetector | One-class anomaly detection (few-shot, <100 images) | Visual Inspection Toolbox | R2026b |
fastFlowAnomalyDetector | Normalizing-flow anomaly detection | Visual Inspection Toolbox | R2026b |
fcddAnomalyDetector | Semi-supervised anomaly detection | Visual Inspection Toolbox | R2026b |
yoloxObjectDetector | Object/defect detection with bounding boxes | Visual Inspection Toolbox | R2026b |
counTRObjectCounter | Counting with intra-class variation (few-shot) | Visual Inspection Toolbox | R2026b |
objectInsertionDatastore | Synthetic labeled training data (copy-paste) | Visual Inspection Toolbox | R2026b |
regionprops | Blob area, perimeter, shape features | Image Processing Toolbox | R2006a |
imageToWorldPlane | Calibrated world-unit measurement | Computer Vision Toolbox | R2022b |
Decision Trees
"I need to detect defects"
| Situation | Approach |
|---|---|
| Defect appearance unpredictable; good-part images only | Anomaly detection |
| Classify defect types with bounding boxes; have labeled data | YOLOX object detection |
| Defects are geometric deviations from known shape | Shape matching + gauging |
| Dimensional out-of-tolerance (too wide, too narrow) | Caliper / fitcircle / fitangle |
"I need to measure parts"
| Situation | Approach |
|---|---|
| Widths, gaps, edge-to-edge distances | caliper |
| Angles between edges | fitangle |
| Hole diameters, arc radii | fitcircle |
| Area, perimeter, shape of segmented blobs | regionprops |
| Need measurements in mm (physical units) | Camera calibration + rectification |
| Part position varies image-to-image | Vision fixturing with matchshape |
"I need to count objects"
| Situation | Approach |
|---|---|
| Identical rigid parts, controlled background | shapemodel / matchshape |
| Intra-class variation, few labeled examples | CounTR (few-shot counting) |
| Multiple classes, complex scenes, have labels | YOLOX detection → count |
"I need to locate parts"
| Situation | Approach |
|---|---|
| Rigid part, known edge geometry | shapemodel / matchshape |
| Variable appearance, multiple classes | YOLOX detection |
"I don't have enough training data"
| Situation | Approach |
|---|---|
| Have segmented defect examples + clean images | Copy-paste synthetic data |
| Good-part images only (no defect labels) | Anomaly detection (one-class) |
| No data at all, defects are geometric | Rule-based (no training needed) |
"I need to deploy"
| Target | Path |
|---|---|
| Standalone app, full MATLAB features | MATLAB Compiler |
| Monitoring dashboard or UI app | MATLAB Compiler |
| Embedded CPU (x86, ARM) | MATLAB Coder |
| NVIDIA Jetson/Orin or dGPU | GPU Coder + TensorRT |
| NPU or AI accelerator | ONNX export |
| Smart camera with vendor SDK | ONNX export |
| Beckhoff industrial PLC | MATLAB Coder → TwinCAT |
Patterns
Each reference file contains executable code patterns, API calling conventions, and worked examples for its topic. Load the relevant reference before writing code.
Conventions
- Always: use machine vision terminology (rule-based, tool chaining, fixturing)
- Always: recommend the specific VIT function, not generic IPT building blocks
- Always: consider hybrid pipelines (rule-based fixturing + AI classification)
- Always: ensure anomaly detector receives images in consistent pose
- Always: use
persistent+isemptyfor model loading in codegen entry points - Always: pass detector objects directly to
exportONNXNetwork(never extract.Network) - Never: recommend AI when the problem is purely dimensional measurement
- Never: recommend rule-based when defect appearance is highly variable
- Never: use
coder.loadfor deep learning models (usecoder.loadDeepLearningNetwork) - Never: use ONNX export for NVIDIA targets (use GPU Coder + TensorRT)
- Prefer: anomaly detection when only good-part images are available
- Prefer: YOLOX when labeled defect examples with bounding boxes exist
- Prefer:
shapemodel/matchshapeover YOLOX for locating rigid parts - Prefer:
images.geotrans.Warperfor production rectification throughput - Prefer: MATLAB Compiler for dashboards and UI-based monitoring apps
References
| Load when... | Reference |
|---|---|
| Training or using anomaly detectors | references/detect-anomaly.md |
| Locating parts by shape, vision fixturing | references/match-shape.md |
| Training or using YOLOX object detectors | references/detect-objects.md |
| Counting objects (shape matching, CounTR, YOLOX) | references/count-objects.md |
| Measuring with caliper, fitcircle, fitangle | references/measure-gauging.md |
| Blob analysis, regionprops, calibrated measurement | references/blob-analysis.md |
| Deploying models (Compiler, Coder, TensorRT, ONNX) | references/deploy-inspection.md |
| Generating synthetic training data | references/synthetic-data.md |
Copyright 2026 The MathWorks, Inc.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 117k | Updated | Claude CodeCodex | Verified | Install |