matlab/matlab-agentic-toolkit / skills-catalog/image-processing-and-computer-vision/matlab-process-images
matlab-process-images
VerifiedLoad this first for any MATLAB task involving images, photos, scans, frames, volumes, or visual data — reading, writing, filtering, enhancing, denoising, sharpening, deblurring, thresholding, segmenting, masking, detecting, counting objects, removing backgrounds, measuring, registering, stitching, aligning, cropping, resizing, or transforming. Covers image I/O (TIFF, PNG, JPEG, RAW, HDR), histograms, edge detection, color, morphology, deep learning for images, quality assessment, and 3-D volumes. Also for image problems (dark, noisy, blurry, low contrast, uneven lighting, hazy, color cast) or data exploration (folder of images). Includes medical imaging I/O (DICOM, NIfTI, NRRD, medicalVolume) and large/blocked images (blockedImage, whole-slide). Skip if the user just wants you to look at an image, with no MATLAB work. Not for Simulink video-block models, C/C++/CUDA code generation or GPU deployment of a network, or point clouds. Hyperspectral and visual inspection have sibling skills.
references/segmentation-automated.md
View on GitHub# Automated Image Segmentation
Decision guidance for segmentation methods that require no user interaction — thresholding, clustering, watershed, automatic SAM, and deep learning pipelines.
> **Often loaded with:** `morphological-ops.md` (cleaning up masks), `analysis-measurement.md` (measuring segmented regions), `segmentation-interactive.md` (when automatic methods fail and user input is available).
## Method Selection
| Image type | Task | Constraints | Algorithm |
|-----------|------|-------------|-----------|
| 2-D gray | Binary, high contrast | Automatic | `imbinarize` (global) |
| 2-D gray | Binary, uneven lighting | Automatic | `imbinarize("adaptive")` |
| 2-D gray/color | Multi-level threshold | Automatic | `multithresh` + `imquantize` |
| 2-D gray/color | Multi-region, known K | Automatic | `imsegkmeans` |
| 2-D gray/color | Multi-region, unknown K | Automatic | `imsegisodata` (R2024b+) |
| 2-D/3-D | Splitting touching convex objects (cells, particles) | Have binary mask | Marker-controlled watershed |
| 2-D | Complex scene, zero-shot | No training data | `imsegsam` **(CVT+DLT)** (R2024b+) |
| 2-D/3-D | Pixel-level classes | Have labeled data | `semanticseg` pipeline **(CVT+DLT)** |
| 2-D | Instance (per-object masks) | Have labeled data | `maskrcnn` or `solov2` **(CVT+DLT)** |
| 3-D volume | Multi-region | Automatic | `imsegkmeans3` |
**When uncertain:** prefer simpler method first; escalate if it fails.
## Thresholding
### Global
```matlab
BW = imbinarize(I); % Otsu's method (default)
BW = imbinarize(I, 0.45); % Manual threshold
```
### Adaptive (Uneven Illumination)
```matlab
BW = imbinarize(I, "adaptive", Sensitivity=0.4);
```
### Multi-Level
```matlab
thresh = multithresh(I, 3); % 3 thresholds → 4 regions
L = imquantize(I, thresh);
```
## Marker-Controlled Watershed — Critical Pattern
**When to use:** Use watershed segmentation to separate touching objects or to segment regions from an image gradient. To reduce oversegmentation, modify the distance or gradient image using suitable foreground and background markers before applying watershed.
**When NOT to use:** For general scene segmentation, use SAM (`imsegsam`) or clustering. Superpixels over-segment uniformly (don't solve the "split touching blobs" problem).
**Applying `watershed` directly to an unmodified gradient image can cause severe oversegmentation.** For gradient-based watershed segmentation, first compute suitable foreground and background markers, then use `imimposemin` to constrain the minima before applying `watershed` (see gradient-based example below). For distance-based watershed, use `imhmin` to suppress shallow minima:
```matlab
% 1. Create binary mask
BW = imbinarize(I);
BW = imfill(BW, "holes");
% 2. Distance transform (peaks at object centers)
D = -bwdist(~BW);
% 3. Suppress shallow minima (prevents over-segmentation)
D = imhmin(D, 2);
% 4. Watershed
L = watershed(D);
L(~BW) = 0; % Remove background label
```
**Tuning `h` in `imhmin(D, h)`:** Choose `h` based on the scale of `D` and the depths of the minima that represent unwanted regions. Increasing `h` generally reduces oversegmentation, but a value that is too large can merge distinct objects. Tune using representative images and verify the resulting object boundaries.
### Alternative: Gradient-Based with `imimposemin`
When you don't have a clean binary mask:
```matlab
gmag = imgradient(im2double(I));
markers = imextendedmin(imcomplement(im2double(I)), 0.1);
gmag2 = imimposemin(gmag, markers);
L = watershed(gmag2);
L(L == 0) = 1; % Assign watershed ridge pixels to region 1
```
## Clustering — `imsegkmeans` vs `imsegisodata`
```matlab
% Known number of regions (grayscale uint8 input works directly)
L = imsegkmeans(I, 4);
% Multichannel input — double must be converted to single; uint8/uint16/int16 work directly
L = imsegkmeans(single(I), 4); % Use when I is double or single
% Unknown number of regions (auto-determines K)
[L, centers] = imsegisodata(I); % R2024b+
```
**Color images:** For color-based segmentation, convert to L\*a\*b\* before clustering — do not cluster in RGB:
```matlab
lab = rgb2lab(I);
L = imsegkmeans(single(lab), 4);
```
**With texture features** (Gabor filter bank):
```matlab
gBank = gabor(2.^(2:5), 0:45:135);
features = cat(3, single(I), single(imgaborfilt(im2gray(I), gBank)));
L = imsegkmeans(features, 5);
```
## SAM 2: Automatic Zero-Shot Segmentation (R2024b+)
Use `imsegsam` to automatically segment an entire image or objects within a specified ROI using SAM 2. By default, the function uses SAM 2; you can also specify a base SAM model.
Requires IPT Model for Segment Anything Model support package. Check availability:
```matlab
try
sam = segmentAnythingModel; % Verifies support package and required add-on are available
hasSAM = true;
catch
hasSAM = false; % Help the user install the support package
end
```
> **Do not use `which` alone** — it can return true even when required add-ons are missing. Attempt construction to confirm usability.
```matlab
[cc, scores] = imsegsam(I, MinObjectArea=500, ScoreThreshold=0.7);
% cc is a connected component struct (like bwconncomp output)
% cc.NumObjects, cc.PixelIdxList — use labelmatrix(cc) for a label map
```
**Alternatives when SAM is unavailable:** If the image contains regions that can be separated by intensity, color, or selected image features, consider `imsegisodata` when the number of clusters is unknown or `imsegkmeans` when you can specify the number of clusters. These methods are not direct substitutes for object-aware segmentation.
`imsegsam` is automatic only (no prompts). For point/box-prompted segmentation, see `segmentAnythingModel` in `segmentation-interactive.md`.
## Deep Learning Segmentation
For pixel-level classification with labeled training data, see `deep-learning-imaging.md` for the full pipeline. Quick inference pattern:
```matlab
result = semanticseg(testImage, net); % Returns categorical label map
```
### Instance Segmentation
For per-object masks (each object gets a unique ID), use Mask R-CNN or SOLOv2 **(CVT+DLT)**. SOLOv2 is more lightweight and often preferable when per-instance bounding boxes are not required:
```matlab
% SOLOv2 — lightweight instance segmentation
net = solov2("resnet50-coco");
[masks, labels, scores] = segmentObjects(net, testImage);
% Mask R-CNN — instance segmentation with bounding boxes
net = maskrcnn("resnet50-coco");
[boxes, scores, labels, masks] = detectObjects(net, testImage);
```
To convert polygon-labeled ground truth into training data for Mask R-CNN or SOLOv2, use `instanceSegmentationTrainingData` **(R2026b+)**.
## Preprocessing for Segmentation
| Problem | Fix | Function |
|---------|-----|----------|
| Noise | Smooth first | `imgaussfilt`, `medfilt2` |
| Low contrast | Enhance | `imadjust`, `adapthisteq` |
| Uneven illumination | Flatten background | `imtophat` or `imflatfield` |
| Color → cluster | Convert to L\*a\*b\* | `rgb2lab` then `imsegkmeans` on Lab channels |
### Adaptive Thresholding with Masked Regions
`imbinarize(I, "adaptive")` calls `adaptthresh` internally. When you need to modify the threshold map before binarizing, compute it explicitly with `adaptthresh` + `imbinarize(A, T)` instead.
When parts of the image are zeroed out (e.g., background outside a brain, removed ROI), `adaptthresh` uses those zeros in local neighborhood calculations, corrupting thresholds near the mask boundary. Setting masked pixels to NaN also fails (produces black blocks).
**Fix: post-process the threshold map** — let `adaptthresh` run, then interpolate over the affected region:
```matlab
T = adaptthresh(A, "NeighborhoodSize", M);
B2 = imdilate(mask, ones(M, M)); % Expand mask by neighborhood size
T2 = regionfill(T, B2); % Interpolate threshold values from surroundings
BW = imbinarize(A, T2);
```
This works because `regionfill` smoothly fills in threshold values from the unaffected boundary — the actual image zeros never influence the final threshold in the masked region.
## Displaying Segmentation Results
```matlab
% Direct overlay display (preferred — one step, interactive)
imageshow(gray, OverlayData=L);
% For generating an RGB array (e.g., saving to file or further processing)
rgb = labeloverlay(gray, L, Transparency=0.5);
```
Use `imageshow` with `OverlayData` to interactively inspect segmentation results. Use `labeloverlay` when you need an RGB overlay array for saving, compositing, or further processing.
## Conventions
- Always use `imbinarize` — never `im2bw` (not recommended) or manual `graythresh` + threshold
- For watershed, always suppress or constrain minima first — never `watershed(imgradient(I))` on an unmodified gradient
- For color segmentation, convert to L\*a\*b\* before clustering
- Prefer `imsegisodata` over manual ISODATA when cluster count is unknown
- Use `bwareafilt` or `imfill(BW, "holes")` to clean up after thresholding (see `morphological-ops.md`) — `imfill(BW)` without `"holes"` is interactive (opens a figure for seed point selection), not automatic
## Common Mistakes
| Mistake | Why it's wrong | Correct approach |
|---------|---------------|-----------------|
| `watershed(imgradient(I))` | Severe over-segmentation (thousands of regions) | Marker-controlled: `imhmin` or `imimposemin` first |
| `im2bw(I, level)` | Not recommended since R2016a | `imbinarize(I)` or `imbinarize(I, level)` |
| Clustering in RGB space | RGB not perceptually uniform | Convert to L\*a\*b\*: `imsegkmeans(single(rgb2lab(I)), K)` |
| `segnetLayers` or `fcnLayers` | Legacy / not recommended | Use `deeplabv3plus`, `unet`, or `solov2` (see `deep-learning-imaging.md`) |
| `imshow(I, OverlayData=L)` | `imshow` does not support `OverlayData` | `imageshow(I, OverlayData=L)` (R2024b+) |
----
Copyright 2026 The MathWorks, Inc.
----
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |