Skip to content

matlab/matlab-agentic-toolkit / skills-catalog/signal-processing/matlab-prepare-signal-data

matlab-prepare-signal-data

Verified

Use this skill when conditioning, loading, preparing, or labeling signal data for analysis or ML training. Covers: cleaning a single signal (fill gaps, remove drift, deoutlier, denoise, resample/align a time base) BEFORE analysis; building a `signalDatastore` pipeline; creating a `labeledSignalSet` for Signal Labeler; deriving labels (filename, folder, in-file, ROI, time-frequency ROI); stratified train/val/test splits; framing long signals; parallel processing; and shaping datastore output for `trainnet`. Triggers include "clean up this signal", "remove drift / detrend", "fill gaps", "remove spikes / outliers", "denoise", "resample to a uniform rate", "align channels", "labels from filenames", "stratified split", "prepare for Signal Labeler", and function names like `fillgaps`, `fillmissing`, `detrend`, `filloutliers`, `smoothdata`, `resample`, `synchronize`, `signalDatastore`, `labeledSignalSet`, `filenames2labels`, `folders2labels`, `splitlabels`, `framesig`, `framelbl`, `createDatastores`.

FileKindSize
SKILL.mdskill md9.5 KB
manifest.yamlconfig650 B
references/fn-countlabels.mdmarkdown2.1 KB
references/fn-detrend.mdmarkdown4.6 KB
references/fn-filenames2labels.mdmarkdown4.8 KB
references/fn-fillgaps.mdmarkdown3.3 KB
references/fn-fillmissing.mdmarkdown4.2 KB
references/fn-filloutliers.mdmarkdown5.0 KB
references/fn-folders2labels.mdmarkdown3.4 KB
references/fn-framelbl.mdmarkdown7.1 KB
references/fn-framesig.mdmarkdown4.1 KB
references/fn-hampel.mdmarkdown3.3 KB
references/fn-labeledsignalset.mdmarkdown9.1 KB
references/fn-labelspectrogramoptions.mdmarkdown4.6 KB
references/fn-medfilt1.mdmarkdown3.3 KB
references/fn-partition.mdmarkdown4.6 KB
references/fn-resample.mdmarkdown4.6 KB
references/fn-retime.mdmarkdown4.7 KB
references/fn-sgolayfilt.mdmarkdown3.5 KB
references/fn-signaldatastore.mdmarkdown11.9 KB
references/fn-signallabeldefinition.mdmarkdown5.9 KB
references/fn-signalmask-getmask.mdmarkdown8.6 KB
references/fn-smoothdata.mdmarkdown4.8 KB
references/fn-splitlabels.mdmarkdown5.3 KB
references/fn-subset.mdmarkdown3.9 KB
references/fn-synchronize.mdmarkdown4.7 KB
references/fn-wdenoise.mdmarkdown5.2 KB
references/wf-align-channels.mdmarkdown6.6 KB
references/wf-custom-readfcn.mdmarkdown5.0 KB
references/wf-denoise.mdmarkdown2.6 KB
references/wf-detrend-smooth-deoutlier.mdmarkdown6.6 KB
references/wf-envelope.mdmarkdown5.3 KB
references/wf-frame-and-label.mdmarkdown4.7 KB
references/wf-handoff-to-dl.mdmarkdown9.1 KB
references/wf-label-and-export.mdmarkdown8.7 KB
references/wf-load-and-split.mdmarkdown4.6 KB
references/wf-parallel-process.mdmarkdown4.8 KB
references/wf-repair-missing.mdmarkdown6.7 KB
references/wf-uniform-rate.mdmarkdown4.2 KB

references/wf-parallel-process.md

View on GitHub
# Workflow: parallel processing across a parpool

> Functions used: `signalDatastore`, `partition`, `parfor`

Compute per-signal results across multiple workers when serial processing
is too slow. The canonical pattern is **one datastore shard per worker**,
not one datastore per file.

> **Off-ramp.** If you only need a train/val/test split (single worker),
> this isn't your workflow → see wf-load-and-split.md (uses `subset`, not
> `partition`).

## Recipe

### Step 1 — Bootstrap the parallel pool

You need `pool.NumWorkers` **before** the `parfor` to call
`partition(ds, N, k)`. `parfor` will spin up an implicit pool, but you
can't read `NumWorkers` until the pool exists. Bootstrap explicitly:

```matlab
pool = gcp("nocreate");
if isempty(pool)
    pool = parpool;
end
N = pool.NumWorkers;
```

If files are very uneven in size, ask the datastore for a
size-aware partition count instead:

```matlab
N = numpartitions(sds, pool);
```

`numpartitions` returns the datastore's recommended shard count given the
pool — usually `pool.NumWorkers`, but it can suggest more shards when
file sizes are skewed.

### Step 2 — Partition the datastore by worker

```matlab
results = cell(1, N);
parfor k = 1:N
    workerDs = partition(sds, N, k);
    workerOut = [];
    while hasdata(workerDs)
        s = read(workerDs);
        workerOut(end+1) = rms(s); %#ok<AGROW>
    end
    results{k} = workerOut;
end
```

Each worker gets a disjoint shard (~`numel(sds.Files)/N` files); the
union of all shards is exactly `sds.Files`. The shard is a real
`signalDatastore`, so all the usual `read` / `hasdata` / `readall`
operations work.

**Split is by alphabetical block, not round-robin.** Shard 1 gets the
alphabetically-first files, shard N gets the last. If your files are
class-grouped by name (e.g. `cat_*.mat` then `dog_*.mat`), one class can
end up entirely on one worker. Don't assume `partition` randomizes.

About `workerOut(end+1) = rms(s); %#ok<AGROW>`: `workerOut` is a
worker-local temporary, so the AGROW pragma is harmless. Don't import
this pattern to a `parfor`-sliced output array — `parfor` doesn't allow
those to grow.

### Step 3 — Reduce results

```matlab
results = [results{:}];   % concatenate per-worker arrays
```

If each worker produces a more complex shape (struct, table), use the
appropriate concat (`vertcat`, `outerjoin`, etc.) in step 3.

## Anti-pattern — datastore per file

Don't construct a fresh datastore inside the `parfor` body for each file.
That spawns `N=files` datastores (defeating the partition abstraction)
and serializes file metadata work onto every worker.

```matlab
% Bad — datastore per file:
parfor i = 1:numel(sds.Files)
    workerSds = signalDatastore(sds.Files{i});   % one fresh per file
    s = read(workerSds);
    results(i) = rms(s); %#ok<AGROW>
end

% Good — datastore per worker:
parfor k = 1:N
    workerDs = partition(sds, N, k);
    while hasdata(workerDs)
        s = read(workerDs);
        % ...
    end
end
```

## Anti-pattern — eager-load inside `ReadFcn`

If the datastore uses a custom `ReadFcn`, eager-loading inside it
(`load(file)` then discarding) wastes per-worker memory. Use named load or
`matfile`. See fn-signaldatastore.md.

## Gotchas

- **Shards are by file count, not data size.** Files with very uneven
  length give uneven worker runtime. For balanced parallelism, group
  files into bins of equal total samples first, or use
  `numpartitions(sds, pool)` for the size-aware shard count.
- **Scalar accumulator breaks on multichannel signals.**
  `workerOut(end+1) = rms(s)` assumes `rms(s)` is scalar. If `s` is
  multichannel (matrix), `rms(s)` returns a row vector and the row-write
  errors or silently broadcasts depending on shape. Either constrain to
  one channel (`workerOut(end+1) = rms(s(:,1));`) or use cell
  accumulation (`workerOut{end+1} = rms(s);`) and concatenate at the
  end.
- **Two senses of `partition`.** This workflow uses `partition(ds, N, k)`
  (parallel sense). The split sense uses `subset(ds, idx)` — see
  fn-subset.md and fn-partition.md disambiguator.

## Worked end-to-end example

```matlab
sds = signalDatastore("dataset", FileExtensions=".mat", ...
    SignalVariableNames="x");

pool = gcp("nocreate");
if isempty(pool)
    pool = parpool;
end
N = pool.NumWorkers;

results = cell(1, N);
parfor k = 1:N
    workerDs = partition(sds, N, k);
    workerOut = [];
    while hasdata(workerDs)
        s = read(workerDs);
        workerOut(end+1) = rms(s); %#ok<AGROW>
    end
    results{k} = workerOut;
end
results = [results{:}];
```

## Next in the chain

- **Building the datastore in the first place** → wf-load-and-split.md
  step 1.
- **Per-frame supervision** → wf-frame-and-label.md (you can also
  parallelize the framing transform — same partition pattern applies).

----

Copyright 2026 The MathWorks, Inc.

----
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 197kUpdated Claude CodeCodexVerifiedInstall