Skip to content

matlab/matlab-agentic-toolkit / skills-catalog/ai-and-statistics/matlab-engineer-tabular-features

matlab-engineer-tabular-features

Unverified

Use when engineering or selecting the best features for single-response classification or regression in MATLAB, whatever the data's modality — for non-tabular data it routes extraction to a domain skill, then selects, assesses, and delivers on the resulting table. Not for multi-response problems, model training, or raw data acquisition.

FileKindSize
SKILL.mdskill md18.9 KB
manifest.yamlconfig570 B
references/assess.mdmarkdown16.7 KB
references/deliver.mdmarkdown13.0 KB
references/domain-routing.mdmarkdown18.1 KB
references/feature-pool.mdmarkdown10.9 KB
references/intake.mdmarkdown16.9 KB
references/internal-helpers.mdmarkdown12.2 KB
references/provider-protocol.mdmarkdown12.3 KB
references/report.mdmarkdown9.7 KB
references/reproduce-pool.mdmarkdown7.1 KB
references/select.mdmarkdown7.1 KB
scripts/BordaVoter.mother2.3 KB
scripts/ConsensusVoter.mother2.5 KB
scripts/MajorityVoter.mother2.0 KB
scripts/RRFVoter.mother1.9 KB
scripts/assessGenerationStability.mother11.9 KB
scripts/assessKFold.mother20.6 KB
scripts/assessSelectionStability.mother9.8 KB
scripts/baselineComparison.mother18.9 KB
scripts/buildRecipe.mother8.2 KB
scripts/chooseNumBins.mother5.0 KB
scripts/chooseStandardization.mother4.5 KB
scripts/computeNSelect.mother2.9 KB
scripts/createConsensusVoter.mother2.2 KB
scripts/describeFeatures.mother5.1 KB
scripts/expandBlockMap.pbinary440 B
scripts/featureSetQuality.mother12.3 KB
scripts/generateFeatureReport.mother67.2 KB
scripts/generateFeatureReportPdf.mother64.9 KB
scripts/generateFeatures.mother14.6 KB
scripts/hasReportGenerator.mother630 B
scripts/isDomainRecipe.mother1.4 KB
scripts/isStringEmpty.mother577 B
scripts/lambdaSequence.pbinary637 B
scripts/lassoFeatureRanking.pbinary971 B
scripts/macroF1Score.mother931 B
scripts/mrmrFeatureRanking.pbinary239 B
scripts/mutualInformationPanel.pbinary1.1 KB
scripts/ncaFeatureRanking.pbinary464 B
scripts/nogueiraStability.mother3.8 KB
scripts/oobFeatureRanking.pbinary414 B
scripts/plotSelectionConsensus.mother2.7 KB
scripts/plotSelectionDecision.mother6.3 KB
scripts/plotSelectionStability.mother10.2 KB
scripts/plotValidationBaseline.mother6.1 KB
scripts/printSummary.mother18.3 KB
scripts/profileForSplit.mother6.2 KB
scripts/rankAgreement.mother1.3 KB
scripts/rankFeatures.mother6.2 KB
scripts/reserveHoldoutForUser.mother5.5 KB
scripts/runConsensusSelection.mother9.4 KB
scripts/sampleSkewness.mother600 B
scripts/saveResults.mother2.1 KB
scripts/scoreFeaturePanel.pbinary1.6 KB
scripts/screenPredictors.mother13.3 KB
scripts/selectFeatures.mother2.3 KB
scripts/splitStrategy.mother5.4 KB
scripts/stratifiedDownsampleRows.mother2.2 KB
scripts/stratifiedSubsampleIndices.mother3.4 KB
scripts/transformFeatures.mother6.1 KB
scripts/validateRecipe.mother12.3 KB
scripts/writeInferenceScript.mother10.5 KB

references/internal-helpers.md

View on GitHub
# Internal helpers — interface reference

These are the leaf helpers under `scripts/` that the **documented entry points**
(intake, feature-pool, select, assess, deliver, report) call internally. The agent
does not invoke them by hand during a run — it drives the pipeline through the
entry points, which wire these together. This file exists so every shipped script
has its name, inputs, outputs, and an example call documented in Markdown: at
runtime the agent never needs to open a helper's source to use it.

Seven of these are **p-coded at packaging** (marked ⓟ below). P-coding hides the
implementation, not the interface — the signatures here remain the contract. Call
them exactly as documented; do not rely on any internal behavior beyond what is
stated.

Every helper follows the MathWorks Coding Guidelines (`arguments` blocks,
lowerCamelCase, `end` terminators).

---

## Consensus voters

The selection layer runs several ranking methods, then fuses their rankings into
one consensus order. `createConsensusVoter` is the factory; the three concrete
voters differ only in how they score a feature from its positions across methods.

### `createConsensusVoter`

```matlab
Voter = createConsensusVoter(Method)
Voter = createConsensusVoter(Method, Name=Value)
```

Factory that returns a voter instance with `rank()` and `select()` methods.

- **Method** — `(1,1) string`, one of `"borda"`, `"majority"`, `"rrf"`.
- **Name-Value** — forwarded to the chosen voter: `Weights` (borda), `TopK`
  (majority), `K` (rrf).
- **Returns** — a `ConsensusVoter` subclass instance.

```matlab
Voter = createConsensusVoter("rrf", K=10);
[RankedIdx, Scores] = Voter.rank({r1, r2, r3}, nFeatures);
[Selected, Tbl]     = Voter.select({r1, r2, r3}, FeatureNames, 10);
```

### `ConsensusVoter` (abstract base)

Base class for the voters. Not instantiated directly — use the factory. Defines
the `select(Rankings, FeatureNames, NumSelect)` method (rank, then take the top
`NumSelect`) shared by every subclass.

### `BordaVoter`

Consensus by Borda count: a feature at rank *k* of *N* earns *N−k+1* points,
summed across methods. Favors features that rank consistently high everywhere.

```matlab
Voter = BordaVoter();                 % uniform method weights
Voter = BordaVoter(Weights=[1 2 1]);  % trust method 2 more
[RankedIdx, Scores] = Voter.rank({[3 1 2], [1 3 2]}, 3);
```

- **Property `Weights`** — `(1,:) double`, one per ranking; empty = uniform.
- **`rank(Rankings, NumFeatures)`** — `Rankings` is a `(1,:) cell` of index
  vectors; `NumFeatures` is the feature-universe size. Returns best-first
  `RankedIdx` and per-feature `Scores`.

### `MajorityVoter`

Consensus by majority vote: each method's top-*K* features get one vote; score is
the vote count. Binary — ignores rank within the top-*K*.

```matlab
Voter = MajorityVoter(TopK=5);
[RankedIdx, Scores] = Voter.rank({[3 1 2 5 4], [1 3 5 2 4]}, 6);
```

- **Property `TopK`** — `(1,1) double`; `0` (default) auto-sets to
  `ceil(nFeatures/3)`.
- **`rank(...)`** — same signature as `BordaVoter.rank`.

### `RRFVoter`

Consensus by Reciprocal Rank Fusion: score = `sum(1 ./ (K + rank))` across
methods. The reciprocal damps outlier ranks, so one method ranking a feature very
low barely moves the fused score.

```matlab
Voter = RRFVoter(K=60);   % K=60 is the standard constant; smaller K = top ranks dominate
[RankedIdx, Scores] = Voter.rank({[3 1 2], [1 3 2]}, 3);
```

- **Property `K`** — `(1,1) double`, positive, default `60`.
- **`rank(...)`** — same signature as `BordaVoter.rank`.

---

## Feature-ranking forks

Each fork runs one supervised ranking method over the whole predictor set and
returns a **complete** best-first permutation (never truncated), so a voter can
fuse full rankings. All share the signature
`RankedIdx = <fork>(Texp, y, ProblemType)`:

- **Texp** — `table` of predictors (numeric and/or categorical).
- **y** — response vector aligned to `Texp` rows.
- **ProblemType** — `(1,1) string`, `"classification"` or `"regression"`.
- **RankedIdx** — `(1,width(Texp))` best-first column indices into `Texp`.

```matlab
RankedIdx = mrmrFeatureRanking(Texp, y, "classification");
```

| Fork | Ranks by |
|---|---|
| `mrmrFeatureRanking` ⓟ | minimum-redundancy maximum-relevance |
| `oobFeatureRanking` ⓟ | bagged out-of-bag permuted predictor importance |
| `ncaFeatureRanking` ⓟ | neighborhood component analysis feature relevance |
| `lassoFeatureRanking` ⓟ | order of entry along a lasso penalty path |

The four forks are tuned to match the feature-generation stage's own selection
configuration, so their rankings are faithful to it.

### `expandBlockMap` ⓟ

```matlab
[BlockMap, Xexp, yPrep] = expandBlockMap(X, y)
```

Dummy-expands categorical predictors the way the fit-family learners do, and maps
each expanded column back to its origin predictor — so a length-*E* coefficient
vector can be collapsed to per-feature scores.

- **X** — `table` of predictors, no response column.
- **y** — response vector (numeric-coded for a lasso path, e.g. `grp2idx(y)-1`).
- **BlockMap** — `(1,E) double`; `BlockMap(j)` is the origin predictor index of
  expanded column *j*.
- **Xexp** — `(N,E) double`, expanded numeric predictor matrix.
- **yPrep** — response aligned to `Xexp` rows (NaN-response rows removed).

```matlab
[BlockMap, Xexp] = expandBlockMap(X, grp2idx(y)-1);
Score = accumarray(BlockMap(:), abs(Coef(:)), [width(X) 1], @max, 0);
```

### `lambdaSequence` ⓟ

```matlab
lambda = lambdaSequence(Xexp, y, ProblemType)
```

Builds the log-spaced lasso penalty sequence used by `lassoFeatureRanking`. The
largest value is the smallest penalty that zeros every coefficient.

- **Xexp** — `(N,E) double`, expanded predictor matrix (from `expandBlockMap`).
- **y** — `(N,1) double`, numeric-coded response aligned to `Xexp` rows.
- **ProblemType** — `(1,1) string`, `"classification"` or `"regression"`.
- **lambda** — `(1,100) double`, ascending penalty sequence.

---

## Mutual information and scoring

### `mutualInformationPanel` ⓟ

```matlab
[Ix, Iy] = mutualInformationPanel(X, y, ProblemType)
[Ix, Iy] = mutualInformationPanel(X, y, ProblemType, Name, Value)
```

The single MI kernel behind the model-free relevance and redundancy reads. Bins
predictors and response consistently with the MRMR ranking method, then estimates
MI on the bin codes.

- **X** — `(N,P)` numeric matrix or table; numeric columns are adaptively binned,
  categorical/ordinal detected and re-indexed, missing handled as code 0.
- **y** — `(N,1)` response; grouped into class codes for classification, adaptively
  binned for regression.
- **ProblemType** — `"classification"` or `"regression"`.
- **Name-Value** — `Verbose` (nonnegative integer, default 0), `UseMissing`
  (logical, default false).
- **Ix** — `(P,P)` MI (nats) among features, symmetric; off-diagonal is the
  redundancy read.
- **Iy** — `(P,1)` MI (nats) between each feature and the response — the relevance
  read.

### `macroF1Score`

```matlab
F1 = macroF1Score(TrueY, Scores, ClassNames)
```

Macro-averaged F1: per-class max F1 across thresholds (via `rocmetrics`),
averaged over classes.

- **TrueY** — `(:,1)` categorical or numeric true labels.
- **Scores** — `(N,K)` posterior scores from `predict`.
- **ClassNames** — `(1,K)` / `(K,1)` class names (from `model.ClassNames`).
- **F1** — `(1,1)` scalar.

### `nogueiraStability`

```matlab
[Stability, Info] = nogueiraStability(Z)
```

Chance-corrected selection-stability index (Nogueira, Sechidis & Brown, JMLR 2018,
eq. 2). Scores how consistently a selector picks the same features across runs.
Handles variable per-run subset sizes and is feature-type agnostic.

- **Z** — `(K,p)` binary matrix over a fixed feature universe; `Z(i,j)=1` if
  feature *j* was selected in run *i*. `K≥2` runs, `p≥1` features.
- **Stability** — `(1,1)` index (typically `[0,1]`; may be slightly negative;
  `1` = identical every run, `~0` = no better than chance).
- **Info** — struct: `.SelectionFrequency (1,p)`, `.MeanSubsetSize`, `.NumRuns`,
  `.NumFeatures`.

---

## Ranking utilities

### `rankFeatures`

```matlab
[RankedIdx, Scores] = rankFeatures(TrainX, TrainY, Method)
[RankedIdx, Scores] = rankFeatures(TrainX, TrainY, Method, Name=Value)
```

Unified supervised ranking dispatcher. Accepts a table for methods that support
mixed numeric/categorical input; extracts numeric columns automatically for the
numeric-only methods.

- **TrainX** — predictor table or matrix; **TrainY** — `(:,1)` response.
- **Method** — `(1,1) string`, one of `"fscmrmr"`, `"fsrmrmr"`, `"relieff"`,
  `"fscnca"`, `"fsrnca"`, `"lasso"`, `"chi2"`, `"ftest"`, `"sequentialfs"`,
  `"permutation"`, `"predictorImportance"`, `"stepwiselm"`, `"stepwiseglm"`.
- **Name-Value** — `ProblemType` (`"classification"`|`"regression"`, default
  classification), `MethodOptions` (`struct`).
- **RankedIdx / Scores** — best-first indices and their scores.

### `rankAgreement`

```matlab
[MeanRho, PairwiseRho] = rankAgreement(Rankings)
```

Mean pairwise Spearman correlation among ranking vectors — used to pick the
consensus voter (high agreement, `MeanRho ≥ 0.7`, favors Borda; low favors RRF).

- **Rankings** — `(1,M) cell`, each a `(1,P)` best-first index vector.
- **MeanRho** — `(1,1)` scalar; **PairwiseRho** — `(M,M)` symmetric matrix.

---

## Data-prep utilities

### `chooseNumBins`

```matlab
[NumBins, Info] = chooseNumBins(X, ContinuousMask, NEff)
[NumBins, Info] = chooseNumBins(X, ContinuousMask, NEff, Options)
```

Data-driven bin count for the chi-square / F-test filters (`fscchi2` / `fsrftest`),
replacing the fixed default of 10. Uses the median per-column Freedman-Diaconis
count, capped so the sparsest contingency cell stays populated (Cochran's rule) and
floored to keep resolution.

- **X** — table or numeric matrix of predictors.
- **ContinuousMask** — `(1,p) logical`, true for continuous predictors to bin.
- **NEff** — `(1,1)` effective sample size, already reduced to the binding
  constraint (minority-class count for classification, row count for regression).
- **Options** — `MinPerCell` (default 5), `MinBins` (default 5).
- **NumBins** — `(1,1)` positive integer.
- **Info** — struct: `.FDBins`, `.MedianFDBins`, `.ValidityCap`, `.Reasoning`.

### `chooseStandardization`

```matlab
[Mode, Reasoning] = chooseStandardization(RawTbl, Response, TargetModel)
```

Resolves the generator's `TransformedDataStandardization` mode from the numeric
predictors and the declared model family. Returns a concrete mode (never `"auto"`).

- **RawTbl** — table of raw predictors + response.
- **Response** — `(1,1) string`, response variable name.
- **TargetModel** — `(1,1) string`, `"agnostic"` | `"tree_ensemble"` | `"linear"`
  | `"kernel_distance"`.
- **Mode** — `(1,1) string`: `"none"` (tree ensembles), `"mad"` (heavy-skew
  predictors dominate), or `"zscore"` (scale-sensitive families, low skew).
  `"range"` is reached only via explicit override, never auto-selected.
- **Reasoning** — `(1,1) string`, justification citing the measured skew fraction.

### `sampleSkewness`

```matlab
S = sampleSkewness(Col)
```

Sample skewness (standardized third central moment) of a numeric vector.

- **Col** — `(:,1)` numeric vector (NaNs pre-removed).
- **S** — `(1,1)` scalar (`0` = symmetric).

### `stratifiedDownsampleRows`

```matlab
KeepIdx = stratifiedDownsampleRows(Response, MaxRows)
```

Row indices trimming a table to at most `MaxRows` while preserving class
proportions — bounds the cost of a stability read on large data. Applied once, up
front, before any resampling.

- **Response** — `(n,1)` response vector; its type decides stratification
  (categorical/logical/string → per-class `Holdout` trim; continuous → random
  draw).
- **MaxRows** — `(1,1)` positive integer, default 3000.
- **KeepIdx** — `(:,1)` sorted row indices to keep (all rows when `n ≤ MaxRows`).

### `isStringEmpty`

```matlab
Tf = isStringEmpty(Index)
```

True when an optional `Index` argument was effectively omitted — the empty
string/char default the contract verbs (`transformFeatures`, `describeFeatures`)
use to mean "no subset requested, return the whole pool". Numeric, logical, or
non-empty string returns false.

- **Index** — the value under test; **Tf** — `(1,1) logical`.

----

Copyright 2026 The MathWorks, Inc.

----
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 470kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall