Skip to content

matlab/matlab-agentic-toolkit / skills-catalog/ai-and-statistics/matlab-engineer-tabular-features

matlab-engineer-tabular-features

Unverified

Use when engineering or selecting the best features for single-response classification or regression in MATLAB, whatever the data's modality — for non-tabular data it routes extraction to a domain skill, then selects, assesses, and delivers on the resulting table. Not for multi-response problems, model training, or raw data acquisition.

FileKindSize
SKILL.mdskill md18.9 KB
manifest.yamlconfig570 B
references/assess.mdmarkdown16.7 KB
references/deliver.mdmarkdown13.0 KB
references/domain-routing.mdmarkdown18.1 KB
references/feature-pool.mdmarkdown10.9 KB
references/intake.mdmarkdown16.9 KB
references/internal-helpers.mdmarkdown12.2 KB
references/provider-protocol.mdmarkdown12.3 KB
references/report.mdmarkdown9.7 KB
references/reproduce-pool.mdmarkdown7.1 KB
references/select.mdmarkdown7.1 KB
scripts/BordaVoter.mother2.3 KB
scripts/ConsensusVoter.mother2.5 KB
scripts/MajorityVoter.mother2.0 KB
scripts/RRFVoter.mother1.9 KB
scripts/assessGenerationStability.mother11.9 KB
scripts/assessKFold.mother20.6 KB
scripts/assessSelectionStability.mother9.8 KB
scripts/baselineComparison.mother18.9 KB
scripts/buildRecipe.mother8.2 KB
scripts/chooseNumBins.mother5.0 KB
scripts/chooseStandardization.mother4.5 KB
scripts/computeNSelect.mother2.9 KB
scripts/createConsensusVoter.mother2.2 KB
scripts/describeFeatures.mother5.1 KB
scripts/expandBlockMap.pbinary440 B
scripts/featureSetQuality.mother12.3 KB
scripts/generateFeatureReport.mother67.2 KB
scripts/generateFeatureReportPdf.mother64.9 KB
scripts/generateFeatures.mother14.6 KB
scripts/hasReportGenerator.mother630 B
scripts/isDomainRecipe.mother1.4 KB
scripts/isStringEmpty.mother577 B
scripts/lambdaSequence.pbinary637 B
scripts/lassoFeatureRanking.pbinary971 B
scripts/macroF1Score.mother931 B
scripts/mrmrFeatureRanking.pbinary239 B
scripts/mutualInformationPanel.pbinary1.1 KB
scripts/ncaFeatureRanking.pbinary464 B
scripts/nogueiraStability.mother3.8 KB
scripts/oobFeatureRanking.pbinary414 B
scripts/plotSelectionConsensus.mother2.7 KB
scripts/plotSelectionDecision.mother6.3 KB
scripts/plotSelectionStability.mother10.2 KB
scripts/plotValidationBaseline.mother6.1 KB
scripts/printSummary.mother18.3 KB
scripts/profileForSplit.mother6.2 KB
scripts/rankAgreement.mother1.3 KB
scripts/rankFeatures.mother6.2 KB
scripts/reserveHoldoutForUser.mother5.5 KB
scripts/runConsensusSelection.mother9.4 KB
scripts/sampleSkewness.mother600 B
scripts/saveResults.mother2.1 KB
scripts/scoreFeaturePanel.pbinary1.6 KB
scripts/screenPredictors.mother13.3 KB
scripts/selectFeatures.mother2.3 KB
scripts/splitStrategy.mother5.4 KB
scripts/stratifiedDownsampleRows.mother2.2 KB
scripts/stratifiedSubsampleIndices.mother3.4 KB
scripts/transformFeatures.mother6.1 KB
scripts/validateRecipe.mother12.3 KB
scripts/writeInferenceScript.mother10.5 KB

scripts/runConsensusSelection.m

View on GitHub
function [SelectedNames, VoteTable, PanelInfo] = runConsensusSelection(TrainTbl, Response, ProblemType, Options)
%runConsensusSelection Rank a feature pool with a panel and cut via consensus.
%
%   [SelectedNames, VoteTable, PanelInfo] = runConsensusSelection(TrainTbl, ...
%       Response, ProblemType) ranks the engineered feature pool in TrainTbl with
%       a diverse ranker panel (scoreFeaturePanel), fuses the rankings with a
%       data-driven consensus voter, and cuts the fused score vector at its elbow.
%
%   Running several complementary selectors and fusing their verdicts by vote is
%   this skill's distinctive value -- it is deliberately NOT what gencfeatures /
%   genrfeatures do internally (they select with a single learner-tied criterion).
%
%   The single selection path is: rank -> consensus -> elbow. There is
%   deliberately no per-N wrapper search: the rankers disagree by design, so
%   there is no well-defined "optimal N" until consensus has merged them into one
%   score vector, and cutting that one vector is exactly the elbow.
%
%   VOTER CHOICE. By default ("auto") mean pairwise Spearman agreement across the
%   panel's rankings picks the fusion rule: high agreement (>= 0.7) -> Borda count
%   (positional, rewards consistent placement); lower agreement -> Reciprocal Rank
%   Fusion (robust when methods disagree, emphasizes each method's top picks).
%   Set VoterMethod to "borda"/"rrf"/"majority" to override this data-driven
%   choice. "majority" (binary top-K vote, order-blind) is an expert opt-in for
%   when you trust WHICH features each ranker picked but not the ORDER it picked
%   them in (e.g. tie-heavy significance filters) -- the agreement gate cannot
%   detect that, so it is never chosen automatically.
%
%   FEATURE COUNT. The number kept is the score elbow of the fused consensus
%   vector (computeNSelect): the point of maximum below-chord distance on the
%   sorted scores. A convex curve cuts at its knee; a linear or concave decline
%   has no knee and keeps the whole pool.
%
%   Inputs:
%     TrainTbl    - table of engineered TRAINING features + response column. The
%                   full mixed pool is ranked; numeric and categorical engineered
%                   features share one column universe (scoreFeaturePanel aligns
%                   every ranker to it), so the fused vote is well defined.
%     Response    - (1,1) string, response variable name in TrainTbl
%     ProblemType - (1,1) string, "classification" or "regression"
%     Options.ExcludeFeatures - (1,:) string names to drop before ranking (e.g.
%                   GenInfo.BinaryReliant WoE columns from a dummy binarization,
%                   which are meaningless once the multiclass response is restored)
%     Options.VoterMethod - (1,1) string fusion rule: "auto" (default, agreement
%                   picks Borda vs RRF), or an explicit "borda" / "rrf" /
%                   "majority" override
%     Options.TargetModel - (1,1) string declared model family, gating the ranker
%                   panel (see scoreFeaturePanel): "agnostic" (default, full five-
%                   ranker panel), or "linear"/"tree_ensemble"/"kernel_distance",
%                   each of which runs its own embedded probe plus mrmr + the
%                   univariate filter. Mirrors how gencfeatures ties its embedded
%                   selector to the target learner. The consensus vote is unchanged.
%
%   Outputs:
%     SelectedNames - (1,:) string of selected feature names
%     VoteTable     - table: Feature, Score, Rank, Selected (consensus order)
%     PanelInfo     - struct with fields:
%                       .Rankers       - (1,:) string ranker methods used
%                       .Skipped       - (1,:) string any rankers that were skipped
%                       .PanelMode      - (1,1) string "full" or "family:<name>",
%                                        which ranker panel the family gate selected
%                       .VoterMethod   - "borda" | "rrf" | "majority"
%                       .VoterSelectedBy - "auto" | "user" (how VoterMethod was set)
%                       .MeanAgreement - mean pairwise Spearman rho
%                       .NSelect       - number of features selected (the elbow)
%                       .PoolSize      - features ranked
%                       .Excluded      - names excluded before ranking
%                       .PredVars      - (1,p) string shared column universe, the
%                                        row order of RankMatrix
%                       .RankMatrix    - (p,M) double per-method ranks (row k =
%                                        PredVars(k), col m = Rankers(m); lower =
%                                        more important); feeds plotSelectionConsensus
%                       .Reasoning     - human-readable explanation

% Copyright 2026 The MathWorks, Inc.

    arguments
        TrainTbl table
        Response (1,1) string
        ProblemType (1,1) string {mustBeMember(ProblemType, ["classification","regression"])}
        Options.ExcludeFeatures (1,:) string = string.empty(1,0)
        Options.VoterMethod (1,1) string ...
            {mustBeMember(Options.VoterMethod, ["auto","borda","rrf","majority"])} = "auto"
        Options.TargetModel (1,1) string {mustBeMember(Options.TargetModel, ...
            ["agnostic","tree_ensemble","linear","kernel_distance"])} = "agnostic"
    end

    if ~ismember(Response, string(TrainTbl.Properties.VariableNames))
        error('runConsensusSelection:responseNotFound', ...
            'Response variable "%s" is not a column of the input table.', Response);
    end

    % --- Candidate universe: all engineered predictors, minus exclusions -------
    AllVars = string(TrainTbl.Properties.VariableNames);
    PredVars = setdiff(AllVars, Response, "stable");

    Excluded = intersect(PredVars, Options.ExcludeFeatures, "stable");
    PredVars = setdiff(PredVars, Options.ExcludeFeatures, "stable");

    if ~isempty(Excluded)
        fprintf('runConsensusSelection: excluding %d feature(s) before ranking (%s).\n', ...
            numel(Excluded), join(Excluded, ", "));
    end

    p = numel(PredVars);
    if p < 2
        error('runConsensusSelection:tooFewFeatures', ...
            'Need at least 2 candidate features to select; have %d.', p);
    end

    % --- Diverse ranker panel over the shared mixed-type universe --------------
    [Rankings, Rankers, PanelNotes] = scoreFeaturePanel(TrainTbl, Response, ProblemType, ...
        IncludeFeatures=PredVars, TargetModel=Options.TargetModel);

    % Per-method rank matrix for the consensus heatmap (plotSelectionConsensus).
    % Rankings{m} is a best-first permutation of 1:p over PredVars, so writing
    % 1:p into those positions gives each PredVars(k) its rank under method m.
    % Rows align to PredVars, columns to Rankers; surfaced on PanelInfo so the
    % plot needs no re-ranking.
    RankMatrix = zeros(p, numel(Rankings));
    for m = 1:numel(Rankings)
        RankMatrix(Rankings{m}, m) = 1:p;
    end

    % --- Voter: agreement picks Borda vs RRF, unless the caller overrides ------
    % MeanRho is always computed (the report narrates it) even when the caller
    % forces a method, so the two cases stay comparable.
    MeanRho = rankAgreement(Rankings);
    if Options.VoterMethod == "auto"
        VoterSelectedBy = "auto";
        if MeanRho >= 0.7
            VoterMethod = "borda";
        else
            VoterMethod = "rrf";
        end
    else
        VoterSelectedBy = "user";
        VoterMethod = Options.VoterMethod;
    end
    Voter = createConsensusVoter(VoterMethod);
    [RankedIdx, Scores] = Voter.rank(Rankings, p);

    % --- Cut the fused score vector at its elbow -------------------------------
    NSelect = computeNSelect(Scores, p);

    SelectedNames = PredVars(RankedIdx(1:NSelect));

    % --- Vote table (consensus order) ------------------------------------------
    ConsensusRank = zeros(p, 1);
    ConsensusRank(RankedIdx) = 1:p;
    SelectedMask = false(p, 1);
    SelectedMask(RankedIdx(1:NSelect)) = true;
    VoteTable = table(PredVars(:), Scores(:), ConsensusRank, SelectedMask, ...
        VariableNames={'Feature','Score','Rank','Selected'});
    VoteTable = sortrows(VoteTable, 'Rank', 'ascend');

    if VoterSelectedBy == "user"
        VoterClause = sprintf("voter=%s (user override; mean rho=%.2f)", VoterMethod, MeanRho);
    else
        VoterClause = sprintf("voter=%s (mean rho=%.2f)", VoterMethod, MeanRho);
    end
    if startsWith(PanelNotes.PanelMode, "family:")
        Family = extractAfter(PanelNotes.PanelMode, "family:");
        PanelClause = sprintf("%d rankers (%s, gated to the %s family probe + agnostic rankers)", ...
            numel(Rankers), join(Rankers, "+"), Family);
    else
        PanelClause = sprintf("%d rankers (%s)", numel(Rankers), join(Rankers, "+"));
    end
    CutClause = sprintf("elbow -> %d of %d", NSelect, p);
    Reasoning = sprintf("%s, %s; %s", PanelClause, VoterClause, CutClause);

    PanelInfo = struct( ...
        Rankers = Rankers, ...
        Skipped = PanelNotes.Skipped, ...
        PanelMode = PanelNotes.PanelMode, ...
        VoterMethod = VoterMethod, ...
        VoterSelectedBy = VoterSelectedBy, ...
        MeanAgreement = MeanRho, ...
        NSelect = NSelect, ...
        PoolSize = p, ...
        Excluded = Excluded, ...
        PredVars = PredVars, ...
        RankMatrix = RankMatrix, ...
        Reasoning = string(Reasoning));

    fprintf('Consensus selection: %d of %d features (voter=%s).\n', ...
        NSelect, p, VoterMethod);
end
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 466kUpdated Claude CodeCodexVerifiedInstall
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 466kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 319kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 319kUpdated Claude CodeCodexVerifiedInstall