matlab/matlab-agentic-toolkit / skills-catalog/matlab-data-import-and-analysis/matlab-analyze-data
matlab-analyze-data
VerifiedAnalyze data using MATLAB. Use when the task involves tables, timetables, time-series data, numeric arrays, sensor matrices, or gridded data — including but not limited to exploring, row filtering, sorting, cleaning, transforming, aggregating, smoothing, padding, trimming, and answering questions about data. MATLAB provides extensive, easy-to-use built-in functions for these workflows with no additional products required.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 13.0 KB |
| manifest.yaml | config | 492 B |
| references/answering-data-questions.md | markdown | 4.8 KB |
| references/array-and-grid-data.md | markdown | 19.6 KB |
| references/data-cleaning.md | markdown | 10.5 KB |
| references/data-transformation.md | markdown | 13.4 KB |
| references/data-types.md | markdown | 9.8 KB |
| references/eventtables.md | markdown | 12.4 KB |
| references/exploration.md | markdown | 10.0 KB |
| references/grouping-and-aggregation.md | markdown | 11.3 KB |
| references/smoothing-and-trends.md | markdown | 8.4 KB |
| references/tables-and-timetables.md | markdown | 19.0 KB |
references/exploration.md
View on GitHub# Data Exploration
**Contents:** first look (head/summary/size), missing data assessment, variable distributions, relationships (corrcoef/pivot), uniqueness/cardinality, duplicates, outlier screening, time-based exploration
Strategies for systematic data exploration: understanding structure, identifying patterns, summarizing distributions, and forming hypotheses before deeper analysis.
## General Guidance
- **Explore before transforming.** Understand the data before cleaning or reshaping it. Premature cleaning can mask important signals (e.g., removing "outliers" that are actually a meaningful subgroup).
- **Check your assumptions.** If a variable is called "Age", verify it contains plausible age values. Sentinel values (-1, 999, 0) are common.
- **Start broad, then narrow.** Use `summary` and `groupcounts` to get the big picture before drilling into specific rows or subsets.
- **Document what you find.** If exploring interactively, note surprises: unexpected missing patterns, outliers, duplicate keys, or variables that don't contain what their name suggests.
- **Add value where AI helps most.** Text-heavy tasks like populating `VariableDescriptions`, `VariableUnits`, or renaming variables to be more descriptive are a natural fit for AI assistance. Offer to enrich table metadata when it is sparse or missing.
## 1. First Look
Start with structure and shape before any analysis. **Prefer wrapping exploration output with `jsonencode` — particularly for tables and structs.** The default displays are better for human readers, but it is easy for agents to misinterpret which values belong to which variables in plain-text table output. JSON gives unambiguous, structured results:
```matlab
jsonencode(summary(T)) % per-variable stats as nested struct
jsonencode(head(T)) % first 8 rows as structured JSON
```
`summary` has two modes:
- **Display mode** (no output captured): prints human-readable stats to the command window.
- **Struct mode** (output captured or passed to a function, as in `jsonencode(summary(T))`): returns a struct whose field names are the variable names. Each field contains `Size`, `Type`, and `NumMissing`, plus type-specific statistics listed below.
Default statistics by variable type:
- **numeric, datetime, duration:** `Min`, `Median`, `Max`, `Mean`, `Std`
- **categorical (ordinal):** `Min`, `Median`, `Max`, plus category counts
- **categorical (non-ordinal):** category counts only
- **logical:** true/false counts only
- **string, calendarDuration, other:** `NumMissing` only (no additional stats)
**This single call already provides variable names, types, dimensions, missing counts, and descriptive statistics — AVOID recomputing these with `varfun(@class,...)`, `sum(ismissing(...))`, or similar.**
For standalone scripts intended for human readers, leave the semicolon off to invoke the `display` method — it shows dimensions, variable names, and a truncated preview. Avoid `disp` (omits headers and prints every row, flooding output on large tables) and `fprintf` in a loop (verbose, old-style):
```matlab
head(T) % first 8 rows - see what the data looks like
summary(T) % per-variable: type, stats, missing counts
size(T) % [nRows, nVars] -- not necessary if also calling summary()
T.Properties.VariableNames % variable names -- not necessary if also calling summary()
T.Properties.VariableTypes % programmatic access to types -- not necessary if also calling summary()
T % dimensions + header + truncated preview
```
`summary` is the single most informative exploration command. It always includes the basics for every variable: Size, Type, and Description (when available). It also shows a default set of per-variable descriptive statistics (min, median, max for numeric; counts for categorical), and missing value counts. For wide tables it is more useful than `head` since it shows every variable without truncation. From R2024b, use `Statistics`, `DataVariables`, and `Detail` name-value arguments to customize:
```matlab
summary(T, Statistics=["mean" "std" "min" "max"], DataVariables=vartype("numeric")) % R2024b+
summary(T, Statistics="allstats") % everything available; silently skips stats that don't apply to a given type
summary(T, Detail="high") % show all metadata (descriptions, units, category counts)
```
Note: `Detail="high"` only affects the printed display. When capturing output (`s = summary(T)`), the returned struct always includes all metadata regardless of `Detail`.
## 2. Missing Data Assessment
If you already called `summary(T)`, each variable's struct field includes `NumMissing` — you do not need `sum(ismissing(T))` to repeat that count. Use the functions below for targeted missing-data questions if the full `summary(T)` has not been already calculated:
```matlab
anymissing(T) % quick check: any missing values at all?
allfinite(T) % quick check: all values finite? (catches NaN and Inf)
sum(ismissing(T)) % missing count per variable
sum(ismissing(T)) ./ height(T) * 100 % missing percentage per variable
T.Properties.VariableNames(sum(ismissing(T)) > 0) % which variables have any missing
```
Look for patterns: is missingness concentrated in a few variables, or scattered? Are rows with missing values in one variable also missing in others? This informs whether to fill, drop, or ignore.
```matlab
% Variables with >50% missing — candidates for dropping
pctMissing = sum(ismissing(T)) ./ height(T) * 100;
T.Properties.VariableNames(pctMissing > 50)
```
## 3. Variable Distributions
Summarize differently depending on type:
**Numeric variables:**
```matlab
% Or per-variable
[min(T.Age) median(T.Age) max(T.Age) std(T.Age)]
prctile(T.Price,[25 50 75]) % quartiles
```
**Categorical/string variables:**
```matlab
groupcounts(T,"Status") % value counts with percentages
numunique(T.City) % number of distinct values (R2025a+)
countcats(T.Category) % counts for each defined category
categories(T.Category) % list defined category names
```
### Density estimation
Use `kde` (base MATLAB, R2023b) for smooth univariate density estimates:
```matlab
% Basic density estimate
[f, xf, bw] = kde(T.Price);
% Positive data with plug-in bandwidth — good for multimodal or skewed distributions
% Support: "unbounded" (default), "positive", "nonnegative", "negative", or [L U]
% Bandwidth: "normal-approx" (default), "plug-in", or positive scalar
[f, xf] = kde(T.ResponseTime, Support="positive", Bandwidth="plug-in");
% Cumulative distribution function
% ProbabilityFcn: "pdf" (default) or "cdf"
[f, xf] = kde(T.Value, ProbabilityFcn="cdf", Support=[0 100]);
```
Also accepts `Kernel` (`"normal"`, `"box"`, `"triangle"`, `"parabolic"`, or function handle), `EvaluationPoints` or `NumPoints` to control where the estimate is evaluated, and `Weight` for weighted observations.
For bivariate density, censored data, or survival functions (`"survivor"`, `"cumhazard"`, `"icdf"`), use `ksdensity` (requires Statistics and Machine Learning Toolbox).
## 4. Relationships Between Variables
These are examples of common techniques, not an exhaustive list. Choose what is appropriate for the data and question — consider other approaches beyond these based on the dataset's characteristics:
```matlab
% Pearson correlation (corrcoef is base MATLAB)
% Use Rows="complete" to drop rows with any NaN (default propagates NaN)
R = corrcoef(T{:,vartype("numeric")}, Rows="complete");
% Cross-tabulation of two categorical variables
pivot(T, Rows="Department", Columns="Status", Method="count")
% Group means - does a numeric variable differ across groups?
groupsummary(T,"Region","mean","Revenue")
```
For Kendall or Spearman rank correlation, use `corr` (requires Statistics and Machine Learning Toolbox) with the `Type` name-value argument. When complete cases are required and you need to explicitly remove rows, see the `rmmissing` caution in [data-cleaning.md](data-cleaning.md).
## 5. Uniqueness and Cardinality
Understanding cardinality helps decide how to use a variable (group by it? filter on it? ignore it?):
```matlab
% Unique value counts for all variables at once
summary(T, Statistics="numunique")
% Check if a variable could serve as an ID (all unique, no missing)
allunique(T.ID) && ~anymissing(T.ID) % allunique: R2025a+
```
## 6. Duplicates
```matlab
% Find duplicate rows (all variables)
[~,idx] = unique(T,"rows","stable");
dupIdx = setdiff(1:height(T),idx);
T(dupIdx, :) % inspect duplicate rows
% Duplicates in a key variable
[~,ia] = unique(T.CustomerID,"stable");
dupKeys = T.CustomerID(setdiff(1:height(T),ia));
T(ismember(T.CustomerID,dupKeys), :) % all rows with duplicated keys
```
## 7. Outlier Screening
Use `isoutlier` for quick detection:
```matlab
isoutlier(T.Price) % default: median + 3 scaled MAD
isoutlier(T.Price,"quartiles") % IQR-based (1.5x IQR beyond Q1/Q3)
isoutlier(T.Price,"grubbs") % Grubbs' test (assumes normality)
isoutlier(T.Price,"movmedian",20) % sliding window (good for time series with drift)
% Per-group outlier detection (via groupfilter)
T = groupfilter(T,"Department",@(x) ~isoutlier(x),"Salary");
```
See [data-cleaning.md](data-cleaning.md) for handling outliers after detection.
## 8. Time-Based Exploration
For timetables or tables with datetime variables:
```matlab
[min(TT.Time) max(TT.Time)] % date range
TT.Time(end) - TT.Time(1) % total span
% Check regularity and get step size
[tf,step] = isregular(TT); % timetables, datetime, and duration arrays
[tf,step] = isuniform(x); % numeric arrays
issorted(TT.Time,"strictascend") % check for monotonically increasing timestamps
% Quick resample to see trends
daily = retime(TT,"daily","mean");
monthly = retime(TT,"monthly","mean");
```
---
Copyright 2026 The MathWorks, Inc.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 197k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |