matlab/matlab-agentic-toolkit / skills-catalog/matlab-data-import-and-analysis/matlab-choose-big-data-solution
matlab-choose-big-data-solution
VerifiedGuide users or agents to the correct MATLAB tool for processing large tabular data in file-based formats (CSV, Parquet, delimited text, spreadsheets, MDF) that may not fit in memory. Use when a user or agent mentions large files, big data, out-of-memory errors, OOM, scaling up, tall arrays, datastores, or needs to process multiple tabular files. Covers the decision between datastore + tall, datastore + transform, and parallel execution. Also use when a user or agent has working in-memory code (readtable, parquetread) that runs out of memory and needs a migration path. Also covers building custom datastore classes for proprietary or non-standard formats — use when the task requires subclassing matlab.io.Datastore, implementing a custom reader, building an extensible datastore, or integrating a new file format with tall arrays or parallel computing. Do NOT use for MAT files (use matfile instead).
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 18.8 KB |
| manifest.yaml | config | 576 B |
| references/custom-datastore/implementation.md | markdown | 20.0 KB |
| references/custom-datastore/mixin-decision-tree.md | markdown | 7.0 KB |
| references/custom-datastore/testing-guidelines.md | markdown | 5.2 KB |
| references/parquetread-to-datastore-migration.md | markdown | 2.2 KB |
| references/readtable-to-datastore-migration.md | markdown | 3.7 KB |
| references/tall-array-effective-use.md | markdown | 8.4 KB |
manifest.yaml
View on GitHub# Copyright 2026 The MathWorks, Inc.
schema-version: 1
license: https://www.mathworks.com/content/dam/mathworks/license/pmrl/license.md
scope: cross-product
version: "2.1"
human-description: Choose the right MATLAB tool for processing large tabular data that may not fit in memory.
matlab-release: ">=R2021a"
required-tools:
- evaluate_matlab_code
- run_matlab_test_file
- check_matlab_code
required-skills: []
required-products: []
optional-products:
- Parallel Computing Toolbox
- MATLAB Parallel Server
- Vehicle Network Toolbox
toolkit: matlab-agentic-toolkit
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| deprecation-and-migrationaddyosmani/agent-skillsManages deprecation and migration. Use when removing old systems, APIs, or features. Use when migrating users from one implementation to another. Use when migrating a database schema in production, such as renaming or dropping a column without downtime (expand/contract). Use when deciding whether to maintain or sunset existing code. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| performance-optimizationaddyosmani/agent-skillsOptimizes application performance across frontend, backend, queries, and databases. Use when performance requirements exist, when you suspect performance regressions, when Core Web Vitals or load times need improvement, when N+1 query patterns need fixing, or when profiling reveals bottlenecks. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| agent-cost-reportthedotmack/claude-memBelievable agent cost report for any period, default the last 7 full days PT, not counting today. Measured tokens from Claude Code transcripts priced at OpenRouter list prices (ESTIMATED), measured provider spend when a sanctioned source exists, note-taker cost separate, Timing-style HTML/PDF plus report.json, line-items.csv, evidence.json. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |
| agent-cost-reportthedotmack/claude-memBelievable agent cost report for any period, default the last 7 full days PT, not counting today. Measured tokens from Claude Code transcripts priced at OpenRouter list prices (ESTIMATED), measured provider spend when a sanctioned source exists, note-taker cost separate, Timing-style HTML/PDF plus report.json, line-items.csv, evidence.json. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |