matlab/matlab-agentic-toolkit / skills-catalog/reporting-and-database-access/matlab-use-duckdb
matlab-use-duckdb
VerifiedUse DuckDB from MATLAB via Database Toolbox (R2026a+) as a non-math operations engine on large tabular files (CSV/Parquet/JSON) and as a zero-config embedded database. Use when connecting to DuckDB, querying CSV, Parquet, and JSON files directly with SQL, reducing or profiling large data before MATLAB analysis, creating portable development databases, or installing DuckDB extensions. Triggers on: DuckDB, duckdb(), large CSV/Parquet/JSON, file too large for readtable, filter/aggregate at source, deduplicate, reduce before analysis, profile large file, persistent file import, analytical engine, SQL on CSV, SQL on Parquet, SQL on JSON, query CSV with SQL, query Parquet with SQL, run SQL on files, SQL queries on files, query files directly, SQL without database, in-process SQL.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 15.2 KB |
| manifest.yaml | config | 621 B |
| reference/INDEX.md | markdown | 594 B |
| reference/cards/development-database.md | markdown | 2.6 KB |
| reference/cards/extensions.md | markdown | 1.8 KB |
| reference/cards/file-analytics.md | markdown | 3.4 KB |
| reference/cards/reduce-large-data.md | markdown | 3.2 KB |
reference/cards/file-analytics.md
View on GitHub# File Analytics with DuckDB
Query CSV, Parquet, and JSON files directly with SQL — no import step needed. DuckDB processes files on disk, so data larger than MATLAB memory can be filtered, aggregated, and joined before results enter the workspace.
## DuckDB SQL Functions for File Access
| SQL Function | File Type | Example |
|-------------|-----------|---------|
| `read_csv('path')` | CSV | `fetch(conn, "SELECT * FROM read_csv('data.csv')")` |
| `read_parquet('path')` | Parquet | `fetch(conn, "SELECT * FROM read_parquet('data.parquet')")` |
| `read_json('path')` | JSON | `fetch(conn, "SELECT * FROM read_json('events.json')")` |
| `read_parquet('dir/*.parquet')` | Multiple Parquet | Glob patterns for multiple files |
| `read_csv('dir/*.csv')` | Multiple CSV | Glob patterns for multiple files |
**Rules:**
- **ALWAYS** use `fetch` (not `sqlread`) for file queries — file queries require SQL syntax.
- **ALWAYS** use single quotes for file paths inside SQL: `read_csv('file.csv')`.
- File paths are relative to the MATLAB current directory (`pwd`). Use absolute paths if needed.
## Example: Query a CSV File
```matlab
conn = duckdb();
orders = fetch(conn, "SELECT customer_id, SUM(total) as revenue " + ...
"FROM read_csv('sales_2024.csv') " + ...
"GROUP BY customer_id " + ...
"HAVING SUM(total) > 10000 " + ...
"ORDER BY revenue DESC");
close(conn);
```
## Example: Query a Parquet File
```matlab
conn = duckdb();
sensors = fetch(conn, "SELECT sensor_id, AVG(temperature) as avg_temp " + ...
"FROM read_parquet('iot_readings.parquet') " + ...
"WHERE timestamp >= '2024-01-01' " + ...
"GROUP BY sensor_id");
close(conn);
```
## Example: Query Multiple Files with Glob Patterns
```matlab
conn = duckdb();
% All Parquet files in a directory
allSales = fetch(conn, "SELECT * FROM read_parquet('data/sales_*.parquet') " + ...
"WHERE region = 'North America'");
% All CSV files matching a pattern
logs = fetch(conn, "SELECT type, COUNT(*) as cnt " + ...
"FROM read_csv('logs/app_*.csv') " + ...
"GROUP BY type ORDER BY cnt DESC");
close(conn);
```
## Example: Out-of-Memory Data Preprocessing
When data is too large for MATLAB memory, use DuckDB to preprocess on disk and bring only the results into MATLAB.
```matlab
conn = duckdb();
% A 50GB Parquet file — cannot fit in memory
% DuckDB preprocesses on disk: filter + aggregate + sort
dailySummary = fetch(conn, ...
"SELECT date, region, " + ...
" SUM(revenue) as total_revenue, " + ...
" COUNT(*) as num_transactions, " + ...
" AVG(unit_price) as avg_price " + ...
"FROM read_parquet('transactions_2024.parquet') " + ...
"WHERE status = 'completed' " + ...
"GROUP BY date, region " + ...
"ORDER BY date");
% Result fits in memory — use MATLAB for advanced analysis
dailySummary.date = datetime(dailySummary.date);
figure;
stackedplot(dailySummary, ["total_revenue", "num_transactions", "avg_price"]);
title("Daily Sales Summary");
close(conn);
```
## Example: Join Two Large Files
```matlab
conn = duckdb();
% Join a large orders file with a customers file
result = fetch(conn, ...
"SELECT c.name, c.region, SUM(o.total) as lifetime_value " + ...
"FROM read_parquet('orders.parquet') o " + ...
"JOIN read_csv('customers.csv') c ON o.customer_id = c.id " + ...
"GROUP BY c.name, c.region " + ...
"ORDER BY lifetime_value DESC " + ...
"LIMIT 100");
close(conn);
```
----
Copyright 2026 The MathWorks, Inc.
----
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| deprecation-and-migrationaddyosmani/agent-skillsManages deprecation and migration. Use when removing old systems, APIs, or features. Use when migrating users from one implementation to another. Use when migrating a database schema in production, such as renaming or dropping a column without downtime (expand/contract). Use when deciding whether to maintain or sunset existing code. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| performance-optimizationaddyosmani/agent-skillsOptimizes application performance across frontend, backend, queries, and databases. Use when performance requirements exist, when you suspect performance regressions, when Core Web Vitals or load times need improvement, when N+1 query patterns need fixing, or when profiling reveals bottlenecks. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| Agent Cost Reportthedotmack/claude-memBelievable agent cost report for any period, default the last 7 full days PT, not counting today. Measured tokens from Claude Code transcripts priced at OpenRouter list prices (ESTIMATED), measured provider spend when a sanctioned source exists, note-taker cost separate, Timing-style HTML/PDF plus report.json, line-items.csv, evidence.json. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |
| Agent Cost Reportthedotmack/claude-memBelievable agent cost report for any period, default the last 7 full days PT, not counting today. Measured tokens from Claude Code transcripts priced at OpenRouter list prices (ESTIMATED), measured provider spend when a sanctioned source exists, note-taker cost separate, Timing-style HTML/PDF plus report.json, line-items.csv, evidence.json. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |