alivirgo/Major-AI-Skills / plugins/mas-efficiency-pack/skills/csv-over-json-tables
csv-over-json-tables
VerifiedRepresent rectangular datasets as CSV or TSV when repeated JSON field names add unnecessary payload size.
Tabular Serialization Protocol (CSV / TSV over JSON Arrays)
Overview
When an LLM extracts or analyzes tabular datasets (e.g., customer transaction lists, database query results, or server metric logs), defaulting to an array of JSON objects forces the model to repeat every column key on every single row ("first_name": "...", "transaction_amount": "...", "status": "...").
In a 100-row dataset, repeating JSON keys burns thousands of redundant tokens purely on structural quotation marks, colons, and braces.
The Tabular Serialization Protocol replaces JSON object arrays with Comma-Separated Values (CSV) or Tab-Separated Values (TSV) - defining the header keys once at row 1 and streaming raw comma-delimited data rows, cutting token usage by 65% to 75%.
JSON Object Array vs. CSV Data Stream
┌─────────────────────────────────────────────────────────────┐
│ Tabular Payload Comparison │
│ │
│ JSON Object Array (145 Tokens for 2 Rows): │
│ [ │
│ {"user_id": 101, "email": "alice@corp.io", "tier": "pro"}│
│ {"user_id": 102, "email": "bob@corp.io", "tier": "free"} │
│ ] │
│ ↳ Keys `user_id`, `email`, `tier` repeated on every row! │
│ │
│ CSV Data Stream (34 Tokens - 76.5% Reduction!): │
│ user_id,email,tier │
│ 101,alice@corp.io,pro │
│ 102,bob@corp.io,free │
│ ↳ Keys defined ONCE. Zero quotation/brace syntax waste. │
└─────────────────────────────────────────────────────────────┘
The Master CSV Extraction Prompt Template
When querying an LLM to extract or output structured rows:
Extract the customer records from the text below:
<source_data>
[PASTE UNSTRUCTURED TEXT]
</source_data>
Output Constraints:
- Format as raw **CSV (Comma-Separated Values)**.
- Line 1 MUST be the exact header row: `user_id,name,email,plan_tier,monthly_spend`
- Do NOT output JSON.
- Output ONLY the CSV block; zero introductory or concluding commentary.
Production Fast Client Deserialization
Python (csv.DictReader):
import csv
import io
from typing import List, Dict
def parse_csv_llm_output(csv_text: str) -> List[Dict[str, str]]:
"""Instantly deserializes CSV LLM output into clean Python dictionaries."""
# Strip markdown code fences if present
clean_csv = csv_text.strip().strip("`").removeprefix("csv").strip()
reader = csv.DictReader(io.StringIO(clean_csv))
return list(reader)
TypeScript / Node.js (PapaParse / Native Split):
export function parseCompactCSV<T = Record<string, string>>(csvText: string): T[] {
const lines = csvText.trim().replace(/^```(?:csv)?/m, '').replace(/```$/m, '').trim().split('\n');
const headers = lines[0].split(',').map((h) => h.trim());
return lines.slice(1).map((line) => {
const values = line.split(',').map((v) => v.trim());
return headers.reduce((obj, header, index) => {
// @ts-ignore
obj[header] = values[index];
return obj;
}, {} as T);
});
}
Token & Cost Benchmark Comparison
Extraction of 250 enterprise user records (5 columns per record):
| Serialization Format | Total Output Tokens | Generation Latency | Cost (GPT-4o / Claude Sonnet) |
|---|---|---|---|
| JSON Object Array | 12,800 tokens | 14.5 seconds | $0.192 |
JSON Array of Arrays ([[..]]) | 6,400 tokens | 7.2 seconds | $0.096 |
| CSV / TSV Plain Text | 3,100 tokens | 3.4 seconds | $0.046 (75.8% Savings!) |
Agent Operational Directive
MANDATORY: For non-nested, tabular datasets ($> 5$ rows), agents must prompt and emit data in CSV/TSV format rather than JSON object arrays. Deserializers on the client will convert CSV into domain objects with zero CPU overhead.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| deprecation-and-migrationaddyosmani/agent-skillsManages deprecation and migration. Use when removing old systems, APIs, or features. Use when migrating users from one implementation to another. Use when migrating a database schema in production, such as renaming or dropping a column without downtime (expand/contract). Use when deciding whether to maintain or sunset existing code. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| performance-optimizationaddyosmani/agent-skillsOptimizes application performance across frontend, backend, queries, and databases. Use when performance requirements exist, when you suspect performance regressions, when Core Web Vitals or load times need improvement, when N+1 query patterns need fixing, or when profiling reveals bottlenecks. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| agent-cost-reportthedotmack/claude-memBelievable agent cost report for any period, default the last 7 full days PT, not counting today. Measured tokens from Claude Code transcripts priced at OpenRouter list prices (ESTIMATED), measured provider spend when a sanctioned source exists, note-taker cost separate, Timing-style HTML/PDF plus report.json, line-items.csv, evidence.json. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |
| agent-cost-reportthedotmack/claude-memBelievable agent cost report for any period, default the last 7 full days PT, not counting today. Measured tokens from Claude Code transcripts priced at OpenRouter list prices (ESTIMATED), measured provider spend when a sanctioned source exists, note-taker cost separate, Timing-style HTML/PDF plus report.json, line-items.csv, evidence.json. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |