DAAF-Contribution-Community/daaf / .claude/skills/synthetic-data-workflow
synthetic-data-workflow
VerifiedPrivacy-preserving workflow for building synthetic datasets and data source skills when raw data must never enter the DAAF container. The user profiles their sensitive data locally with a disclosure-controlled script; only a summary profile report crosses the boundary; DAAF builds a synthetic dataset and skill from the report alone. Use whenever data is sensitive, proprietary, PII-bearing, HIPAA/FERPA-governed, held in a secure enclave, or the user says the data cannot leave their environment, they cannot upload it, or asks to profile it locally. Covers a four-tier disclosure ladder (T1 schema, T2 marginals, T3 relationships, T4 local high-fidelity synthesis), profile-only generation with simstudy (R) / NumPy-SciPy copulas (Python), and three-part QA (disclosure-safety, report consistency, synthetic-vs-profile validation). Not for synthetic control, the causal-inference method; for that see data-scientist. Synthetic data here is a code-development scaffold, not an analytic substitute.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 17.5 KB |
| assets/profile_data_template.R | other | 78.3 KB |
| assets/profile_data_template.py | script | 69.3 KB |
| assets/synthesize_local_template.R | other | 9.3 KB |
| assets/synthesize_local_template.py | script | 7.4 KB |
| references/disclosure-tiers.md | markdown | 30.5 KB |
| references/generation-patterns-python.md | markdown | 23.7 KB |
| references/generation-patterns-r.md | markdown | 27.4 KB |
| references/local-synthesis-t4.md | markdown | 11.0 KB |
| references/profiling-script-spec.md | markdown | 34.9 KB |
| references/synthetic-data-research.md | markdown | 9.1 KB |
| references/validation-checks.md | markdown | 15.3 KB |
references/synthetic-data-research.md
View on GitHub# Synthetic Data Research — Curated Findings
> **Provenance:** Curated research findings, gathered 2026-07-15 via web survey (package docs on CRAN/PyPI/pkgdown, statistical-agency publications from Census/CMS/ONS, SDC practice guides, and academic + practitioner sources). The URLs cited inline are the authoritative sources — this file is a curated snapshot, and where a factual claim (a package capability, a threshold, an API detail) is load-bearing, verify against the linked source, which may have moved on since capture. Lines marked **[synthesis]** are design reasoning derived from the sources, not direct quotations.
This is the grounding evidence for the `synthetic-data-workflow` skill: the library split (profile-only vs. data-fitted generation), the disclosure-control practice the four-tier ladder rests on, and the common failure modes the workflow is designed against. Section numbers (§1–§4) are stable anchors referenced throughout the skill.
## §1. Profile-driven / metadata-only synthesis (the critical case)
The clean split: **`simstudy` and `fabricatr` generate FROM declarations alone** (no microdata) — exactly DAAF's boundary. SDV/synthpop/synthcity **fit on real data**.
- **`simstudy` (R)** — declarative. `genCorGen`/`addCorGen` draw correlated data via a **Gaussian copula** given a **distribution family + parameters + a correlation matrix (or rho + structure: exchangeable, AR-1)**. Supported marginals: poisson, binary, gamma, uniform, negBinomial, normal. Correlation is honored from the matrix alone; caveat — **binary correlation is recovered less faithfully than Poisson** ([CRAN vignette](https://cran.r-project.org/web/packages/simstudy/vignettes/correlated.html), [genCorGen ref](https://rdrr.io/cran/simstudy/man/genCorGen.html), [author blog](https://www.rdatagen.net/post/correlated-data-copula/)). Can honor: marginals + a correlation matrix. Conditional means / regression coefficients are expressible via `simstudy`'s `defData` formula syntax **[synthesis]**.
- **`fabricatr` (R, DeclareDesign)** — "imagine your data before you collect it." Builds from `N` + variable definitions using any R function; helpers `draw_binary/count/categorical/likert`, hierarchical `add_level`, and `correlate()` for a **target rank correlation** one variable at a time ([getting started](https://declaredesign.org/r/fabricatr/articles/getting_started.html), [variable generation](https://declaredesign.org/r/fabricatr/articles/variable_generation.html)). Honors: marginals, categorical proportions, nested structure, pairwise rank correlation. No native full correlation-matrix solve **[synthesis]**.
- **SDV (Python) metadata** — SDV is metadata-driven (a `Metadata` schema is required), **but the synthesizers `fit()` on real rows**; docs describe learning patterns from real data, not from summary stats alone ([GaussianCopulaSynthesizer](https://docs.sdv.dev/sdv/single-table-data/modeling/synthesizers/gaussiancopulasynthesizer), [PyPI](https://pypi.org/project/sdv/)). `enforce_min_max_values` clamps to real ranges. Not a profile-only generator without a custom setup.
- **Faker / Mimesis (Python)** — identifier-shaped fields only (names, emails, phones, addresses, card numbers), locale-aware, **seedable** for reproducibility; Mimesis is faster/typed, Faker has the larger provider ecosystem ([Faker PyPI](https://pypi.org/project/Faker/), [Mimesis](https://mimesis.name/)). No statistical relationships — cosmetic realism for categorical/PII columns.
## §2. Data-fitted synthesis (higher-fidelity, user-run local option only)
- **`synthpop` (R)** — from the ESRC **SYLLS** project (SYnthetic Data for UK LongitudinaL Studies). `syn()` uses **CART by default** (parametric alternatives available), conditional/sequential specification that respects logical constraints and missing-data patterns; ships utility (`compare.synds`) and disclosure-risk vignettes ([synthpop.org.uk](https://www.synthpop.org.uk/about-synthpop.html), [CRAN manual](https://cran.r-project.org/web/packages/synthpop/synthpop.pdf), [JSS 2016 paper](https://doi.org/10.18637/jss.v074.i11)). **Requires microdata.** Mature, agency-grade.
- **SDV GaussianCopula / CTGAN (Python)** — Copula = fast/interpretable/robust on small data; CTGAN = deep, captures non-linearities but risks training instability/mode collapse ([SDV GitHub](https://github.com/sdv-dev/sdv)).
- **`synthcity` (Python, van der Schaar Lab, NeurIPS 2023)** — large generator arsenal (CTGAN, TVAE, PrivBayes, DP-GAN, PATEGAN, ARF, diffusion TabDDPM); actively maintained; **does not handle missing data — must impute first (HyperImpute)** ([GitHub](https://github.com/vanderschaarlab/synthcity), [docs](https://synthcity.readthedocs.io/en/latest/)). Heavier dependency footprint (deep-learning stack) **[synthesis]** — poor fit for a lightweight user-run local script; `synthpop` is the far lighter local high-fidelity option.
**Embeddable in the user-run script so only synthetic output crosses the boundary:** `synthpop` (R) and SDV/synthcity (Python) all fit locally and emit synthetic rows — any could run inside the local script ([synthpop about](https://www.synthpop.org.uk/about-synthpop.html)).
## §3. Disclosure control practice
- **Five Safes** (Ritchie, ONS, 2003): Safe Projects/People/Settings/Data/Outputs; "Safe Data" is a **residual** — remove only as much as the other safes require ([GOV.UK](https://www.gov.uk/data-ethics-guidance/the-five-safes-framework), [Wikipedia](https://en.wikipedia.org/wiki/Five_safes)).
- **Small-cell suppression:** threshold rule, common thresholds **5 (sometimes 10)**; **primary suppression alone is insufficient** — values recover from marginal totals or by **differencing** across queries, requiring complementary/secondary suppression ([UK Data Service SDC handbook](https://ukdataservice.ac.uk/app/uploads/sdc-handbook-v2.0.pdf), [PMC review](https://pmc.ncbi.nlm.nih.gov/articles/PMC5409873/), [NCES SPWP22](https://nces.ed.gov/FCSM/pdf/SPWP22_rev.pdf)).
- **Min/max & outliers are disclosure risks:** same rules as frequency tables apply to min/max; cap long-tail outliers **unless structural** (e.g., age floor); **"prefer median, quartiles, percentiles to mean/SD/min/max" for extreme data** ([SDC handbook](https://ukdataservice.ac.uk/app/uploads/sdc-handbook-v2.0.pdf)).
- **Access tiering:** PUF > SUF > enclave in required protection ([SDC Practice Guide](https://sdcpractice.readthedocs.io/en/latest/measure_risk.html)).
- **Census SIPP Synthetic Beta:** SRMI-synthesized; tiered files **Synthetic → Completed → Gold Standard**; develop on synthetic via validation server, vetted code re-run on confidential data, DRB output review; "a cell near 10 in synthetic may be smaller in confidential" ([Census SSB](https://www.census.gov/programs-surveys/sipp/guidance/sipp-synthetic-beta-data-product.html), [request guidelines](https://www.census.gov/content/dam/Census/programs-surveys/sipp/methodology/SSB%20_Request_Guidelines_Feb2015.pdf)).
- **CMS DE-SynPUF:** explicitly for **development/training, NOT inference** — "univariate statistics and regression coefficients... will be biased" ([CMS SynPUF](https://www.cms.gov/data-research/statistics-trends-and-reports/medicare-claims-synthetic-public-use-files/cms-2008-2010-data-entrepreneurs-synthetic-public-use-file-de-synpuf)).
- **OpenSAFELY dummy data:** develop analysis against fake data locally; same command emits dummy locally / real inside the enclave; dummy is **"cleaner than real" — won't reproduce impossible values, missingness, or comorbidity patterns**; categorical values respect declared categories; **seeded** for reproducibility ([what is dummy data](https://docs.opensafely.org/ehrql/tutorials/dummy-data/what-is-dummy-data/), [how-to](https://docs.opensafely.org/ehrql/how-to/dummy-data/)).
## §4. Common pitfalls to design against
- **"Works on synthetic, breaks on real" / silent failure** — data looks right, metrics fine, fails in production ([Humans in the Loop](https://humansintheloop.org/3-ways-synthetic-data-breaks-models-and-how-human-validators-fix-them/)).
- **Non-random missingness not reproduced** — real missingness is systematic; generators smooth it away ([Humans in the Loop](https://humansintheloop.org/3-ways-synthetic-data-breaks-models-and-how-human-validators-fix-them/), [IBM](https://www.ibm.com/think/insights/ai-synthetic-data)).
- **Structural fidelity mistaken for statistical validity** — naive random/uniform sampling "can yield dangerous misinterpretations... implementation errors... program crashes or statistical biases" ([medRxiv fidelity/privacy](https://www.medrxiv.org/content/10.1101/2024.12.06.24317239.full.pdf)); fidelity metrics that only check marginals + pairwise correlation miss deeper structure ([same]).
- **Categorical-level leakage as PII** — unique category values (names, free text) are direct identifiers; quasi-identifiers combine to re-identify (87% of US pop by gender+ZIP+DOB) ([Mimesis/Faker masking](https://medium.com/in-a-byte-size/tokenizing-and-masking-pii-phi-data-in-python-using-faker-and-mimesis-a538373ab572)).
- **Outlier leakage via min/max** ([SDC handbook](https://ukdataservice.ac.uk/app/uploads/sdc-handbook-v2.0.pdf)).
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |