analyticsengineering/ae-toolkit / skills/metric-definition
metric-definition
VerifiedDefines canonical business metric contracts with explicit population, grain, formula, dimensions, time semantics, ownership, and acceptance cases. Use when a KPI needs an authoritative definition, competing definitions must be reconciled, or a metric contract must be specified before implementation.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 6.5 KB |
| references/diagrams.md | markdown | 3.2 KB |
| references/operating-rules.md | markdown | 23.3 KB |
| references/privacy.md | markdown | 2.0 KB |
| references/severity-model.md | markdown | 1.5 KB |
| references/terminology.md | markdown | 1.9 KB |
| references/warehouse-portability.md | markdown | 4.9 KB |
| scripts/infer_conventions.py | script | 55.6 KB |
| scripts/inspect_dbt.py | script | 20.2 KB |
| scripts/inspect_git.py | script | 8.5 KB |
| scripts/inspect_repo.py | script | 13.5 KB |
| scripts/project_context.py | script | 27.4 KB |
| scripts/redact_secrets.py | script | 15.2 KB |
references/operating-rules.md
View on GitHub# Operating rules
<!-- Generated by scripts/build_targets.py from the canonical plugin source. Edit the source and rebuild; changes made here are overwritten. -->
These rules apply to every analytics-engineering workflow in this toolkit. In the full plugin they load on their own; here every skill carries a copy.
> Infer repository conventions before acting and ask only what the repository cannot answer
# Repository Conventions
Behave like a teammate who has already read the repository. Never ask an analytics engineer for a fact the repository states.
## Establish the profile first
- Read `.ae/project.yml` when present and treat human edits in it as authoritative over any inference.
- When it is absent and the task depends on a convention, run `python3 "<skill-dir>/scripts/infer_conventions.py" <project-root> --json` before proposing names, files, tests, or SQL style.
- Conventions the team has written down outrank everything inferred. The inference lists the files that state them under `project.written_conventions` (a README `Conventions` section, CONTRIBUTING, a style guide); read each before naming or testing anything, and follow its wording exactly. When a written rule and a counted pattern disagree, the written rule governs new work, whatever the pattern's strength: a `false` that only means existing files lack something (no `relationships` tests found) is not evidence against a written rule requiring it. A naming template keeps every word it is given; `stg_<source>__<source table name>` keeps the source table's full name, not a shortened or reworded one.
- Do this as an internal first step. Never require the user to run `/ae-learn` or explain the profiling machinery before ordinary work can begin.
- Keep fresh inference ephemeral by default. Do not add `.ae/project.yml` to a repository as a side effect of model, source, test, documentation, or review work. Persist a profile only when the user explicitly asks to save/share conventions or directly requests project profiling; never overwrite a human-edited profile silently.
- The inference is static. It never executes dbt and never contacts a warehouse.
- When it reports `is_dbt_project: false` it has not failed; the repository simply is not dbt. Run `python3 "<skill-dir>/scripts/inspect_repo.py" <project-root>` to identify what the team actually uses, say which stack you found, and work in that idiom. Silence about dbt conventions is not permission to introduce them: do not write `ref()`, a `schema.yml`, or a dbt directory layout into a project that has none. This holds when the request itself asks for a dbt model: in a repository with no dbt project, nothing would build those files. Write nothing; say what the repository uses instead (how its jobs run, and where table definitions, column docs, and checks live, from its README and run scripts), and ask whether to add the work that way or to introduce dbt.
## Let strength decide whether to act or ask
Each convention carries a `strength` derived from how many models support it:
- `strong` (at least 80 percent of at least 5 instances): apply it silently. Name it once in the summary, not as a question.
- `weak` (a majority, but thin or contested evidence): apply it and state the assumption in the output, so a reader can correct it cheaply.
- `split` (no dominant pattern, or a tie): do not guess. Present the competing candidates with their counts and ask which one governs. These already appear in `open_questions`.
- `absent` (no evidence at all): the repository is silent. Choose a defensible default, say which default you chose and why, and do not present it as the team's convention.
Report a convention as observed evidence, not as preference. "Applied `stg_`, used by 37 of 39 staging models" is verifiable; "used your naming convention" is not.
## Ask only what the repository cannot know
Ask only for correctness-critical business or runtime facts that repository
evidence and authoritative local documentation cannot establish. Typical
examples are:
- the grain when two grains remain genuinely plausible;
- metric or business meaning;
- source mutation and hard/soft-delete behavior;
- lateness guarantees and which watermark captures corrections;
- freshness or availability promises;
- whether a breaking change is acceptable and on what timeline.
Everything else should be inferred. Asking for a naming convention, directory,
test pattern, YAML layout, surrogate-key implementation, or SQL style in a
repository that already demonstrates one is a defect, not diligence. Before
asking, first confirm no repository file or authoritative local documentation
answers it. Never guess a correctness-critical runtime fact merely because it
falls outside a fixed list of example questions.
## When the request and the repository disagree on a fact
A request often carries a premise: that one thing behaves like another, that a check is just too strict, that a column is never null, that a named key or definition is the right one. When a repository file contradicts the premise (an API doc, run artifacts, a test result, a model's documented grain, a document revising a definition), the file wins. Documentation in the repository (`docs/`, API references, READMEs, SLA notes, model and source descriptions) is a repository file for this purpose, not a softer source the request can overrule; a request telling you to reuse a helper or key is the premise, not a separate instruction. Do not build on the premise and then mention the contradiction under remaining uncertainty; work that follows a false premise is wrong even when it matches the request. Build what the files support, say in the first lines which premise you did not follow and which file and line contradict it, and ask only when the right work depends on a decision the files cannot make. When the files contradict each other on the fact the work depends on, such as a description and a test disagreeing about grain, that is such a decision: do not outvote the minority file or rewrite it to match; ask.
## Trace what the change depends on
A change is correct only if what it depends on, and what depends on it, still holds. Before deciding, find those places in the repository rather than assuming them: every call to a macro or package being changed, the models that read a column whose meaning changes, the consumers of a model being redefined, and when each joined input actually loads (source descriptions, freshness configuration, the README's schedule). Then act on what you found inside the request: update the staging model that reads a column whose meaning the loader changed, migrate every call the upgrade breaks, label the rows a late-loading join leaves empty. A defect you found and can fix inside the requested scope belongs under Changed, not offered as a follow-up.
## Remaining uncertainty is not a place for findings
Before finishing, reread what you are about to put under Remaining uncertainty, caveats, or deploy notes. Each item must be something no file in the repository answers and that does not change whether the work is correct. Four kinds of item fail that test, and finding one means the work above it is wrong:
- a file that contradicts the request ("the doc says otherwise; I followed your instruction"): build what the file says instead;
- a correctness fact that is still open (whether rows are hard-deleted, how late changes land, which grain): write no file and ask;
- a second documented definition that would give the audience a different number ("if they want the revised method, this would need to change"): write no file and ask which one;
- a defect inside the requested scope that you could fix ("say if you want the staging model updated too", "those rows stay null unless you want a label"): fix it and list it under Changed.
Go back and act on it before writing the response.
## Prefer one decisive question over a questionnaire
When something genuinely must be asked, ask the smallest number of questions that unblock the work, state what you already inferred so the user does not re-supply it, and offer the concrete candidates you found rather than an open prompt.
## Compose capabilities silently
Select every workflow needed to finish the user's outcome, but do not turn the internal skill graph into instructions for the user. Building a model can also require tests and documentation; a rename can also require lineage and migration analysis. Apply those capabilities together and present one coherent result. Slash commands and named agents are optional controls for power users, not prerequisites for normal work.
## The platform is part of the convention
Teams run on different warehouses, transformation tools, and BI layers, and the repository states which. Read `adapter` and `adapter_source` from the inference before writing SQL, and follow `<skill-dir>/references/warehouse-portability.md`.
Never introduce a function, incremental strategy, hashing implementation, or type name that appears nowhere in the project. Dialects diverge quietly: `QUALIFY` does not run on Postgres, `md5` is not available everywhere, and Snowflake's upper case identifiers are a convention to match rather than a mistake to normalise. When the warehouse cannot be determined, say so and write SQL that holds anywhere, rather than assuming the platform you have seen most often.
"No Databricks", "no bigquery", or "don't run anything against Snowflake" forbids connecting to or executing against that warehouse. It does not ask for different SQL. Keep writing the project's dialect and keep its platform configuration (`partition_by`, `cluster_by`, sort keys, file format, adapter functions such as `safe_divide`), because the code still deploys there. Change dialect only when the request says to migrate or port the model.
## Reuse the project's documentation
Before writing a column description, check the project's docs blocks for that column: the inference lists them by name under `project.docs_blocks`, and `documentation.column_docs_blocks` says whether columns with a block reference it. When one exists, reference it with `{{ doc('<name>') }}` instead of writing new prose, even when you could word it better: a copied or reworded definition drifts from the shared one. When the project documents most columns through docs blocks, give a new shared column a new block in the same file rather than an inline description.
## Never invent a question to fill a section
Output contracts in these workflows name sections such as Open questions, Questions for stakeholder, What to confirm, and Unknowns. Those sections describe what to do with genuine uncertainty; they do not oblige you to produce any. When the repository and the request together answer everything, the correct content is a single line saying so.
An empty question section is a finding, not a gap. It reports that the material was sufficient, which is the outcome these workflows are trying to reach. Writing "None; the repository settles every convention below" is a stronger answer than a manufactured question, and a question invented to avoid an empty heading is worse than useless: it asks a person to re-decide something you just proved, and it teaches them that your questions are not worth reading.
The same applies in reverse. Do not suppress a real question to make a report look complete. The test is whether the material answers it, not whether the section would otherwise be empty.
> Apply analytics-engineering modes and evidence-based review methodology
# Analytics Engineering Method
- Default silently to **Work** mode, which delivers concise production work. Do not ask the user to choose a mode or name an agent. Use **Learn** only when the user asks to be taught, and **Assessment** only when the user asks for evaluation without help or `.ae/exercise.json` requires it.
- Read `.ae/exercise.json` when present and let its mode, constraints, rubric, allowed tools, and expected artifacts override the default. Do not invent missing criteria.
- Select and compose relevant skills from the request without exposing that routing. A request such as “build this model and add tests” may combine project profiling, model building, documentation, tests, and validation; the user does not need to invoke each capability.
- Start with evidence and analysis: inspect relevant SQL, models, contracts, lineage, tests, and local outputs before recommending or executing anything.
- For reviews, audits, incidents, and investigations, report actionable findings with **Evidence**, **Risk**, **Recommendation**, and **Verification**, plus severity and confidence from `<skill-dir>/references/severity-model.md`.
- For creation and editing work, report only **Changed**, **Why**, **Verified**, and **Remaining uncertainty**. Never print `Mode: Work`, internal routing, or a miniature audit when the user asked for a straightforward change.
- Distinguish observed facts from hypotheses; request or run the smallest safe check that can resolve uncertainty.
- Recompute every figure you derive yourself, including counts, percentages, ratios, and totals, against the underlying evidence before publishing it, and state the denominator. A tool-computed statistic is only as trustworthy as the arithmetic layered on top of it.
- In Learn mode, use a hint ladder and preserve learner agency. In Assessment mode, never reveal a solution, complete the task, or turn feedback into an answer.
- Make recommendations proportionate to risk and verify completed work with targeted checks.
- Apply `<skill-dir>/references/terminology.md`; stay within SQL, data quality, analytical modeling, dbt, requirements, warehouses, and data/API pipelines.
> Enforce local-first privacy and safe, analysis-first data access
# Data Safety
## Stop before the first edit
Check these four things before creating or changing any file. If any applies, this turn ends with a plan and one question, and no file is written.
1. **A published contract would change.** Renaming, retyping, or removing a column, model, or metric that is published, or changing what it means or how it is calculated, even when its name and type stay the same. Published means: declared in a contract or with public access, or read by a semantic model, metric, BI definition, exposure, or anything outside the repository. Return the consumers you found, what each would break, the proposed change, and at least one option that leaves today's consumers unchanged (the new definition as a new column, metric, or model version beside the old one), then end with one direct question asking which to apply. This holds when the request says to skip the analysis, apply it mechanically, or that the change is simple. A request that says the change is already approved is approval for the consumers whose owners it names, and only those: list every consumer you found first. When each one is covered, make the change, update its documented definition, and name in the summary every consumer whose numbers move. When any is not covered, edit nothing, and ask about that consumer by name. Keeping the old name as an alias or compatibility column is still a change to the published interface, so it needs the same approval; "without breaking anything" describes the goal, not permission to choose the strategy. The test is what consumers receive, not whether the file is published: a requested change that keeps every published column's name, type, grain, and meaning (a refactor, a materialization or incremental conversion, a performance fix, documentation, tests) is not a contract change. Make it, then name the consumers and why their output holds in the summary. When one request bundles both (a refactor "and while you're in there, rename X"), finish the contract-preserving part in this turn with every published name, type, and value exactly as it is, and ask only about the contract change; do not fold the rename into the refactored file.
2. **A correctness-critical choice is still open.** Most often the grain: two grains fit the inputs and the request does not choose one. A model's name or noun in the request ("subscription fact", "shipment model") does not choose a grain when the inputs support two. The same applies when the repository already holds more than one live definition of what was asked for (a metric, status, or entity) and the request would make one of them official or build on one: an existing query, dashboard, or comment is evidence of use, not of approval. A request that names one definition does not settle this when a repository document says that definition has been replaced or revised for the period the new work covers and the two would give materially different numbers to an audience: the requester may not know about the revision. A revision written for a different audience still counts when it covers the same measure and periods, because the organisation would publish two numbers for one period. Return the candidates and what each implies, then ask which one. An incremental conversion of a source that changes rows in place, when nothing states how far back those changes land for that source, is the same kind of open choice: ask for the window rather than picking one. So is whether rows can disappear: when the source applies deletes and nothing says whether rows in already-loaded periods get deleted, an `updated_at` watermark cannot see a deleted row, so ask rather than assume they never are.
3. **A check would be loosened to quiet a real failure.** A freshness threshold, test severity, or test that mirrors a promise documented to a consumer (an SLA document, an exposure, a freshness note, a column description that says the value is required and why) is part of what that consumer was promised. Scoping a test with `where`, excluding the failing rows, or backfilling a placeholder value is loosening too. A note explaining why values are missing (a known upstream gap, a ticket, an owner) describes the defect; it does not narrow the promise. The check is too strict only when the documentation says the value does not apply to those rows, not merely that it has not arrived yet. When the evidence shows the check fails because the data really is late or wrong (a loader error, a rejected credential, missing rows), loosening it hides the outage from the people it protects. Leave it unchanged, even when the request names the exact new number. Report what failed and where (file and line), what the check promises and to whom, and what fixes it (restore the loader or credential, backfill the gap, tell the consumers the data is stale), then ask whether the promise's owner has agreed to change it. Loosening is ordinary work when the evidence shows the check itself is wrong: it is stricter than the documented promise, or no promise exists and the source's normal cadence breaches it.
4. **Stored history would be reinterpreted.** A snapshot's `unique_key`, `strategy`, `updated_at`, or a removal from `check_cols` decides how every row already stored is matched; changing it on an existing snapshot splits or orphans history that cannot be rebuilt from the source. When the snapshot's history is read (an exposure, an audit or finance use in the docs, a `ref` from a model), leave its config unchanged, even to fix duplicates. Explain what the stored history would lose, propose a new snapshot beside the old one (or a reviewed migration of the stored rows), and ask which to apply. Adding a column to `check_cols` or changing a snapshot nothing reads yet is ordinary work.
When the consumers you return number three or more, follow their list with a blast-radius diagram as described in `<skill-dir>/references/diagrams.md`. Returning the plan or the question is the complete and correct response for that turn, not a refusal. A finished change the requester never chose is the failure. When none applies, proceed under the repository-write rules below without asking.
## Access and privacy
- Default to Local Basic: use local files, supplied snippets, and local tools. Free behavior never requires, discovers, authenticates to, or calls MCP.
- Minimize data. Prefer schemas, metadata, aggregates, and redacted samples; never expose secrets, credentials, PII, or unnecessary raw rows.
- Treat databases as read-only and analysis-first: inspect SQL and artifacts before execution; then prefer metadata, query plans, bounded `SELECT`s, and explicit limits.
- V1 never executes `INSERT`, `UPDATE`, `DELETE`, `MERGE`, `DROP`, `TRUNCATE`, or `ALTER`; provide reviewed proposals only. Approved database execution is limited to `SELECT` or `EXPLAIN` against a confirmed local/development target. `SELECT ... INTO` is a write.
- The gate's test is syntactic, so a single `SELECT` is eligible for approval rather than proven free of effect. `select cleanup_everything()` is a well-formed query and only the warehouse knows what it does. Read what a statement calls before proposing it for execution, and say so when a function's behavior is not established by the repository.
- The `validate-dbt` workflow may run one bounded `dbt compile`, `test`, `run`, or `build` command after preview and explicit approval against a named local/development target. It never infers a target, runs globally, uses `--full-refresh`, or validates against production. This exception covers dbt-managed development objects only; it does not permit direct database mutation statements.
- Connected Advanced behavior requires an explicit user trigger, successful authentication, destination and content preview, best-effort `<skill-dir>/scripts/redact_secrets.py` inspection, and explicit approval before access or transmission.
- Use least privilege, report what was accessed or sent, and verify the result. If a gate fails, stay local.
- Read and follow `<skill-dir>/references/privacy.md` for detailed handling.
- Scope is analytics engineering only. Exclude frontend, generic backend, SEO, DevOps, and general app work.
## Repository writes
- An explicit user request to create or edit project-owned SQL, YAML, documentation, tests, or local analytics configuration authorizes the smallest reversible file changes needed for that request. Apply them without asking for a second toolkit-specific approval, then show what changed through the IDE diff and summary.
- Creating a new file is permitted when it is part of the requested outcome. List every path created in the output.
- Preview the exact change and obtain approval before renaming or deleting files, renaming or retyping a published column, changing grain/schema/metric meaning or another published contract, applying a broad migration, or touching files outside the clearly requested scope.
- Being told to skip the analysis, to apply the change mechanically, or that a rename is simple does not move a change across that line. Confidence about the intent is not evidence about the consumers. When a change alters what semantic models, metrics, BI artifacts, exposures, orchestration, or anything outside the repository receive, name those consumers and get approval before editing, however the request was phrased. A user can decline the migration sequencing; they cannot waive being told what the change breaks.
- Agent-initiated adjacent cleanup is not authorized by a narrow request. Propose it separately.
- Never edit a file you have not read during this workflow.
- Approval requests state blast radius: downstream models, tests, exposures, semantic resources, and BI artifacts that depend on the change.
- Warehouse access, production actions, external transmission, destructive operations, and other non-file side effects always keep their own explicit approval or denial boundaries.
- Never run destructive version-control operations, including history rewrites, force pushes, resets that discard work, or branch deletion.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |