Skip to content

getaero-io/gtm-eng-skills / skills/deepline-scoring

deepline-scoring

Verified

Use when discovering niche signals, auditing ICP or won/lost evidence, rescoring accounts, or building account and lead scoring Plays. Triggers on fit scoring, engagement scoring, backtesting, holdout evaluation, predictive lift, external proxies, and scoring leakage. Skip pure outreach copy or contributor skill installation tasks.

FileKindSize
.gitignoreother26 B
SKILL.mdskill md12.4 KB
package.jsonmanifest114 B
references/account-website-features.mdmarkdown10.3 KB
references/backtesting-task.mdmarkdown10.0 KB
references/buyer-language-research.mdmarkdown8.6 KB
references/capacity-evidence.mdmarkdown4.4 KB
references/dedupe.mdmarkdown1.5 KB
references/jev-feature-extraction.mdmarkdown8.2 KB
references/keyword-catalog.mdmarkdown4.9 KB
references/pitfalls.mdmarkdown1.6 KB
references/proven-signals.mdmarkdown1.1 KB
references/quality-gate.mdmarkdown2.8 KB
references/report-template.mdmarkdown6.2 KB
references/scorecard-creation.mdmarkdown3.3 KB
references/scorecard-training.mdmarkdown6.5 KB
references/scoring-delivery.mdmarkdown8.3 KB
references/scoring-diagnostics.mdmarkdown8.9 KB
references/scoring-pitfalls.mdmarkdown2.8 KB
references/signal-interpretation.mdmarkdown2.3 KB
references/step-7-prospects.mdmarkdown1.6 KB
references/technology-evidence.mdmarkdown9.4 KB
references/testing-and-evaluation.mdmarkdown6.8 KB
scripts/analyze_signals.pyscript16.9 KB
scripts/analyze_signals_v2.pyscript28.3 KB
scripts/build_feature_matrix.pyscript4.7 KB
scripts/collect_sources.pyscript16.1 KB
scripts/dedupe_utils.pyscript15.6 KB
scripts/dedupe_utils_v2.pyscript15.7 KB
scripts/find_contacts_v2.pyscript1.9 KB
scripts/hubspot_accounts.pyscript7.1 KB
scripts/jev_features.pyscript7.1 KB
scripts/replay-score.play.tsscript2.3 KB
scripts/requirements-scorecard.txtmarkdown263 B
scripts/scorecard/__init__.pyscript514 B
scripts/scorecard/config.pyscript5.0 KB
scripts/scorecard/explainer_artifacts.pyscript8.4 KB
scripts/scorecard/pipeline.pyscript38.9 KB
scripts/scorecard/report.pyscript30.4 KB
scripts/scorecard/scorer.pyscript17.3 KB
scripts/scorecard/specs.pyscript1.3 KB
scripts/scoring_contract.tsscript7.5 KB
scripts/technology_evidence.pyscript3.8 KB
scripts/tests/contract_check.tsscript1.9 KB
scripts/tests/test_jev_features.pyscript3.4 KB
scripts/tests/test_scorecard.pyscript9.5 KB
scripts/train_scorecard.pyscript20.0 KB
skill-metadata.jsonconfig3.8 KB

references/scorecard-creation.md

View on GitHub
# Create an artifact-backed scorecard

Build scoring from explicit raw inputs, saved preprocessing and model artifacts, one deterministic scorer, and explanations derived from the same contributions. Reproduce the requested customer's approved outputs; do not transplant another customer's weights, thresholds or feature requirements.

## Build the scoring path

1. **Define the input row.** Resolve account identity and preserve source evidence, dates and coverage. Specify required raw fields, types, units, unknown values and the decision cutoff. Keep fit inputs separate from activity and outcomes. Choose fields for the defined decision rather than copying another model’s inputs.
2. **Freeze preprocessing.** Save raw-to-feature mappings, categorical/numeric bins, missing/unseen-category handling and transforms. Apply the identical artifact to training, batch scoring and a new singleton account.
3. **Save the model.** Store feature/bin points, intercept, scale, optional calibration, fixed reference cutoffs and explicit overrides. Keep a content hash for every artifact. Any approved extraction, override, grading or routing change creates a new full-system version.
4. **Score once.** Load the artifacts into a shared pure runner. Compute per-feature contributions, total points and model output. Calibration and percentile/tier assignment are separate transformations. Both the Play and offline report consume this result; neither recomputes a competing formula.
5. **Explain the actual result.** Join each contribution to its raw value, chosen bin, source and applied override. Build account drivers and the readable scorecard from the same model artifacts. Show how points reconcile and distinguish points, probabilities, percentiles, tiers and routing decisions.
6. **Wrap collection around scoring.** A Play resolves the input identifier, collects or reuses evidence, applies the frozen feature definitions and calls the runner. Keep enrichment receipts separate so rescoring does not repurchase data. Export every input row, including unknowns and errors.
7. **Promote explicitly.** Separate fit, engagement, expected value and routing policy. Use approved output parity and the customer's decision-specific acceptance evidence before replacing the incumbent. Keep the old artifact bundle for audit and rollback. See the testing contract; executable evals are maintained separately.

## Keep artifacts and outputs consistent

Use one preprocessing specification, model definition and contribution breakdown across scoring, explanation and reporting. Define override precedence explicitly. Freeze calibration and reference cutoffs separately from raw points. A probability, percentile and routing tier are different outputs; return only those supported by the approved definition.

A readable account export includes `account_id`, raw feature values, chosen bins, per-feature points, total points, model output, calibration/reference versions, requested percentile/tier, evidence coverage and routing reason. The UI can show the primary score and a few drivers while retaining the complete breakdown for audit.

## Fail explicitly

Validate required artifacts and return explicit errors or unknowns when they are missing. Avoid silent failures in explanations or grading. A saved-output snapshot is a compatibility target, not independent evidence of predictive quality.
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall