mbfinotti/advertising-skills / skills/ad-creative-test-plan
ad-creative-test-plan
VerifiedDesign a pre-launch ad creative test plan - falsifiable hypothesis, isolation level, test cells with per-cell budgets, required sample, spend and duration, and kill/scale rules pre-registered before any money moves. Every cell carries an explicit read standard, so an underpowered screen is never dressed up as an A/B test. Use whenever the user wants to test ads, mentions an A/B or split test on creative, asks how much budget or how long a test needs, or mentions sample size, statistical significance, single-variable vs big-swing testing, or test cell structure - even if they never say 'test plan'. Covers B2B and B2C. Do NOT use to read results from a test already running - use mbfinotti/advertising-skills@ad-creative-fatigue instead.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 32.1 KB |
| evals/evals.json | config | 38.5 KB |
| references/example-test-plan.md | markdown | 8.2 KB |
| references/sizing-reference.md | markdown | 14.8 KB |
evals/evals.json
View on GitHub{
"skill_name": "ad-creative-test-plan",
"evals": [
{
"id": 1,
"prompt": "I run paid social at Fernweh Goods, a DTC travel-accessories brand. We have $15,000/month set aside purely for testing, separate from scaling spend. Current numbers: $20 CPA on purchases, $1.25 CPC, 2.5% click-to-purchase rate, about 700 purchases a month account-wide. We shot two new concepts (a packing-hack demo and a lost-luggage horror story) and want to run a proper statistically significant A/B test against our current best ad to pick which concept gets the holiday production budget. Our marketing lead wants to announce 'the winner at 95% confidence' at the all-hands in 5 weeks. Can you design the test?",
"expected_output": "A pre-launch test plan with three concurrent fixed-budget cells, feasibility math showing the purchase read cannot reach significance inside the window, an explicit Directional-read verdict instead of a promised 95% winner, and pre-registered decision rules.",
"files": [],
"expectations": [
"Computes the stable-delivery minimum daily budget per cell of roughly $143/day (target CPA $20 x 50 / 7) and checks the proposed cells against it",
"Includes the current best ad as a control cell running concurrently at its own fixed budget, not as a historical baseline",
"Computes required sample per cell for a purchase-rate significance read at 80% power and alpha 0.05 (exact two-proportion formula or the n = 16 x p(1-p) / MDE^2 shortcut), showing the number",
"Shows with arithmetic that reaching significance on the purchase metric at any moderate MDE requires more days than the test window allows",
"Labels the purchase-metric read with the explicit verdict 'Directional read' (a screening heuristic), not a significance-tested winner",
"Explicitly tells the user the '95% confidence winner in 5 weeks' promise cannot be honored on the purchase event, rather than designing a plan that pretends it can",
"Applies a duration floor of one full week (day-of-week coverage) and a ceiling of roughly 4-6 weeks tied to creative freshness",
"Pre-registers a kill threshold and a scale threshold per asset or cell before launch",
"Pre-declares the inconclusive outcome as keep the control/champion, decided before launch rather than at readout",
"States a fixed stopping rule as a date or a sample size, whichever comes first",
"Writes a falsifiable hypothesis containing a predicted direction, magnitude, named metric, audience, and date or sample, rather than 'see which wins'",
"Raises powering the read on a higher-volume up-funnel event (such as add-to-cart), or asks for that event's rate, instead of only accepting the underpowered purchase read",
"Instructs turning automated or AI creative-optimization features off inside the test cells"
]
},
{
"id": 2,
"prompt": "I do demand gen at Veyra Systems, a B2B procurement software company - roughly $45k ACV and a 4-month sales cycle. We get demo-request form fills from LinkedIn at about $120 each, and roughly 30% of those get accepted by sales as qualified. Test budget is $9,000/month. Our CRO wants us to test a 'CFO cost-cutting' angle against our current 'automation time-saving' ads and have a statistically significant winner on form fills in three weeks, because form fills are the number we report upward. Lay out the test for me.",
"expected_output": "A B2B screening plan judged on CRM-qualified leads over several weeks with a declared Directional verdict, delivery-limitation flagged with the floor math, a quality guardrail, and a scheduled second look one sales cycle later.",
"files": [],
"expectations": [
"Sets the primary decision metric to a CRM-fed pipeline-quality event (qualified/accepted lead), explicitly rejecting raw form fills as the decision metric",
"Treats cheap form fills that sales rejects as a guardrail breach, not a win",
"Declares the verdict for this structure as 'Directional read' and states it in the plan, refusing the statistical-significance framing",
"Computes the stable-delivery floor on the form event at roughly $857/day per cell ($120 x 50 / 7) and shows the affordable cells sit far below it",
"Flags the cells as delivery-limited, meaning their numbers are unstable regardless of sample math",
"Extends the judgment window to several weeks (roughly 4-6), pushing back on the three-week deadline for a qualified-lead read",
"Schedules a second look roughly one sales cycle (~4 months) after the creative verdict for pipeline or opportunity outcomes, instead of treating the in-window read as final",
"Adds a lead-quality guardrail: lead-to-qualified rate compared against the account's own trailing median",
"Keeps the current 'automation' champion running concurrently as a control cell",
"Pre-registers kill and scale thresholds and a fixed stop before launch",
"Places the hypothesis magnitude on the qualified-lead proxy metric and says the revenue read lags by a sales cycle",
"Describes the readout as judgment-based relative ranking, with no '95% confidence' claim anywhere",
"Does not resolve the budget shortfall by assuming a budget increase; works within $9,000/month or names consolidation/up-funnel levers explicitly"
]
},
{
"id": 3,
"prompt": "Growth lead at Peakform Nutrition here (DTC protein snacks). We paused our long-running bestseller ad last month when it was sitting at $18 CPA in Q2, and now we have three fresh concepts from our agency. My plan: put all three into one campaign with campaign budget optimization on and the platform's AI creative enhancements enabled so the algorithm finds the winner efficiently, then compare whichever wins against the champion's Q2 CPA of $18 to decide if we beat it. $450/day for two weeks. Anything you'd change?",
"expected_output": "A restructured plan that revives the champion as a concurrent control, replaces automatic budget allocation with manual fixed-budget cells, switches AI creative features off, and fixes the below-floor per-cell budget with a named lever.",
"files": [],
"expectations": [
"Rejects comparing new cells to the champion's historical Q2 CPA and requires the champion to run concurrently as a control cell in the same structure and budget",
"Gives the reason for the concurrent control: unequal delivery history and/or seasonality make old numbers incomparable",
"Rejects campaign-level automatic budget allocation as a test structure",
"Explains the allocation failure concretely: spend concentrates on early leaders (up to roughly 90% on one cell) before the others collect data",
"Recommends manual fixed-budget cells (or a platform-native deterministic split) with a stated fixed budget per cell",
"Instructs turning the automated/AI creative-optimization feature off inside the test cells",
"Notes the auction-overlap caveat of manual cells: cells still compete in the same auctions against overlapping audiences",
"Computes the stable-delivery floor of roughly $129/day per cell ($18 x 50 / 7) and flags that $450/day split four ways falls below it",
"Resolves the floor breach with a named lever (fewer cells, sequential concepts, up-funnel read, or budget change) rather than shipping four delivery-limited cells silently",
"Emits an explicit read-standard verdict per cell (Powered, Directional read, or Not testable as designed)",
"States with arithmetic what claim the two-week window can support on the purchase metric (screening/Directional versus Powered)",
"Pre-registers kill and scale rules and pre-declares inconclusive as keep the champion"
]
},
{
"id": 4,
"prompt": "I'm a very data-driven founder at Cartelle, a made-to-order furniture brand - $6k/month total ad spend, roughly $70 CPA. I want a genuinely rigorous testing program: one variable at a time, everything else held constant. My backlog: 'Shop Now' vs 'Shop the collection' CTA text, headline with vs without a dash, beige vs off-white background, model facing left vs right, sans-serif vs serif overlay font, and price shown vs hidden. Six clean single-variable tests over the next six months. Long-term I care much more about building reusable learnings than any single winner. Build me the schedule and per-test budgets.",
"expected_output": "A refusal of the micro-variable program with the feasibility math showing why, a redirect to high-leverage levers under a bundled-concept posture with the unlearnable label, and the named condition under which strict isolation gets promoted later.",
"files": [],
"expectations": [
"Refuses to schedule the six micro-variable single-variable tests as proposed, naming items like font, background hue, dash punctuation, CTA wording, and model direction as micro-variables not worth isolated spend at this budget",
"Folds the micro-variations into concept-level executions or drops them, instead of testing them individually",
"Names the high-leverage levers worth isolating (concept, angle, hook, format, creator/talent) and redirects the program there",
"Computes the stable-delivery floor of roughly $500/day per cell ($70 x 50 / 7) and shows $6,000/month (~$200/day total) cannot fund even a compliant two-cell test on the purchase event",
"Rejects strict isolation on feasibility grounds despite the user's stated compounding-learning preference - the preference alone does not promote isolation without the power to read the lever",
"Presents the bundled-concept posture as the default at this budget, because it is the only posture producing a reliably detectable effect at normal budgets",
"Labels any bundled result 'unlearnable at element level' (or an equivalent explicit label) so no element-level insight is mined from it later",
"States the isolation trade-off explicitly (bundled buys detectability; strict isolation buys element-level learning only when powered) rather than treating single-variable testing as inherently more rigorous",
"Names the condition that would promote strict isolation later (feasibility returning Powered on the lever's own MDE, e.g. after volume grows)",
"Emits an explicit per-cell verdict (Powered, Directional read, or Not testable as designed) for whatever structure it does propose",
"Keeps one concept per cell with roughly 3-6 assets per cell in the proposed structure",
"Pre-registers kill and scale thresholds and writes a falsifiable hypothesis with direction and magnitude, not 'see which performs'"
]
},
{
"id": 5,
"prompt": "Quick one - I run ads for Brightlark, a meditation app: $30 CPA, about $18k/month spend, and I'm solo, checking the account maybe twice a week between everything else. Launching 6 new ads next Monday and I need the go/no-go rules written down before then, since we pick the winning batch in exactly 4 weeks for an App Store feature window. I've seen Barry Hott's rule everywhere - spend 1x CPA before judging and 3x CPA before killing - and I was going to combine that with cost caps so the platform kills losers automatically, plus a hard rule to cut anything at 2x CPA spend with no install. Formalize this into our kill rules?",
"expected_output": "A corrected, single-anchor decision-rule set: the folklore attribution debunked, Hott's real position stated, incompatible anchors deleted by name against the user's constraints, and Denney's defaults adopted with the numbers carried through.",
"files": [],
"expectations": [
"Corrects the attribution: the 'spend 1x CPA before judging / 3x CPA before killing' rule is untraceable folklore with no identifiable originator, not Barry Hott's",
"States Hott's actual published position: he rejects mechanical per-ad CPA kill rules (ad-level CPA/ROAS irrelevant) and judges new ads by comparative benchmarking against the account's best ads",
"Refuses to combine multiple kill-rule anchors and states the plan must adopt exactly one",
"Gives the reason mixing fails: the anchors contradict each other, so a blended rule never fires or fires twice",
"Rules out the no-manual-kill / cost-cap-only approach (Faris) by name, tied to the hard 4-week deadline because it is the slowest to answer",
"Rules out Hott's comparative benchmarking by name, tied to the missing weekly reviewer and absent best-ads library",
"Adopts Dara Denney's defaults as the single anchor, identified as the default when nothing promotes another",
"Carries Denney's sizing: test budget approximately CPA x 50 (about $1,500 at a $30 CPA) and roughly 6 assets per test",
"Carries the no-evaluation-before-day-3 rule",
"Carries the asset kill at 2x CPA spend ($60) with no conversion, and the cell kill after 5-7 days with no winner",
"Carries the scale rule: winners scaled +50-100%, done 2-3 times",
"Pre-registers a fixed stopping rule and the inconclusive path (keep the current champion/batch) before launch",
"Never presents the 1x/3x folklore thresholds as authoritative anywhere in the output"
]
},
{
"id": 6,
"prompt": "We're Nordvik Outdoor, a DTC hiking-gear brand. Launching a creative test Thursday: two new video concepts vs our current winner, $250/day per cell, $45 CPA. My reading plan: I check every morning, and the moment any variant is ahead at 95% significance I pause the others and shift budget over - realistically that's day 3 or 4, our tests usually reach significance fast. Also, we'll launch the two new ones Thursday and add the control the following Monday since it needs new post IDs. Write this up as the official plan so the team follows it.",
"expected_output": "A plan that replaces daily stop-at-significance checking with a pre-registered fixed stopping rule and earliest evaluation moment, launches all cells simultaneously, flags the below-floor budgets, and explains why past 'fast significance' was an artifact.",
"files": [],
"expectations": [
"Rejects the stop-at-first-95% daily-checking rule and names peeking/early stopping as the failure being prevented",
"Explains why peeking fails: stopping on a favorable early number inflates false positives, and early data is systematically noisy",
"Replaces it with a fixed stopping rule pre-registered as a date or a sample size, whichever comes first",
"Sets an earliest evaluation moment (day 3 or later plus a minimum volume floor) before which no judgment is made",
"Applies the one-full-week duration floor for day-of-week coverage and flags a day 3-4 decision as below it",
"Requires all cells including the control to launch at the same time, rejecting the Thursday/Monday stagger",
"Ties the stagger rejection to its reason: cells launched at different times pick up day-of-week/seasonality contamination and become incomparable",
"Flags the novelty effect: new creatives get an early boost that fades, so short reads over-credit new variants",
"Reframes 'our tests usually reach significance fast' as an artifact of unadjusted peeking rather than validating it",
"Pre-declares the inconclusive outcome as keep the control",
"Computes the stable-delivery floor of roughly $321/day per cell ($45 x 50 / 7) and flags the $250/day cells as delivery-limited or fixes them with a named lever",
"Emits an explicit per-cell read-standard verdict instead of implying significance is reachable",
"Pre-registers kill and scale thresholds so no real-time judgment call is needed during the run"
]
},
{
"id": 7,
"prompt": "Creative strategist at Vantail, a DTC apparel brand. Five new videos go into a test next week. Purchases are slow to accumulate, so I want hook rate as the deciding metric - whichever video holds the best 3-second view rate takes the scale budget. I read that anything above a 30% hook rate is strong, so that's our bar. Two of the five will run mostly in Reels placements, the other three across feed. Budget is comfortable: $400/day per cell, $28 CPA. Draft the test plan and the success criteria.",
"expected_output": "A plan with a three-layer metric ladder where hook rate is demoted to a gate that only kills losers, thresholds set from the account's trailing median with like-for-like placement comparison, a conversion-level primary metric, and the feasibility math run despite the comfortable budget.",
"files": [],
"expectations": [
"Refuses to make hook rate the primary or deciding metric; assigns it as a gate metric used only to kill obvious losers early and cheaply",
"States that gate metrics screen and never crown winners",
"Cites the evidence class: a multi-account analysis of roughly $1.47M in ad spend found no statistically significant correlation between hook rate and revenue",
"Notes two ads at an identical hook rate can differ several-fold in return, because the gate says nothing about who survives to the call to action",
"Rejects the published 30% bar and sets the gate threshold from the account's own trailing median, noting published 'good' bands conflict (roughly 18% to 40%)",
"Requires like-for-like gate comparison by placement, flagging that comparing Reels-heavy assets against feed assets on raw hook rate crowns accidental winners",
"Names a conversion-level primary metric (purchase CPA or conversion rate) as the decision metric",
"Adds guardrail metrics that must not degrade (e.g. frequency, cost inflation vs account baseline, refund/return rate)",
"Notes that high hook rate combined with low conversion usually signals an offer or landing-page problem, not a creative winner",
"Runs the feasibility check anyway: projects conversion volume per cell (about 14/day at $400/day and $28 CPA) and emits an explicit Powered-or-Directional verdict for the purchase read",
"Reads the gate only after a minimum impression volume per asset (on the order of 1,000-2,000 impressions)",
"Pre-registers kill and scale rules and a fixed stop before launch"
]
},
{
"id": 8,
"prompt": "Merrow & Finch sells premium mattresses online - $180 CPA on purchase. Marketing gave me $500/day for testing, split across 4 cells (3 new concepts plus control). Finance has already rejected our budget-increase request twice this quarter, so that door is closed. My workaround: run the test 10 weeks instead of 4 so the sample accumulates. We do have a solid add-to-cart event, about $12 per add-to-cart, verified firing correctly last month - not sure that matters. Design the plan.",
"expected_output": "A plan that rejects the 10-week stretch against the duration ceiling, declares the purchase cells Not testable as designed with the floor math, deletes the budget lever by name, moves the read to the verified add-to-cart event, and records both read standards.",
"files": [],
"expectations": [
"Rejects the 10-week duration, citing the roughly 4-6 week ceiling because novelty decay, fatigue, and external events contaminate longer windows",
"Computes the stable-delivery floor on purchase of roughly $1,286/day per cell ($180 x 50 / 7) and shows the $125/day cells sit an order of magnitude below it",
"Declares the purchase-event cells with the named verdict 'Not testable as designed' rather than shipping them with hedges",
"Applies the ranked fix levers with the up-funnel read first (up-funnel > widen MDE > fewer cells > raise budget), not more budget or more time by default",
"Deletes 'raise the budget' from the menu by name because finance already refused (no budget authority), instead of listing it as a fallback option",
"Moves the optimization and read to the add-to-cart event, whose floor of roughly $86/day per cell ($12 x 50 / 7) the $125/day cells clear",
"Records both reads in the plan: the add-to-cart read at its computed standard AND the purchase metric explicitly labelled Directional",
"Notes the up-funnel lever is available because the add-to-cart event is verified (an unverified or broken event would delete that lever until tracking is cleared)",
"States the stable-delivery requirement of roughly 50 optimization events per cell per week",
"Frames the delivery floor as a read-validity requirement, not a performance unlock (exiting the learning state is worth only a modest ~5-10% efficiency gain)",
"Keeps the control cell concurrent and pre-registers kill and scale thresholds and a fixed stop",
"Includes a one-sentence decision statement and a falsifiable hypothesis with direction and magnitude"
]
},
{
"id": 9,
"prompt": "Solent & Co, DTC eyewear. Board meeting in 12 days and the CEO wants 'a scientifically clean creative test' presented - so I'm setting up the ad platform's native A/B test tool for our two new concepts, since deterministic assignment means no contamination and the result will be causal proof of which creative is better. $150/day total test budget, $32 CPA. Help me configure the cells and the writeup for the board.",
"expected_output": "A plan that steers away from the native split under the hard deadline, corrects the causal-proof claim with the divergent-delivery evidence, uses manual fixed-budget cells, computes the floor breach at this budget, and states what claim the board can actually be given.",
"files": [],
"expectations": [
"Recommends against the platform-native deterministic split for this test, tied to the hard 12-day deadline (splits take a week or more longer to answer)",
"Notes that native splits at typical budgets are underpowered and frequently return 'no winner'",
"Recommends manual fixed-budget cells instead and states the structure ranking (manual fixed-budget > native deterministic split > campaign-level automatic allocation)",
"Corrects the causality claim: even a deterministic split suffers divergent delivery - each ad is served to a distinct, undetectably optimized mix of users",
"Marks the severity of divergent delivery: it can confound the magnitude and even flip the sign of the result (Braun & Schwartz 2025 or equivalent attribution)",
"States that in-platform reads, splits included, are relative screening rather than causal proof, and that holdout/lift designs are what buy causality",
"Refuses to promise the board a causal or significance-tested winner in 12 days; the plan states exactly what claim the read supports",
"Computes the stable-delivery floor of roughly $229/day per cell ($32 x 50 / 7) and flags $150/day total split across cells as far below it",
"Emits the named verdict for the as-proposed structure ('Not testable as designed' on purchase, or Directional only after a named fix)",
"Fixes the floor breach with a ranked lever (up-funnel read, wider MDE, or fewer cells) rather than silently shipping delivery-limited cells",
"Asks whether a current champion exists to serve as the control cell, or includes a concurrent control",
"Instructs turning automated creative-optimization features off inside test cells",
"Pre-registers the stopping rule, earliest evaluation moment, and kill/scale thresholds before launch"
]
},
{
"id": 10,
"prompt": "Head of growth at Tindra Home (smart lighting). Our freelancers delivered 9 ads: three are variations on a 'movie night' scene idea, four riff on 'wake up naturally', and two take a security/away-mode angle. I was going to spread them across three ad sets, mixing them up so each ad set gets one of each type for fairness, name the files as they come in like tindra_final_v2_new1.mp4, and launch to see what happens. $220/day per ad set, $24 CPA, decision in two weeks. Structure this properly?",
"expected_output": "A restructured plan with one concept per cell, a decision statement and falsifiable hypothesis, a structured variant-naming convention, the multiple-comparisons correction surfaced, the feasibility math run, and pre-registered decision rules.",
"files": [],
"expectations": [
"Rejects mixing concepts within a cell and restructures to one concept per cell (a movie-night cell, a wake-up cell, a security cell)",
"Gives the reason: more concepts than cells contaminates the read - results stop being attributable to any concept",
"Keeps roughly 3-6 assets per cell and flags the 2-asset security concept as below the band (produce another asset or handle the shortfall explicitly)",
"Writes the one-sentence decision statement ('Based on this test, we will ___') before anything else",
"Rejects 'launch and see what happens' and writes a falsifiable hypothesis containing direction, magnitude, named metric, audience, and a date or sample",
"Replaces the ad-hoc file naming with a structured variant convention: fixed field order, one delimiter, encoding concept/angle and other tested levers plus a version",
"States the mechanical reason naming matters: reporting tools parse names, and a naming failure silently destroys the roll-up by dimension",
"Flags the multiple-comparisons problem of three concept cells against control: some variant wins by chance",
"Quantifies or corrects it: roughly 30-40% more sample per cell at ~3 variants under a standard alpha-division correction, or fewer cells - and at minimum discloses the number of comparisons",
"Adds a concurrent control cell (current champion) or asks whether one exists",
"Computes the stable-delivery floor of roughly $171/day per cell ($24 x 50 / 7) and projected weekly conversions (~60-65 per cell), confirming stable delivery",
"Emits an explicit read-standard verdict per cell for the two-week window (Directional versus Powered, with the sample arithmetic) instead of implying a significant winner",
"Pre-registers kill and scale thresholds, an earliest evaluation moment, a fixed stop, and inconclusive resolving to keep the control"
]
}
],
"trigger_queries": [
{ "query": "How do I set up an A/B test for my Facebook ad creatives before launch?", "should_trigger": true },
{ "query": "Design a creative test plan for 3 new video concepts on Meta", "should_trigger": true },
{ "query": "How much budget do I need to test two ad concepts against each other?", "should_trigger": true },
{ "query": "What sample size do I need to call a winner between two ads?", "should_trigger": true },
{ "query": "How long should I run a creative split test before deciding?", "should_trigger": true },
{ "query": "Should I test one variable at a time or make big swings in my ad creative?", "should_trigger": true },
{ "query": "Help me structure test cells for our Q4 creative testing", "should_trigger": true },
{ "query": "Is $5k enough to get statistical significance on an ad test?", "should_trigger": true },
{ "query": "We're launching 5 new statics next month - how do I decide which one wins fairly?", "should_trigger": true },
{ "query": "My CMO wants a 95% confidence winner between our two hero videos. What do I need?", "should_trigger": true },
{ "query": "single variable vs bundled creative testing - which should we do", "should_trigger": true },
{ "query": "How many conversions per variant before an ad test result means anything?", "should_trigger": true },
{ "query": "Set up a controlled experiment comparing our new UGC concept to the current champion ad", "should_trigger": true },
{ "query": "What kill rules should I pre-register before launching new ad creatives?", "should_trigger": true },
{ "query": "Plan a TikTok creative test - 4 concepts, $300 a day", "should_trigger": true },
{ "query": "How do I know if my ad test is underpowered before I spend the money?", "should_trigger": true },
{ "query": "We keep arguing about which ad concept to scale - design a fair test", "should_trigger": true },
{ "query": "creative testing framework for a B2B lead gen account", "should_trigger": true },
{ "query": "how big should my test budget be per ad set", "should_trigger": true },
{ "query": "Is a week long enough to test new ad creative?", "should_trigger": true },
{ "query": "I have 6 new ads and $2,000. What's the smartest way to find the best one?", "should_trigger": true },
{ "query": "Write me a test matrix for hook variations on our best performing video ad", "should_trigger": true },
{ "query": "MDE and sample size for an ad test with a 2% conversion rate", "should_trigger": true },
{ "query": "before we spend on these new concepts, how should we structure the experiment", "should_trigger": true },
{ "query": "What's the minimum spend before judging a new creative?", "should_trigger": true },
{ "query": "Should I use Meta's A/B test tool or split budgets manually for creative testing?", "should_trigger": true },
{ "query": "How many ad variants can I test at once with $150/day?", "should_trigger": true },
{ "query": "statistical significance for ad creative tests - how do I calculate it", "should_trigger": true },
{ "query": "Our agency proposed testing 12 creatives simultaneously. Sanity check the plan?", "should_trigger": true },
{ "query": "Design a test to find out whether the founder-story angle beats the discount angle", "should_trigger": true },
{ "query": "I want to test a new ad format without tanking performance - how do I structure it?", "should_trigger": true },
{ "query": "how do I set up ongoing creative experiments with a protected testing budget", "should_trigger": true },
{ "query": "success criteria to define before launching new ad variations", "should_trigger": true },
{ "query": "how do I avoid calling an ad winner too early", "should_trigger": true },
{ "query": "We're a startup with 40 conversions a month. Can we even A/B test our ads?", "should_trigger": true },
{ "query": "test plan for comparing 3 messaging angles in paid social", "should_trigger": true },
{ "query": "big swing testing vs isolating the hook - what's right for us?", "should_trigger": true },
{ "query": "champion vs challenger setup for testing new ads", "should_trigger": true },
{ "query": "how much of my budget should go to each cell in a creative test", "should_trigger": true },
{ "query": "Plan a creative experiment where we'll actually learn which element drove the lift", "should_trigger": true },
{ "query": "boss asked for a 'proper scientific test' of our new ads - where do I start", "should_trigger": true },
{ "query": "when is a creative test result trustworthy enough to act on - planning next quarter's tests", "should_trigger": true },
{ "query": "pre-launch checklist for an ad creative experiment", "should_trigger": true },
{ "query": "How do I test two YouTube ad cuts against each other properly?", "should_trigger": true },
{ "query": "we've got 3 concepts from the agency - decide the test order and budgets", "should_trigger": true },
{ "query": "what metrics should decide the winner in an ad creative test", "should_trigger": true },
{ "query": "our last creative test came back 'inconclusive' - design the next one so that can't happen", "should_trigger": true },
{ "query": "help me size an ad experiment: $80 CPA, need an answer in 3 weeks", "should_trigger": true },
{ "query": "do I need a control group when testing new ad creatives?", "should_trigger": true },
{ "query": "LinkedIn ads - want to test a practitioner pain angle vs an ROI angle, what's the plan", "should_trigger": true },
{ "query": "how do I keep the algorithm from ruining my creative test", "should_trigger": true },
{ "query": "How many assets per concept when testing new ad angles?", "should_trigger": true },
{ "query": "set up a fair bake-off between our in-house creative and the agency's", "should_trigger": true },
{ "query": "should the creative test have its own budget or share the campaign's?", "should_trigger": true },
{ "query": "I read most ad tests never reach significance - how should we test then?", "should_trigger": true },
{ "query": "hypothesis format for ad creative experiments", "should_trigger": true },
{ "query": "test roadmap for new ad concepts this quarter on a $25k/month account", "should_trigger": true },
{ "query": "can I trust a 4-day read on a new ad? planning our testing rules now", "should_trigger": true },
{ "query": "what does a good ad testing process look like before any money is spent", "should_trigger": true },
{ "query": "My winning ad's CPA doubled over three weeks - is it fatigued or something else?", "should_trigger": false },
{ "query": "How do I read the results of the creative test we launched last Monday?", "should_trigger": false },
{ "query": "Write 10 ad copy variants for our project management tool", "should_trigger": false },
{ "query": "Draft a creative brief for the UGC creator we hired", "should_trigger": false },
{ "query": "Check whether our Meta pixel purchase event is double-firing", "should_trigger": false },
{ "query": "Our whole ad account has been bleeding for two months - diagnose what's wrong", "should_trigger": false },
{ "query": "Which ad platforms should a $10k/month DTC brand be on?", "should_trigger": false },
{ "query": "How should I split $50k across Google, Meta and TikTok this quarter?", "should_trigger": false },
{ "query": "We're pacing 40% over budget mid-month - what do I do?", "should_trigger": false },
{ "query": "Should I switch this campaign from max conversions to target CPA bidding?", "should_trigger": false },
{ "query": "Build an audience targeting plan for our meal-kit launch", "should_trigger": false },
{ "query": "Which customers should seed our lookalike audience?", "should_trigger": false },
{ "query": "Design a retargeting sequence for cart abandoners", "should_trigger": false },
{ "query": "Audit the landing page our ads send traffic to", "should_trigger": false },
{ "query": "Set up an A/B test for our landing page headline", "should_trigger": false },
{ "query": "How do I A/B test email subject lines for our newsletter?", "should_trigger": false },
{ "query": "Sample size for an A/B test on our checkout flow redesign", "should_trigger": false },
{ "query": "We want to test two pricing tiers against each other - how long and how many users?", "should_trigger": false },
{ "query": "Score these five video hooks and tell me which deserve budget", "should_trigger": false },
{ "query": "Build a swipe file of competitor ads organized by hook type", "should_trigger": false },
{ "query": "Our campaign is profitable - how fast can we scale the budget?", "should_trigger": false },
{ "query": "Too many overlapping ad sets - which should we merge?", "should_trigger": false },
{ "query": "Mine our search terms report for negative keywords", "should_trigger": false },
{ "query": "Is a 3.2 ROAS healthy given our margins?", "should_trigger": false },
{ "query": "Meta says 120 purchases, Shopify says 74 - explain the gap", "should_trigger": false },
{ "query": "Write UGC scripts with hook variants for our skincare brand", "should_trigger": false },
{ "query": "What interview loop should we run for a media buyer hire?", "should_trigger": false },
{ "query": "How do I move from PPC specialist to creative strategist?", "should_trigger": false },
{ "query": "Set a maximum allowable CAC policy for the whole company", "should_trigger": false },
{ "query": "Is a carousel the right format for our top-of-funnel campaign?", "should_trigger": false },
{ "query": "Map the buying committee for our $80k ACV security product", "should_trigger": false },
{ "query": "Should we promote our CEO's LinkedIn posts as paid ads?", "should_trigger": false },
{ "query": "Adapt our ad copy for AI assistant answer placements", "should_trigger": false },
{ "query": "Build me a watch list of paid media newsletters and podcasts", "should_trigger": false },
{ "query": "A/B test my onboarding flow - control vs the new tooltip tour", "should_trigger": false },
{ "query": "How do I run a feature-flag experiment for our new checkout service?", "should_trigger": false },
{ "query": "Design a survey experiment to measure brand recall", "should_trigger": false },
{ "query": "statistical significance calculator for my NPS survey results", "should_trigger": false },
{ "query": "Which of these two logo designs should we pick? Set up a test", "should_trigger": false },
{ "query": "Test two versions of our app store screenshots against each other", "should_trigger": false },
{ "query": "Help me plan a geo holdout to measure whether our Meta ads are incremental at all", "should_trigger": false },
{ "query": "Can we split test our SEO title tags?", "should_trigger": false },
{ "query": "Set up a multivariate test on the homepage in our CRO tool", "should_trigger": false },
{ "query": "Compare our two email nurture sequences and tell me which converts better", "should_trigger": false },
{ "query": "We printed two direct mail postcard versions - how do we measure which one works?", "should_trigger": false },
{ "query": "How many user interviews do I need before the findings are reliable?", "should_trigger": false },
{ "query": "Run a power analysis in R for our product experiment", "should_trigger": false },
{ "query": "Why did my ad get rejected by Meta's ad review?", "should_trigger": false },
{ "query": "The test ended - write up the results deck for leadership", "should_trigger": false },
{ "query": "How often should we rotate and refresh our ad creatives?", "should_trigger": false },
{ "query": "Which thumbnail should I use for our YouTube video?", "should_trigger": false },
{ "query": "Pick the best subject line for tomorrow's product launch email", "should_trigger": false },
{ "query": "How do I test whether our new sales pitch deck lands better with prospects?", "should_trigger": false },
{ "query": "Analyze last quarter's creative tests and tell me what patterns won", "should_trigger": false },
{ "query": "Set up experiments on our pricing page to lift signups", "should_trigger": false },
{ "query": "What's a good CTR benchmark for Facebook ads in fashion?", "should_trigger": false },
{ "query": "Brainstorm 10 new ad concepts for our spring campaign", "should_trigger": false },
{ "query": "Our Meta A/B test finished with 'no clear winner' - what does that mean?", "should_trigger": false },
{ "query": "Split test two versions of our Shopify product page", "should_trigger": false }
]
}
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |