notque/vexjoy-agent / skills/content/content
content
UnverifiedContent operations: editorial calendar, marketing, publishing, social media management.
references/llm-marketing-failure-modes.md
View on GitHub---
title: LLM Marketing Failure Modes — Detection, Examples, Mitigation
domain: marketing
level: 3
skill: marketing
---
# LLM Marketing Failure Modes
> **Scope**: Comprehensive taxonomy of how LLMs fail at marketing tasks, with detection heuristics, concrete examples, and mitigation strategies. Load selectively when any marketing mode needs quality enforcement.
> **Generated**: 2026-05-05 — failure modes evolve with model capabilities. Some modes that fail today may improve; new failure modes may emerge.
> **Shared base**: Universal LLM failure modes (hallucination, overconfidence, generic output, arithmetic errors, stale knowledge) are documented in `skills/shared-patterns/llm-domain-failure-modes-base.md`. This file covers marketing-specific failures only.
---
## Overview
LLMs are useful for marketing tasks but fail in predictable, repeatable ways. These failures are not random -- they follow patterns rooted in training data distribution, lack of grounding, and optimization for plausibility over accuracy. Every marketing mode in this skill must guard against these failures explicitly.
The failures cluster into three categories:
1. **Content quality failures** — output that sounds right but is wrong or generic
2. **Data integrity failures** — fabricated, hallucinated, or unverifiable claims
3. **Strategic failures** — recommendations that optimize the wrong thing
---
## Content Quality Failures
### 1. Generic Copy
**What happens**: Output uses interchangeable marketing language that could apply to any product in any industry. "Streamline your workflow", "unlock the power of", "take your X to the next level", "in today's fast-paced world".
**Why it happens**: Training data is saturated with marketing copy. The most statistically likely next token in a marketing context is a cliche. The model optimizes for plausibility, and generic phrases are maximally plausible.
**Detection heuristics**:
- Substitute a different product name into the copy. If it still works, the copy is generic.
- Search the opening sentence verbatim. If similar phrasing appears on 100+ websites, it's generic.
- Ask: does this sentence tell you something specific about this product for this audience? If no, it's generic.
**Examples of failure**:
| Generic (Reject) | Specific (Accept) |
|---|---|
| "Streamline your marketing workflow with our powerful platform" | "Run your quarterly campaign from brief to report in one dashboard instead of switching between Notion, Sheets, and Slack" |
| "Unlock the power of data-driven marketing" | "See which blog posts drive demo requests -- not just traffic -- and kill the ones that don't" |
| "Take your email marketing to the next level" | "Stop guessing send times. Our model picks the hour each subscriber is most likely to open." |
| "In today's fast-paced digital landscape" | [Delete this sentence. It says nothing.] |
| "Our innovative solution empowers teams to achieve more" | "Three-person marketing teams use this to publish 4x more content without adding headcount" |
**Mitigation**:
1. Require user context (audience, product, constraints) before generating any copy
2. After generating, run the substitution test: replace the product name. If the copy still works for a competitor, rewrite.
3. Ban list of phrases to auto-reject: "streamline", "unlock the power", "take to the next level", "in today's [adjective] [noun]", "leverage", "empower", "revolutionize", "cutting-edge", "game-changing", "seamless", "best-in-class", "holistic", "synergy"
4. Every claim must reference a specific feature, outcome, or audience characteristic
---
### 2. Template Regurgitation
**What happens**: Output matches the structure and phrasing of commonly-seen templates in training data rather than responding to the user's specific situation. Blog posts follow the exact same flow as thousands of training examples. Landing pages use the exact SaaS template structure regardless of product type.
**Why it happens**: Patterns from training data are strong attractors. A blog post about marketing starts the same way as every training example of a blog post about marketing. The model follows the statistical path of least resistance.
**Detection heuristics**:
- Does the output match a recognizable template structure without the user requesting that structure?
- Could this content have been generated without any user context? If yes, the user context didn't influence the output.
- Do the examples, stories, or data points feel "stock" rather than specific to this company?
**Mitigation**:
1. Always ground generation in user-specific context: their product, their audience, their data, their competitors
2. Challenge every example: is this from the user's world or from generic marketing lore?
3. If the user hasn't provided enough context, ask rather than fill with defaults
4. Vary structure deliberately -- not every blog post needs the same H2 pattern
---
### 3. Tone Deafness
**What happens**: The model applies the same tone regardless of context. An incident response email sounds like a product launch. A B2B enterprise whitepaper reads like a DTC Instagram caption. A win-back email sounds enthusiastic instead of empathetic.
**Why it happens**: Without explicit tone guidance, the model defaults to a generically positive, enthusiastic tone -- the most common register in marketing training data.
**Detection heuristics**:
- Read the output aloud. Does the tone match how a human in this situation would actually communicate?
- Compare tone across different content types in the same generation session. If they all sound the same, tone adaptation failed.
- Check: would the recipient find this tone appropriate or off-putting?
**Examples of failure**:
| Situation | Tone-Deaf (Reject) | Appropriate (Accept) |
|-----------|--------------------|--------------------|
| Incident response | "We're excited to share an update about the service interruption!" | "We know this disruption affected your work. Here's what happened, what we've fixed, and what we're doing to prevent it." |
| Win-back email | "We've been busy building amazing new features and can't wait for you to try them!" | "We noticed you haven't logged in recently. We'd like to understand what happened -- your feedback helps us get better." |
| Price increase notice | "Great news! We're evolving our pricing to serve you better." | "We're raising prices on [date]. Here's what changes, what stays the same, and why." |
| Enterprise security whitepaper | "Check out these awesome security features! " | "This document details our security architecture, compliance certifications, and data handling practices." |
**Mitigation**:
1. Explicitly set tone for every piece: what situation is this, who's receiving it, what emotional state are they likely in?
2. Situation-tone mapping table (see BRAND mode reference): incident = transparent/empathetic, launch = confident, bad news = honest/respectful
3. Review every piece against the question: "Would I want to receive this message in this situation?"
---
## Data Integrity Failures
### 4. Fabricated Metrics
**What happens**: The model generates plausible-sounding statistics with no source. "Companies using X see 340% improvement in Y." "78% of marketers agree that Z." These numbers are statistically plausible but entirely invented.
**Why it happens**: Training data is full of marketing stats. The model generates numbers that fit the distributional pattern of real statistics. It cannot distinguish between generating plausible text and citing real data.
**Detection heuristics**:
- Every number should have a source. No source = fabricated until proven otherwise.
- Round numbers (10%, 50%, 100%) with no source are especially suspect.
- Improbable precision ("73.2% of B2B marketers") without a named study is almost certainly fabricated.
- "Studies show" or "research indicates" without naming the study = fabricated.
**Examples of failure**:
| Fabricated (Reject) | Grounded (Accept) |
|--------------------|--------------------|
| "Studies show that 78% of consumers prefer personalized emails" | "According to Litmus's 2024 State of Email report, personalized subject lines increase open rates by 26% on average" |
| "Companies see a 5x return on their content marketing investment" | "Content marketing generates leads at approximately 62% lower cost than outbound, according to Demand Metric research" |
| "Email marketing has an ROI of 4200%" | "The DMA's 2024 report estimates email marketing ROI at approximately $36 for every $1 spent, though actual returns vary by industry and list quality" |
| "Our customers experience 340% improvement in conversion rates" | "In a case study with [Customer], conversion rates improved from 2.1% to 4.8% over 6 months" |
**Mitigation**:
1. Never generate statistics without a cited source
2. If no source exists, use qualitative framing: "teams report significant improvement" or "a common industry benchmark suggests"
3. When citing benchmarks, state the source, date, and any caveats
4. For user's own metrics: only report what they've provided. Never estimate or "round up" their data.
5. Flag illustrative numbers explicitly: "For illustration purposes, if your conversion rate is 2%..."
---
### 5. Hallucinated Competitor Data
**What happens**: The model generates plausible competitor information -- pricing, features, market share, employee count, funding -- that is partially or entirely wrong. It fills knowledge gaps with confident-sounding fabrication.
**Why it happens**: The model has training data about many companies but it's outdated, incomplete, or wrong. It generates the most statistically plausible completion, which for a competitor analysis means inventing details that sound right.
**Detection heuristics**:
- Any competitor data point without a web search result or user-provided source is suspect
- Pricing data older than 6 months is likely outdated
- Employee counts, funding amounts, and market share without a dated source = probably fabricated
- Feature comparisons that make the user's product look better than competitors in every dimension = bias, not analysis
**Examples of failure**:
| Hallucinated (Reject) | Grounded (Accept) |
|-----------------------|--------------------|
| "Competitor X has raised $50M in Series B funding" (no source) | "According to Crunchbase (accessed 2026-05-05), Competitor X raised $50M in Series B in March 2025" |
| "Competitor X's pricing starts at $99/month" (from memory) | "Based on their pricing page as of 2026-05-05, Competitor X's pricing starts at $99/month for up to 5 users" |
| "Competitor X has approximately 500 employees" (guess) | "Could not verify current employee count for Competitor X. LinkedIn suggests approximately 400-600 as of early 2026, but this should be verified." |
**Mitigation**:
1. All competitor claims must come from web search results or user-provided data
2. Date-stamp every competitor data point: "as of [date]"
3. State explicitly when data could not be verified: "Unable to confirm Competitor X's current pricing"
4. Never fill gaps with plausible fiction -- leave them as gaps
5. Include a freshness disclaimer on every competitive analysis
---
### 6. Unsubstantiated Claims
**What happens**: The model uses superlatives and absolute claims without evidence. "The best solution for X." "The only platform that does Y." "#1 in customer satisfaction." "Fastest time to value."
**Why it happens**: Marketing training data is full of superlatives. The model reproduces the register of marketing copy, including claims that require legal substantiation.
**Detection heuristics**:
- Flag every superlative: best, fastest, only, #1, most, leading, top, ultimate, guaranteed
- Check: is there evidence to support this claim? If not, it needs qualification or removal.
- Comparative claims ("better than X", "faster than Y") without data = potentially actionable legally
**Mitigation**:
1. Auto-flag superlatives during review
2. Three options for each flagged claim: substantiate (add evidence), qualify ("one of the fastest"), remove
3. Comparative claims require specific, verifiable data
4. "Best" and "only" require documented proof or must be removed
5. Include a legal/compliance review flag for any content with comparative or superlative claims
---
## Strategic Failures
### 7. Vanity Metric Optimization
**What happens**: Recommendations optimize for visible but meaningless metrics: follower count, impressions, page views, email list size. These feel good but don't drive business outcomes.
**Why it happens**: Vanity metrics dominate marketing discourse in training data. "Grow your followers" appears far more often than "reduce CAC by improving lead quality."
**Detection heuristics**:
- For every recommended metric, ask: "What business decision does this metric inform?"
- If a metric can go up while the business gets worse, it's a vanity metric
- Impressions, followers, and page views are vanity metrics unless tied to downstream conversion
**Examples of failure**:
| Vanity (Reject) | Business (Accept) |
|-----------------|-------------------|
| "Goal: reach 10,000 LinkedIn followers" | "Goal: generate 50 demo requests from LinkedIn content per quarter" |
| "Metric: increase blog traffic by 200%" | "Metric: increase blog-attributed pipeline by 30%" |
| "KPI: grow email list to 50,000 subscribers" | "KPI: increase email-sourced MQLs by 25% while maintaining list quality above 30% engagement rate" |
**Mitigation**:
1. Every metric must connect to a business outcome (revenue, pipeline, retention, cost reduction)
2. If a metric is purely a leading indicator, state what it leads to and how it connects
3. Primary KPIs should be business outcomes. Secondary KPIs can be leading indicators with explicit connection stated.
4. Challenge any recommendation that optimizes engagement without defining what engagement produces
---
### 8. Keyword Stuffing
**What happens**: SEO recommendations degenerate into cramming the target keyword into every possible position. Content becomes unreadable and ironically performs worse in modern search algorithms.
**Why it happens**: Older SEO practices in training data emphasized keyword density. The model reproduces these outdated tactics alongside modern SEO advice, creating contradictory output.
**Detection heuristics**:
- Keyword density above 2% in body text
- Same keyword appearing in 3+ consecutive sentences
- Awkward phrasing where the keyword has been forced in ("for marketing automation software that automates marketing software...")
- Every H2 containing the exact primary keyword
**Examples of failure**:
| Keyword-Stuffed (Reject) | Natural (Accept) |
|--------------------------|-------------------|
| "Email marketing software is essential for email marketing. The best email marketing software helps with email marketing campaigns." | "The right platform automates the repetitive parts -- scheduling, segmentation, A/B tests -- so you can focus on the message, not the mechanics." |
| H2: "Best Email Marketing Software Features" / H2: "Email Marketing Software Pricing" / H2: "Email Marketing Software Reviews" | H2: "Features That Actually Matter" / H2: "What It Costs" / H2: "What Users Say" |
**Mitigation**:
1. Primary keyword appears in: headline, first paragraph, one subheading, meta description, URL. That's it.
2. Keyword density cap: 1-2% in body text
3. Use semantic variations and related terms instead of repeating the exact keyword
4. Read aloud test: if it sounds unnatural, the keyword is forced
5. Modern SEO rewards comprehensive topic coverage, not keyword repetition
---
### 9. Channel-Agnostic Recommendations
**What happens**: The model recommends the same channels and tactics regardless of the user's audience, budget, capacity, or business model. "Be on TikTok, LinkedIn, X, Instagram, YouTube, start a podcast, launch a newsletter, and attend 3 conferences." No one can do all of these.
**Why it happens**: Training data contains marketing advice that is aspirational and comprehensive rather than constrained and practical. The model optimizes for completeness over feasibility.
**Detection heuristics**:
- More than 3 active channels recommended without a team size assessment
- No prioritization or sequencing in channel recommendations
- Channels recommended that don't match the audience (TikTok for enterprise B2B, LinkedIn for consumer)
- No effort estimates attached to channel recommendations
**Mitigation**:
1. Always assess capacity: solo operator = max 2 channels, small team = max 3
2. Every channel recommendation must include: why this channel for this audience, effort estimate, expected timeline to results
3. Sequence, don't stack: "Start with X, add Y after 3 months if X is working"
4. Historical performance data trumps best practices -- if the user's email works, lean into email
---
### 10. Recency Bias in Strategy
**What happens**: Recommendations over-weight recent trends and platforms while ignoring proven fundamentals. "AI-powered personalization" appears in every recommendation. New platforms are recommended without considering sustainability.
**Why it happens**: Recent training data is trend-heavy. The model amplifies what's currently being discussed, not what's consistently effective.
**Detection heuristics**:
- Does every recommendation mention AI, even when the task doesn't need it?
- Are new platforms recommended over established ones without a clear audience-fit argument?
- Is the strategy fundamentally different from what would have been recommended 2 years ago, and if so, why?
**Mitigation**:
1. Separate fundamentals (audience research, clear messaging, channel fit) from trends
2. New channels/tools require a specific argument for why they fit this user's situation
3. "Everyone is doing X" is not a strategy rationale
4. Test new approaches at 10-20% of budget/effort, not as primary strategy
---
## Quality Gate: Pre-Delivery Checklist
Before delivering any marketing output, verify:
| Check | How | Fail Action |
|-------|-----|-------------|
| No generic phrases from ban list | Search for banned phrases | Rewrite with specifics |
| All statistics have sources | Check every number for attribution | Remove, qualify, or source |
| All competitor data is sourced and dated | Verify against search results | Flag unverified, state gaps |
| No unsupported superlatives | Flag best/fastest/only/#1 | Substantiate, qualify, or remove |
| Metrics tied to business outcomes | Ask "what decision does this inform?" for each KPI | Replace vanity metrics with business metrics |
| Tone matches situation | Read aloud, check against situation-tone mapping | Adjust tone |
| Keywords not stuffed | Check density < 2%, read aloud | Reduce keyword frequency |
| Channel recommendations match capacity | Count channels vs. team size | Reduce to max 3 |
| Content reflects user's specific context | Substitution test: would this work for a competitor? | Rewrite with user specifics |
| No fabricated examples or case studies | Verify every named example | Remove or mark as illustrative |
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |