Skip to content

afeezadebayo/research-learning-skills / survey-design

survey-design

Verified

Design surveys that produce trustworthy data: objectives tied to decisions, population and sample size with margins of error, neutral single-idea questions, answer formats and scales, ordering and skip logic, pilots, invitations and reminders, cleaning, segment analysis and reports with limitations. Use whenever someone needs to create a survey or questionnaire, run customer or employee feedback surveys, prioritise features with survey data, calculate sample size or fix leading survey questions.

Survey Design

Surveys answer "how many" and "how common" questions that interviews cannot: what share of customers want a feature, how satisfaction varies by segment, which problems are most frequent. They are easy to send and easy to get wrong. Leading or double-barrelled questions, unbalanced scales, self-selected samples, long questionnaires that people abandon and results reported without margins of error produce numbers that look precise but mislead.

This skill designs surveys that produce trustworthy data: defining objectives, choosing who to survey and how many, writing clear unbiased questions, choosing answer formats and scales, ordering and testing the questionnaire, maximising responses, analysing results with appropriate caution and reporting findings with sample sizes and limitations.

Operating principles

  1. Objectives decide every question. If a question does not inform a decision, cut it.
  2. Right people, enough of them. Who answers matters more than how many.
  3. Neutral, simple wording. One idea per question, no leading language.
  4. Short wins. Five to ten minutes keeps completion and quality high.
  5. Test before sending. Pilot with a few people to catch confusing questions.
  6. Report with limits. Sample size, who responded and margin of error.

Question writing rules

ProblemBadBetter
Leading"How much do you love our new dashboard?""How satisfied are you with the new dashboard?"
Double-barrelled"Is our product fast and easy to use?"Two questions: speed, ease of use
Vague time"Do you often use reports?""In the last 30 days, how many times did you use reports?"
Jargon"How useful is our NRR dashboard?"Use plain terms or explain
Hypothetical"Would you pay for feature X?"Ask about current spend and priority of problems
Missing optionsChoices that do not cover all answersAdd "other" and "not applicable"

Answer formats

FormatUse forNotes
Single choiceOne answer from a listMake options mutually exclusive
Multiple choiceSeveral answersLimit options; randomise order where sensible
Rating scale (5 or 7 points)Satisfaction, agreement, likelihoodBalanced, labelled endpoints
RankingPrioritiesKeep to 5 to 7 items
NumericCounts, amountsGive units and ranges
Open textReasons, ideasUse sparingly; one or two per survey
Net Promoter ScoreLikelihood to recommend, 0 to 10Add a follow-up "why" question

Sample size rough guide

For a simple percentage from a random sample, the margin of error at 95% confidence is roughly 1 divided by the square root of the sample size: about ±10% with 100 responses, ±7% with 200, ±5% with 400 and ±3% with 1,000. Self-selected samples can be biased regardless of size, so who responds matters as much as how many.

When to use, and when not to

Use it for customer satisfaction, feature prioritisation, market sizing inputs, employee engagement, event feedback and validating patterns found in interviews.

Don't use surveys to understand why people behave in complex ways (interviews do that better) or when you cannot reach a representative group.

Inputs

  • Objectives and decisions the survey informs.
  • Target population and how to reach them.
  • Hypotheses from interviews or data.
  • Tool (Google Forms, Typeform, SurveyMonkey or similar).
  • Timeline and incentive budget.

Workflow

Step 1. Write objectives

Three to five objectives, each linked to a decision.

Step 2. Define the population and sample

Who exactly, how they will be reached, target number of responses, segments needed for comparison.

Step 3. Draft questions per objective

Use the writing rules; one idea per question; plain language.

Step 4. Choose formats and scales

Consistent scales throughout; include "not applicable" where needed.

Step 5. Order the survey

Easy, relevant questions first; sensitive and demographic questions last; group by topic; use logic to skip irrelevant questions.

Step 6. Pilot

Five to ten people complete it while thinking aloud; fix confusion, length and errors.

Step 7. Send and remind

Clear invitation with purpose, time required and privacy; one or two reminders; incentives if appropriate.

Step 8. Clean and analyse

Remove duplicates, speeders and test responses; analyse by segment; report sample sizes and margins.

Step 9. Report

Key findings with numbers, charts, comparisons, open-text themes and limitations.

Survey invitation template

Subject: 5 minutes to help shape <product or service>
We're asking <who> about <topic> so we can <decision or improvement>.
It takes about 5 minutes and closes on <date>.
<Incentive, if any>. Answers are <anonymous / confidential> and used only for <purpose>.
<Link>

Analysis checklist

  • Response rate and who responded compared with the target population.
  • Results by key segments, with sample sizes shown.
  • Margins of error for main percentages.
  • Open-text responses coded into themes with counts.
  • Differences tested or described cautiously when samples are small.

Scale design tips

  • Use the same direction for all scales (for example 1 = very dissatisfied, 5 = very satisfied).
  • Label every point or at least both ends and the midpoint.
  • Keep a neutral midpoint unless you deliberately want to force a choice.
  • Avoid mixing 5-point and 10-point scales in one survey.

Coding open-text answers

Read a sample of 50 to 100 answers, create a short list of themes, then tag every answer with one or more themes and count them. Keep a few verbatim quotes per theme. AI can draft themes and tags quickly; check a sample of its tagging by hand before reporting counts.

Avoiding survey fatigue

Survey each customer group no more than a few times a year, coordinate across teams so people are not asked the same questions twice and share what changed because of their answers, which raises future response rates.

Report response rates every time.

Edge cases and judgement calls

SituationWhat to do
Low response rateCheck who answered; weight carefully or report as indicative
Small segmentsCombine or report as directional only
Sensitive topicsAnonymous surveys; careful wording; data protection
Customers already over-surveyedShorter surveys; fewer sends; use existing data
Results conflict with interviewsLook for sampling or wording issues; investigate further
International audiencesTranslate and back-translate; adapt examples

Handoffs to related skills

  • Planning: research-planning
  • Interviews to explore first: expert-user-interviews
  • Synthesis: research-synthesis
  • Presenting results: the Communication and Presentations data-storytelling skill
  • Customer feedback programmes: the CRM and Lifecycle skills

Worked example

A product team planned a 40-question survey asking customers which of 15 features they wanted. The redesign set three objectives, cut the survey to 12 questions, replaced "would you use" questions with questions about how often customers faced each problem in the last month and how they solved it, added a ranking of five problems and piloted with eight customers, which removed two confusing questions. With 620 responses (about ±4% margin), the team found two problems far more frequent than the rest and prioritised them, confirming the result with follow-up interviews.

Common mistakes

  • Long surveys with no clear objectives.
  • Leading or double-barrelled questions.
  • Asking about hypothetical behaviour.
  • Reporting percentages without sample sizes.
  • Treating self-selected responses as representative.

Output format

1. Objectives and decisions
2. Population, sample and margin of error
3. Questionnaire with formats and logic
4. Pilot findings and changes
5. Invitation and reminder plan
6. Analysis plan and report outline

Self-check before you answer

  • Does every question map to an objective?
  • Are questions neutral, single-idea and about real behaviour?
  • Is the sample appropriate, with margin of error stated?
  • Was the survey piloted before sending?

Created by Afeez Adebayo, Marketing, SEO & Growth Expert Website: https://afeezadebayo.com LinkedIn: https://www.linkedin.com/in/afeez-adebayo/ X: https://x.com/za_icon Source: https://afeezadebayo.com/skills-md/productivity/research-learning/survey-design/

Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). You may copy, share and adapt this skill, including for commercial use, as long as you credit Afeez Adebayo and link to the source URL above. Keep this notice in any copy or adapted version.

brainstormingobra/superpowersYou MUST use this before any creative work - creating features, building components, adding functionality, modifying behavior, or planning anything new, in software or out of it (a talk, a business, a renovation).Stars: 322kUpdated Claude CodeCodexVerifiedInstall
brand-guidelinesanthropics/skillsApplies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. Use it when brand colors or style guidelines, visual formatting, or company design standards apply.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
frontend-designanthropics/skillsGuidance for distinctive, intentional visual design when building new UI or reshaping an existing one. Helps with aesthetic direction, typography, and making choices that don't read as templated defaults.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
mcp-builderanthropics/skillsGuide for creating high-quality MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools. Use when building MCP servers to integrate external APIs or services, whether in Python (FastMCP) or Node/TypeScript (MCP SDK).Stars: 198kUpdated Claude CodeCodexVerifiedInstall