Survey Design Template for User Research

For a standard product survey with 5-point Likert scales, 100-200 responses give you a margin of error of 5-7% at 95% confidence. For segmented analysis (comparing free vs. paid, mobile vs. desktop), you need 50+ responses per segment. If your population is small (e.g., 50 Enterprise accounts), survey everyone (census) and skip statistical significance testing.

Incentives increase response rates but can attract low-quality responses from people who want the reward but do not care about the questions. For customer surveys, a raffle (e.g., 5x $50 gift cards among all respondents) works well. The incentive should be enough to motivate but not enough to attract professional survey takers. Avoid per-response payments.

If your response rate is below 5%, check three things: email deliverability (are your emails landing in spam?), subject line (does it clearly state the purpose and time required?), and timing (did you send on a weekday morning?). Send one reminder 3-5 days after the initial invitation. Two reminders is the maximum before you annoy users. If the rate is still low, consider an in-app intercept survey instead.

Surveys answer "how many" and "how much" questions: they quantify known issues across a population. Interviews answer "why" and "how" questions: they uncover unknown problems and motivations. The [Product Discovery Handbook](/discovery-guide) recommends using interviews first to identify themes, then surveys to validate the prevalence of those themes across your user base.

Thematic coding. Read all responses once to identify recurring themes (typically 5-10 themes emerge). Then re-read each response and tag it with 1-3 themes. Count the frequency of each theme. Have a second person independently code a sample of 30-50 responses and compare agreement (inter-rater reliability). If two coders disagree on more than 20% of responses, your theme definitions are too vague. ---