Skip to main content

UX and research · Beginner · 14 min

Write unbiased survey questions and answer scales

By Padding · Updated 18 Nov 2026
Any document editor or spreadsheet · November 2026

You will have a corrected set of survey questions, each with an answer scale that fits what it measures and a one-line reason for every change you made.

On this page

A survey answer is only as good as the question that produced it. Interviews let you notice confusion and ask again; a survey does not. If a question bundles two topics, hints at the answer you want, or leaves "recently" open to interpretation, every respondent quietly answers a slightly different question, and the chart you make afterwards averages them together as if they were the same.

This tutorial works through a broken survey rather than a list of rules. You start with eight flawed questions, find what is wrong with each, rewrite it, and choose an answer scale that matches what the question is measuring. You record a reason for every change, because "I made it better" does not survive a stakeholder asking why their favourite question was reworded.

The practice survey belongs to Northgate Bikes, a fictional city bike-share scheme that has just released a new app for unlocking bikes. The product team wants to know how riders use the service and what to fix first. If you have your own draft survey, run your questions through the same steps alongside the practice set.

By the end you will have one table: original question, fault, rewritten question with its answer options, and the reason for the change.

Step 1 · Set up a fix log and paste in the flawed questions

Make a table with four columns: Original, Fault, Rewrite (with answer options) and Reason. A spreadsheet works well because you can sort by fault later; a table in a document is fine too.

Paste these eight questions into the Original column, answer options included. This is the draft the Northgate team handed you:

  1. How satisfied are you with the price and reliability of our bikes? (Very dissatisfied, Dissatisfied, Neutral, Satisfied, Very satisfied)
  2. Don't you agree that the new app makes unlocking a bike faster? (Yes, No)
  3. How often do you ride with Northgate? (Rarely, Sometimes, Often)
  4. With traffic and pollution getting worse every year, how important is it to you that the city builds more bike lanes? (Not important, Important, Very important)
  5. How long is your typical ride? (0–10 minutes, 10–20 minutes, 20–30 minutes, 30 minutes or more)
  6. Have you ever had a problem with a bike? (Yes, No)
  7. "Docking stations are conveniently located." (Strongly disagree, Disagree, Neither, Agree, Strongly agree)
  8. What do you mainly use Northgate for? (Commuting to work, Leisure)

Before reading on, spend two minutes writing your own guess in the Fault column for each one. Comparing your guesses with the steps below is how the patterns stick.

Check it worked

Your table has four labelled columns and eight rows, each with the original question and its answer options in the first column, and a first guess at the fault in the second.

Step 2 · Split questions that ask two things at once

A double-barrelled question asks about two things but allows one answer. Question 1 asks about price and reliability. A rider who finds the bikes cheap but often broken has no honest answer: "Neutral" could mean "both are middling" or "one is great and one is awful". You cannot tell which, so you cannot act on it.

The test: look for "and", "or", and commas joining two nouns or verbs inside one question. If a person could reasonably feel differently about each part, split it.

Rewrite question 1 as two:

  • 1a. How satisfied are you with the price of a Northgate ride?
  • 1b. How satisfied are you with the reliability of Northgate bikes?

Keep the same answer options for both for now; you revisit the scale in a later step. In the Reason column write something like: "Asked about two things with one answer; riders who rate price and reliability differently could not answer honestly."

Scan the rest of the set for the same fault. Question 4 is close: it bundles a premise (traffic and pollution are getting worse) with the question. That is a different fault, which you fix next.

Check it worked

Question 1 is now two rows, 1a and 1b, each about exactly one thing. None of your rewritten questions contains "and" or "or" joining two separate topics.

Step 3 · Strip out leading and loaded wording

A leading question signals the expected answer. Question 2 opens with "Don't you agree", which invites a yes, and it states the conclusion ("makes unlocking faster") as the thing to agree with. A loaded question carries an assumption or emotional framing. Question 4 tells the reader that traffic and pollution are getting worse before asking whether bike lanes matter, so the respondent is primed to say they do.

Fix both by asking what the person experienced, in neutral words, with options that make every answer equally easy to give.

Question 2 rewrite:

  • 2. Compared with the old app, how long does unlocking a bike take with the new app? (Much less time, A little less time, About the same, A little more time, Much more time, I did not use the old app)

The options run both ways around a middle point, so "slower" is as easy to pick as "faster". The last option catches people who cannot compare at all, who would otherwise guess.

Question 4 rewrite:

  • 4. How important or unimportant is it to you that the city builds more bike lanes? (Not at all important, Slightly important, Moderately important, Very important, Extremely important)

The preamble is gone. Writing "important or unimportant" in the question gives both directions equal standing. If the team needs the traffic context for their own analysis, they can ask about it as a separate question.

Other words to watch for in your own drafts: "our great new", "unfortunately", "how much did you enjoy", "wouldn't", "isn't it", and any question that names the team's preferred outcome.

Check it worked

Read questions 2 and 4 aloud. Neither one contains an opinion, a reason, or a hint about which option the team hopes for, and each set of options lets a negative answer be given as easily as a positive one.

Step 4 · Pin down vague time frames and quantities

Vague words let each respondent pick their own yardstick. In question 3, "Rarely", "Sometimes" and "Often" mean different things to a daily commuter and a weekend rider: one person's "often" is twice a week, another's is twice a day. Question 6 has the same problem twice: "ever" covers years, and "a problem" covers anything from a squeaky bell to a brake failure.

Replace vague words with a stated period and concrete amounts.

Question 3 rewrite:

  • 3. In the last four weeks, on how many days did you ride a Northgate bike? (0 days, 1–3 days, 4–7 days, 8–14 days, 15 or more days)

A recent, fixed window (the last four weeks) is easier to recall than "usually", and the answers are counts, not feelings about counts.

Question 6 rewrite, split by what the team needs to know:

  • 6a. In the last four weeks, did any Northgate bike you tried to use have a fault that stopped you riding it? (Yes, No, I did not try to ride in the last four weeks)
  • 6b. If yes: what was the fault? (Flat or damaged tyre, Brakes, Chain or gears, Would not unlock, Would not dock, Other (please describe))

The fault is now defined ("stopped you riding it"), the period is fixed, and 6b gives the team something to act on. Note the reason: "'Ever' and 'a problem' were open to interpretation; now a fixed period and a defined fault."

Check it worked

Every question that asks about behaviour names a time period ("in the last four weeks") or a specific event. No answer option in your table is a frequency word such as "often", "sometimes", "regularly" or "recently".

Step 5 · Make answer options cover everyone, once

For questions with categories or ranges, each respondent should find exactly one option that fits. Two faults break this.

Overlapping ranges. In question 5, a ten-minute ride fits both "0–10 minutes" and "10–20 minutes". Use boundaries that do not touch:

  • 5. Thinking of your most recent Northgate ride, about how long was it? (Under 10 minutes, 10 to 19 minutes, 20 to 29 minutes, 30 minutes or more, I have not ridden yet)

"Most recent ride" also replaces "typical", which asks people to average in their heads.

Missing options. Question 8 offers only commuting and leisure. A student riding to lectures, a nurse riding between sites or someone running errands is forced to pick one that does not fit. Add the common cases you can foresee and an escape route:

  • 8. What did you use Northgate for on your most recent ride? (Travelling to or from work, Travelling to or from study, Errands or shopping, Leisure or exercise, Getting to or from public transport, Other (please describe))

Asking about one ride instead of "mainly" removes the averaging problem again. If the team truly needs "main use", ask it as select-one with the same list; if they want every use, make it select-all and say so in the question ("Select all that apply").

Check it worked

For each question with ranges, pick a number that sits on a boundary (10 minutes, 20 minutes, 4 days) and confirm it fits exactly one option. Every category question has an "Other (please describe)" or "None of these" option, or a written reason in your table for leaving it out.

Step 6 · Match each scale to what you are measuring

Now look at the scales themselves. A scale should measure the same kind of thing the question asks about. Four types cover most product surveys:

  • Frequency or count: how often something happened. Use numeric ranges over a stated period, as in question 3.
  • Satisfaction: how well something met a need, from dissatisfied to satisfied. It has two directions, so it needs a neutral middle.
  • Importance or extent: how much something matters, from none to a lot. It has one direction, so it starts at zero ("Not at all important") and has no negative side, as in question 4.
  • Comparison: better or worse than something else, with "about the same" in the middle, as in question 2.

Then fix the agreement scale in question 7. Agree/disagree statements invite people to agree with whatever is put in front of them, a pattern called acquiescence bias, and the statement wording does the leading. Rewrite the statement as a question about the thing itself, with options that describe it directly:

  • 7. How convenient or inconvenient are the locations of Northgate docking stations for the trips you make? (Very inconvenient, Somewhat inconvenient, Neither convenient nor inconvenient, Somewhat convenient, Very convenient)

Three more checks for every scale in your table:

  • Label every point with words. "1 to 10, where 1 is poor" leaves 2 to 9 for each person to interpret. Five labelled points are enough for most single questions.
  • Keep the same direction throughout. If negative is on the left in question 1a, it is on the left everywhere. Respondents stop reading labels after a few questions and answer by position.
  • Offer "Does not apply" separately from the neutral middle. Someone who never used the old app is not "neutral" about it. Mixing the two hides real indifference behind people who could not answer.

Revisit 1a and 1b: satisfaction is two-directional, so their five-point scale from "Very dissatisfied" to "Very satisfied" with a neutral middle is right. Rename "Neutral" to "Neither satisfied nor dissatisfied" so the middle point is labelled in the same terms as the ends.

Check it worked

Add a fifth column, Scale type, and fill it in for every row: frequency, satisfaction, importance, comparison or category. Each scale's options match its type, every point has a word label, negative answers sit on the same side throughout, and no question in your table is an agree/disagree statement.

Step 7 · Test the set by reading it aloud with one person

You cannot spot your own ambiguity because you know what you meant. Ask one person who is not on the team to read each rewritten question aloud, answer it, and say in their own words what they think it is asking. This is a short version of what researchers call a cognitive pretest.

Do not explain anything while they read. Note in a sixth column, Pretest note, wherever they:

  • paused, re-read, or asked what a word meant
  • described the question differently from your intent
  • could not find an option that fitted, or chose one with a caveat ("sort of the second one")

For each note, make one more change and add the reason. A common catch in the practice set: someone who rides daily but only in summer may ask whether "the last four weeks" means now or a typical month. If that matters to the team, add a follow-up question about seasonal riding rather than complicating question 3.

Finally, move the Original and Fault columns to a separate tab or section, and keep the Rewrite and Reason columns as your final question set. That is what the team builds the survey from.

Check it worked

Your pretest column has at least one entry, every entry has a matching change and reason, and your final set reads as a clean list of questions with labelled options and a reason beside each one.

Next steps

You have a question set that measures what it claims to and a record of why each question changed. Use it to build and send the survey:

Need this done for you?

Padding does this work for growing companies, from the first diagnosis to a system that keeps working.

Digital Transformation

Get new tutorials

New tutorials, about twice a month.