Welcome to our new series of articles called “Research Best Practices.” These articles will cover everything a beginner survey maker needs to know about survey-making, such as tips for building surveys, guides for how to make them engaging, and advice for running agile market research with a platform like Upsiide.
We’re kickstarting our series with a topic that even skilled researchers can have trouble with – writing effective survey questions. So we came up with 6 essential tips and tricks for creating great questions that every survey-maker has to know.
1. Avoid leading questions
Try to keep away from question-wording and answer scales that give more “real estate” to specific answers, because this will bias your results. Here are some ways where you might trip up on leading questions.
Category usage.
Can you spot the problem with this question?
How many times a week do you drink coffee?
While this question might be okay for coffee drinkers, people who don’t have coffee regularly or don’t drink it at all won’t be able to answer it. So, don’t assume category usage unless already established. So here’s how you can fix the above question:
How many times a week (if any) do you drink coffee?
When asking frequency questions like this, include “if any” so that all possible responses are captured.
Option scales.
Another mistake is writing answer options or rating scales that aren’t balanced well. Here’s an example:
How satisfied are you with your current phone?
Extremely satisfied
Very satisfied
Satisfied
Not satisfied
Not at all satisfied
Can you see that there are more positive/satisfactory answer options than negative/dissatisfactory ones? With these answers, you might unintentionally make respondents choose certain answers, which could make your survey results unreliable.
That’s why you’ve got to check that your option scales are balanced. The answer options should include a full range of answer choices and do not have more positive than negative responses (and vice versa).
To bring objective eyes to your research, show your surveys to other people – they can help you find where your question might lead to a specific outcome.
2. Avoid “Double-Barreled” questions
A double-barreled question is a type of question that asks two things in one sentence. For example:
Would you purchase and eat this product?
As a respondent, you might think, “well, I’d purchase it for my kids, but I wouldn’t eat it myself,” or “I’d love to eat this product, but I wouldn’t spend money on it.” To avoid confusion, make sure all of your questions ask only ONE thing. The easiest way to ensure you’re doing this is to break the question into two:
Is this something you would purchase?
Is this something you would eat?
Psst! If you want to get some more tips for writing survey questions for concept testing, this article is for you.
3. Ensure question lists are exhaustive and don’t overlap
We see this issue all the time:
Which of the following beverages do you drink every day?
Tea
Coffee
Wine
Beer
Soft drinks
Well, what if I don’t drink any of these options every day? And what about, ahem, water? This is why you need answers like “none” or “never” or “other (specify).” Sometimes “I don’t know/I don’t recall” could also be relevant.
Another thing to remember is that you should ensure answer options don’t overlap. This is especially important when defining time frames, for example:
When was the last time you purchased espresso coffee to make at home?
Last week
Longer ago than last week, but within the past month
Longer ago than a month, but within the past 6 months
Longer than 6 months ago
Never
4. Make questions detailed and easy to answer
Nobody likes questions that make you overthink. That’s why your questions need to provide details or time frames. For example, a question like…
How much did you spend on cooking products for your kitchen last year?
$_______
…is far too broad for consumers to answer meaningfully. What are the “cooking products”? Are they indoor only? Does it count food/ingredients? Are appliances “cooking products”?
Try to be specific in your descriptions. A better question would be:
How much did you spend on each of the following types of cooking products last year, if anything? Please enter your best estimate, below.
Small appliances (e.g. blenders, toasters, etc.) $_______
Cookware (e.g. pots, pans, frying pans, etc.) $_______
Bakeware (e.g. cookie sheets, muffin tins, etc.) $_______
Outdoor Grilling (e.g. barbecues, rotisseries, etc.) $_______
Utensils & Gadgets (e.g. spatulas, whisks, etc.) $_______
Knives (e.g. knives, knife sets, blocks, etc.) $_______
Etc.
Total $_______
5. Sound like a human
Have you ever seen survey questions that sound like a bot wrote them? They usually use complex sentences or industry jargon that nobody understands. These types of survey questions make respondents confused and tired.
Avoid industry jargon and keep it conversational, staying away from unknown research terms:
Instead of “do you intend to purchase?” try “would you buy this?”
Instead of “shelf-stable snack foods,” try “packaged snacks.”
6. Go through your survey from a respondent’s point of view
The BEST way to improve your survey is to put yourself in the respondent’s shoes and take the survey yourself. Go back to the questionnaire and pretend like you’re a different person, e.g. your perfect customer or, vice versa, someone who never heard about your product.
Upsiide makes it easy to preview your study, either from the screener, the beginning of the survey, or any specific question. Of course, none of the responses are collected when you’re in preview mode, but it will give you a great sense of flow, clarity, and engagement.
If you’ve got a bit of extra time, consider watching someone else take the survey and note any points of confusion they encounter. Upsiide makes this easy to do with the link-sharing option.
Don’t forget to check out our Support Centre for more tips and tricks about conducting online research with Upsiide.
