Writing great questions is an art that like all arts requires a great amount of work, practice, and help from others. The following discussion is one that identifies some of the common pitfalls in creating a great questionnaire.
- Avoid loaded or leading words or questions.
Slight wording changes can produce great differences in results.
Could, Should, Might all sound almost the same, but may produce a
20% difference in agreement to a question (The supreme court could..
should.. might.. have forced the breakup of Microsoft
Corporation). Strong words that represent control or action,
such as prohibit produces similar results (Do you believe that congress
should prohibit insurance companies from raising rates?)
Sometimes wording is just biased: You wouldn't want to go to
Rudolpho's Restaurant for the company's annual party would you?
- Misplaced questions. Questions placed out of
order or out of context should be avoided. In general, a funnel
approach is advised. Broad and general questions at the beginning
of the questionnaire as a warm-up. Then more specific
questions, followed by more general easy to answer questions like
demographics.
- Mutually non-exclusive response categories.
Multiple choice response categories should be mutually exclusive so that
clear choices can be made. Non-exclusive answers frustrate the
respondent and make interpretation difficult at best.
- Nonspecific questions. Do you like orange
juice? This is very unclear...do I like what? Taste,
texture, nutritional content, Vitamin C, the current price,
concentrate, fresh squeezed? Be specific in what you want to know
about. Do you watch TV regularly? (what is
regularly?).
- Confusing or unfamiliar words. Asking about
caloric content, bits, bytes, mbs, and other industry specific jargon
and acronyms are confusing. Make sure your audience understands
your language level, terminology and above all, what you
are asking.
- Non-directed questions give respondents
excessive latitude.
What suggestions do you have for improving tomato juice? The question is about taste, but the respondent may offer suggestions about texture, the type of can or bottle, mixing juices, or something related to use as a mixer or in recipes.
- Forcing answers. Respondents may not want, or
may not be able to provide the information requested. Privacy is
an important issue to most people. Questions about income,
occupation, finances, family life, personal hygiene and beliefs
(personal, political, religious) can be too intrusive and rejected by
the respondent.
- Non-exhaustive listings. Do you have all of
the options covered? If you are unsure, conduct a pretest
using the "Other (please specify) __________" option.
Then revise the question making sure that you cover at least 90%
of the respondent answers.
- Unbalanced listings. Unbalanced scales may be
appropriate for some situations and biased in others. When
measuring alcohol consumption patterns, One study used a quantity
scale that made the heavy drinker appear in the middle of the scale with
the polar ends reflecting no consumption and an impossible amount to
consume. However, we expect all hospitals to offer good care and
may use a scale of excellent, very good, good, fair. We do not
expect poor care.
- Double barreled questions. What is the fastest and most convenient Internet service for you? The fastest is certainly not the most economical. Two questions should be asked.
- Dichotomous questions. Make sure answers are independent. For example the question "Do you think of basketball players as being independent agents or as employees of their team?" Some believe that yes, they are both.
- Long questions . Multiple choice questions are the longest and most complex. Free text answers are the shortest and easiest to answer. When you Increase the length of questions and surveys, you decrease the chance of receiving a completed response.
- Questions on future intentions. Yogi Berra once said that making predictions is difficult, especially when they are about the future. Predictions are rarely accurate more than a few weeks or in some case months ahead.
Print
Feedback
