Randomize order using question block rotation

Randomizing the order of a block of questions:

You can do this using question block rotation. This allows you to take groups of questions and randomly present those blocks of questions to your respondents. This can be used to help eliminate possible bias from your respondent group (bias related to the order they viewed question sets).

How can I set up question block rotation?

Consider the following scenario:

Q1 - Common Question 1
Q2 - Common Question 2

Q3 - Block 1 Question 1
Q4 - Block 1 Question 2
Q5 - Block 1 Question 3

Q6 - Block 2 Question 1
Q7 - Block 2 Question 2
Q8 - Block 2 Question 3
Q9 - Block 2 Question 4

Q10 - Block 3 Question 1
Q11 - Block 3 Question 2

Q12 - Common Question 3
Q13 - Common Question 4

Example: Questions Q1 and Q2 are common questions to be displayed in order. Then there are 3 blocks of questions to be randomized and displayed to the respondents. After respondents answer all three blocks of questions, they continue with Q12. Steps for setting this up:
  • Click on Login >> Surveys >> Edit Survey
  • Set up all the questions in the survey.
  • Identify the different blocks and add the Randomization Logic question just before the first question of the first block. In this case, immediately before Q3 or just after Q2.
  • Click the Add Question button to add the Randomization Logic question from the Advanced Question Types section.
  • Survey Software Help Image
  • Select Randomize The Order of the list of questions (Question Block Rotation) from the Randomization Type drop-down menu.
  • Include any Instructions desired for others who have administrative access to the survey. This text will not be seen by respondents, but can be used to help communicate with other administrators of the survey.
  • Select the Block Termination Question to indicate where the randomization logic will stop. This is the question that all respondents will see after all of the question blocks are displayed. In this example, select Q12.
  • Include any Instructions desired for others who have administrative access to the survey. This text will not be seen by respondents, but can be used to help communicate with other administrators of the survey.
  • Enter the Filler Text. This is the text that will appear on the page before the blocks of questions appear. This text will be seen by respondents.
  • In the questions list, check the box next to the first question of each question block. In this case, the questions Q3, Q6, and Q10 would be checked.
  • Click Save Question.
Important: To complete the randomization process, set up default branching for each of the last questions in the blocks to the block termination question. For this example, Q5, Q9, and Q11 would each be set up to branch to Q12.

I am getting the error message Block randomization only supports unto 40 questions, please reduce the number of questions and try again.

This error is shown when you enter more than 40 questions for block randomization. You may reduce number of questions.

Block randomization supports only 40 questions

Instead of randomizing blocks, I wish to randomize the order of all questions in a given list. But, I only see block rotation and randomly display 1 in the Randomization Type drop down. Can I do this?

Yes, you can. For doing this, you would have to treat each question as one block. Thus, you would have to:

  • Set up the survey, and add Randomization Logic question just before the first question of the block..
  • Select the Question Block Rotation option.
  • Select Randomize The Order of the list of questions (Question Block Rotation) from the Randomization Type drop-down menu.
  • From the questions list, select every question that should be randomized. (This should be after the randomization logic question and before the block termination question).
  • After adding the question, set up branching on each question in the list to go to the block termination question.
  • Rest of the steps will remain the same.

Can I control the order of randomization?

No, QuestionPro uses a complex algorithm that displays the blocks in a random order. Thus, we cannot control which questions are displayed. We also do not keep track of questions that are displayed.

Can I set up randomization within randomization?

No. Currently, randomizing the questions within a block which is already randomized is not possible. You can have multiple randomization logic questions in the survey, however, they shouldn’t overlap.

Block Randomization will not work if Interactive mode is on
License

This feature/tools described here are available with the following license(s) :

Corporate Edition and above- Buy Now - $75/Month