You need to test your survey before it is launched. Qualtrics system can also generate dummy data for you. (See the “Generate dummy data” videos.)
Use the Preview button in Qualtrics to test your survey yourself multiple times, and test each of the branches (if any) your survey can take.
Some of the things to test for:
- Spelling & grammar
- Clarity of questions & wording
- Ask people who test the survey for you about whether or not they understand the meaning of the questions. Sometimes we think we’re being clear, but to others it is not so clear.
- Length of time to completion
- Your survey may be much longer than you think! Have different people take it multiple times, and ask them to time themselves while taking it. You will need to provide this timing average to on this spreadsheet before it can be launched.
- Make sure you limit the survey to 10 minutes or less, which usually implies having less than thirty questions.
- BE CAREFUL: you may have different conditions or versions of the survey that may take different lengths of time. That is, full data collection for one person who is “left handed” may take 8-9 minutes, while the “ambidextrous” person is assigned fewer questions and takes only 3-4 minutes. Be mindful of that.
- The Behavioral Lab will pay the participants based on the longer condition of the study, so we will pay everyone based on the 8-9 minute length, regardless if they only took 3-4 minutes.
- The data file output
- Once people take the survey or you have Qualtrics generate dummy data, download and look at the data file (Excel).
- Does it contain all the information you expect it should?
- Is the data file large and cumbersome?
- What changes do you need to make to the survey to change how the data show on the data file?
- The data itself
- Do the answers to your questions make sense? This goes to the 2nd bullet point above - are the questions themselves understandable to the participants?
- Are your questions in the optimal order? Perhaps you need to ask one set of questions before or after another instead.
