MKTG 344 (Sahni) Survey Guide

Getting the right population can be key to obtaining useful data. In general, if you target too broad or too narrow a population, you could end up with data that doesn’t answer your questions and steers you in making the wrong decisions.

If you are aiming for a population more specific than a “general population” sample, it is important to identify your target population early on in the research process. You may already know the consumer profile you are aiming for, but if you aren’t yet sure, thinking it through now is key, especially while you are still hammering out your survey.

Your survey will be launched on a platform called Prolific. The service provides over 100 built-in “qualifications” - questions that participants have already answered and are saved on the platform. You can use the answers to those questions to target your population.

“Gen Pop”

If you do not need ANY qualifications to target a specific population, and instead are looking for responses from a “general population” of people online, then a “Gen Pop” sample is all you need. If this is the case, please type in “Gen Pop” in line 1 of the list of qualifications on the course survey spreadsheet.

Choose Up To 5 Qualifications

You are allowed to choose up to 5 qualification screeners from the qualifications document. (Be sure to download and view in PDF format.) The list contains over 100 possible qualifications to choose from. Keep in mind that the more qualifications you choose, and the more obscure they are, the smaller the potential pool of qualified participants there could be.

If there are no questions in the qualifications document that will allow you to target your intended population, see the Eligibility Survey section below.

Record Your Qualifications

Enter the qualifications you want into the course survey spreadsheet. Under the “Title of Qualifiers” column, list ONLY the TITLE of the qualification as it shows in the table of contents of the qualification document from above. Follow the example in the “EXAMPLE” tab.

In this example, the target is only people who identify as male, and are left-handed. 

• On page 1 of the list of qualifications I see the qualifier “Handedness.” People were asked “Are you left or right-handed?” I want my survey filled by those who responded “Left-handed, ambidextrous.”
• Similarly, on the same page I see the qualifier “Gender”; people were asked about their gender identity.
• So the spreadsheet specifies the title of the qualifiers and the acceptable responses.

Adding Qualification Questions to Your Survey

Selecting a qualification question in Prolific does not mean you will be able to see what a specific Prolific user responded to that question. For example, if you select that you only want participants who responded ‘Left Handed’, or ‘Ambidextrous’, you will not be able to access a list of what users answered for their handedness in your data file. The screening questions are only used to determine eligibility and pipe qualifying participants into your survey.

If you want to know the specific answers to the qualifying questions you are using on Prolific, you will need to include that question in your survey. If you do so, it MUST be the exact-worded question and all of the possible answers.

However, you are not required to include the qualification questions in your survey, but only if you want to see those answers for yourself. 

For example, if you don’t care whether or not someone is ‘left handed’ or ‘ambidextrous,’ then you can leave that question out of your survey. However, if it is important to you that you know who is one vs the other, then you will want to put that question into your survey.

Again: if you include a Prolific qualification question in your survey, the wording of the question and all of the answers must be exactly the same as how they are written in the qualifications document. (download this and view as PDF)

Eligibility Surveys

If you need very specific criteria for participants that you cannot find in the qualifications document, you will need to create 2 surveys in order to reach the right population.

Eligibility Survey: A 1 min survey with your eligibility questions.

• For example, knowing someone's university major. This kind of question does not exist in the extended set of available Prolific pre-screened questions. So, you will have an initial 1 minute survey asking that.

Final Survey: The actual final survey.

Based on the responses to the Eligibility Survey, eligible individuals will be sent the Final Survey later. If you take this option you will need to explicitly tell the Behavioral Lab that you are requesting this. Email the lab staff member that is supporting your group by finding their email address on the course spreadsheet next to your group. This process will take longer, and will lower the number of participants you will be able to recruit for your final survey. This is because the Eligibility Survey will have a cost associated with it, and will decrease the amount of money you have remaining for your Final Survey. Just bear this in mind when deciding if you want to run an Eligibility Survey. We recommend that you first look at the qualifications document thoroughly, and try to find a combination of a pre-existing question or two that could closely approximate the target population before requesting an Eligibility Survey.

Cost of your survey & the number of participants

The platform the Behavioral Lab will use to run your survey - Prolific - charges 22 cents per minute of survey time per participant. (This price includes the platform service fee.) So, you will need to accurately assess the length of time it takes to complete your survey and include that information on the survey spreadsheet. This will help you calculate roughly how many participants you are able to get given the budget you have to run your survey.

For example, if your survey is six minutes long, that is: 6 minutes x 22 cents = $1.32 per participant. If you have a $400 budget limit for this survey, you should expect to get: $400 / $1.32 = 303 participants. 

Be aware, however, your estimated length of your survey may not match what the lab determines the length of your survey to be. The lab staff will test the length of your survey prior to running a soft launch (see section 5 on Soft Launches). They may test it internally as shorter or longer than your estimate. There is no arguing with the lab support - they are the experts. They will post the soft launch of your survey for that estimated length. 

A soft launch is an inital launch to 30 participants. This is both to gauge whether or not your survey is running properly as well as a more accurate length of the survey. Once the soft launch is complete, the system calculates an average survey length taken by those 30 participants. This is the time that the lab staff will price out your final launch. Again, it may be more or less per participant than you expect, however there is no way around this.