How to design a valid research survey

Target audience: Nonprofits, cause organizations, NGOs, foundations, businesses

Is your organization planning to conduct a survey of your membership or a key constituency? Make sure you do it right.

Nora G. Barnes, Ph.D., professor of marketing at the University of Massachusetts and director of its Center for Marketing Research, offered a presentation today at the Society for New Communications Research about how to properly craft a valid research survey.

Nora has designed scores of major surveys, most recently the First Longitudinal Study of Social Media Usage by the Largest US Charities and The Fortune 500 and Blogging: Slow and Steady.

Your survey can be conducted by mail questionnaire, personal interview, “mall intercept,” telephone interview or electronic (online). You should stick to one chief methodology (say, telephone) and then supplement it with another if you wish (say, offer to send the questions by email). Mail questionnaires tend to have a low response rate (less than 5 percent) while she has been getting a 20 to 25 percent response rate using online polling. Providing incentives, such as the chance to win an iPod or Wii, often increases the response rate and thus increases the survey’s validity.

Tips for composing survey questions

• Keep it clear and concise.
• No double-barreled questions (keep each question focused on one thing
• No overlapping categories
• No vague or ambiguus questions
• Use time frames and context.
• Scales are beest for quick and easy answering. The best scales are 4 to 6 point scales, not 1-10.
• Don’t use middle categories on scales (people tend to jump into the middle).
• Keep yes-no questions to a minimum. (You’re better off asking them how much they use them rather than do you use them.)
• Don’t use loaded questions.
• Allow for other or don’t know.
• Give people the chance to enter: Other (please specify).
• Always collect some demographic information for classification purposes.
• You can ask one open-ended question.
• Don’t ask more than 20-25 questions.

Ultimately, she said, all studies are judged on two things: the sampled audience and the questions you ask.

Nora left us with two age-old verities about designing good surveys:

• The questions you ask will determine the responses you get

• Vague and ambiguous questions will get you vague and ambiguous answers.JD Lasica, founder and former editor of Socialbrite, is co-founder of Cruiseable. Contact JD or follow him on Twitter or Google Plus.