Choosing/writing questions/items:
- Will respondents understand the question?
- Will they have the necessary information to answer it?
- Will they be willing to answer it?
- Does the question “lead” respondents toward a particular answer (influence them to show themselves in a good light or anticipate what they think researchers want to hear)?
- Fewest number of items you can tolerate
- If the item is not measuring a variable to analysis or description of sample, delete it
Instructions: clear, consider participants in choosing how they respond, pilot the survey
Order of questions:
- Influence: will some questions influence the answers to later questions?
- Demographic questions – first (easy, gets participants involved but may be boring) or last (easy to finish when participants are tired)?
Types of questions:
- Closed (easier to analyze, does not allow for option you may not have considered; easier for participants)
- Open (have to be coded but allows for more depth; participants may get tired and skip or not write much)
- Check questions: Questions asked in a different way multiple times
- Polarity rotation questions – questions worded negatively to be sure participants are paying attention
Scales (two most common)
- At least three scale items for any variable measured
- scales allow you to use statistics that open ended questions may not
- need to report reliabilities of scales: Cronbach Alpha
Likert
Within this specific course, how well do the following behaviors, thoughts, and feelings describe you? (1)
Staying up on the readings
1 2 3 4 5
Not characteristic of me at all Very characteristic of me
Semantic Differential
Describe the quality of communication in this conversation (2)
Relaxed 1 2 3 4 5 6 7 8 9 Strained
Resources:
General guide for survey research and design:
http://writing.colostate.edu/guides/guide.cfm
Scale construction and reliability resources
http://www.personal.kent.edu/~dfresco/CRM_Readings/Clark_and_Watson_1995.pdf
http://www.joe.org/joe/1999april/tt3.php
http://www.statisticshell.com/docs/reliability.pdf
Levels of Measurement for scales
Writing survey items - closed versus open ended questions
Web Center for Social Research Methods
Explanation of various types of survey questions
Comparing open and closed ended survey questions - overview
Cornell Unviersity Coopoerative Extension explanation of open versus closed questions
*Source: Reinard, J.C. (2008). Introduction to Communication Research, 4th Edition. New York: McGraw-Hill.
(1) Dixson, M.D. (2010). Creating effective student engagement in online courses: What do students find engaging? Journal of Scholarship of Teaching and Learning, 10(2), pp. 1-13.
(2) Duck, S., Rutt, D.J., Hurst, M.H., & Strejc, H. (1991). Some evident truths about conversations in everyday relationships: All communications are not created equal.Human Communication Research, 18, 228-267.