Surveys and scales

Choosing/writing questions/items:

  • Will respondents understand the question?
  • Will they have the necessary information to answer it?
  • Will they be willing to answer it?
  • Does the question “lead” respondents toward a particular answer (influence them to show themselves in a good light or anticipate what they think researchers want to hear)?
  • Fewest number of items you can tolerate
  • If the item is not measuring a variable to analysis or description of sample, delete it

Instructions: clear, consider participants in choosing how they respond, pilot the survey

Order of questions: 

  • Influence: will some questions influence the answers to later questions?
  • Demographic questions – first (easy, gets participants involved but may be boring) or last (easy to finish when participants are tired)?

Types of questions: 

  • Closed (easier to analyze, does not allow for option you may not have considered; easier for participants) 
  • Open (have to be coded but allows for more depth; participants may get tired and skip or not write much)
  • Check questions: Questions asked in a different way multiple times 
  • Polarity rotation questions – questions worded negatively to be sure participants are paying attention

Scales (two most common) 

  • At least three scale items for any variable measured
  • scales allow you to use statistics that open ended questions may not
  • need to report reliabilities of scales: Cronbach Alpha

Likert

Within this specific course, how well do the following behaviors, thoughts, and feelings describe you? (1)
Staying up on the readings
1                        2                        3                        4                                    5            
Not characteristic of me at all                                                      Very characteristic of me

Semantic Differential

Describe the quality of communication in this conversation (2)
Relaxed             1            2            3            4            5            6            7            8            9        Strained

Resources:

General guide for survey research and design: 

http://writing.colostate.edu/guides/guide.cfm

Scale construction and reliability resources

http://www.personal.kent.edu/~dfresco/CRM_Readings/Clark_and_Watson_1995.pdf

http://www.joe.org/joe/1999april/tt3.php

http://www.statisticshell.com/docs/reliability.pdf

Levels of Measurement for scales

Writing survey items - closed versus open ended questions

Guide for writing survey items - University of Central Florida

Explanation of various types of survey questions

Comparing open and closed ended survey questions - overview

Indepth explanation of open versus closed questions

*Source: Reinard, J.C. (2008). Introduction to Communication Research, 4th Edition. New York: McGraw-Hill.

(1) Dixson, M.D. (2010). Creating effective student engagement in online courses: What do students find engaging? Journal of Scholarship of Teaching and Learning, 10(2), pp. 1-13.

(2) Duck, S., Rutt, D.J., Hurst, M.H., & Strejc, H. (1991). Some evident truths about conversations in everyday relationships: All communications are not created equal.Human Communication Research, 18, 228-267.

Dixson, Mack @ FACET, 2014