Experiments

The “gold standard” of proving cause and effect (Beans in window vs closet)
2 or more treatments
Random assignment to treatment
All else kept the same
Cause and effect established
Often very artificial
Very difficult to do in social science in real world contexts

Quasi-experimental

A weaker test of cause and effect (Teaching strategy A vs B)
2 or more treatments
Random assignment
- Students choose which class sections to take. Not random choices.
All else the same
- Other aspects of class might vary. If two professors, they vary in other ways.

Strategies for Quasi-experimental

Pre-test and Post-test (so same students; look for change. But other things intervening too)
This semester and last semester (diff students; other things may have changed.)
Two sections this semester with different methods (Diff students, possibly dynamics)
Your class vs that of another instructor (many things vary between the two of you)
ABAB design in one section (Unit 1 uses A; Unit 2 uses B; Unit 3 uses A….)

Triangulation of methods:
Do several different ways and see if results are the same every time.
E.g. Two instructors in same semester. Each do A in one section and B in the other.
E.g. Pre- and post-test in two different sections this semester, with A in one section and B in the other.

Triangulate by using several different types of data:

Different sources (students, teachers, parents); different methods (e.g. observations, grades); and/or different times (before and after)

Existing data

(Easiest to gather)

Records

Student grades*
Attendance
Teaching evaluations
Retention rate
% of students in school-sponsored organizations
Standardized test results
Demographics

Archival data

Research literature*
Local policies, rules, and regulations about education
Syllabi and curriculum
Accreditation reports

Conventional sources

(Require selection or development of data-collection instruments. Still relatively easy to acquire.)

Behavioral data (best)

Teacher journal, or field notes*
Number of books read
Library use
Writing samples
Grades on classroom tests*
Variety of materials used
Teacher observations of student participation, interaction, etc.*
Outside observations of classroom$
Student journals
Minutes from meetings
Staff development (hours, types)
Audio or videotapes of classroom$

Perceptual data (attitudes, opinions)

Surveys
Simple interviews

Inventive sources

(Products or performances. May be difficult to acquire and can be difficult to evaluate)

Exhibits$
Portfolios$
Expositions$
Videotapes$

* = excellent source of data
$ = may be expensive in time or money to do well
(adapted from Calhoun, 1994)

Resources

Materials from a workshop on quasi-experimental design
http://www.ipr.northwestern.edu/workshops/past-workshops/quasi-experimental-design-and-analysis-in-education/2012/

Good overview of quasi-experimental design
http://en.wikipedia.org/wiki/Quasi-experiment

A more detailed explanation
http://www.csulb.edu/~msaintg/ppa696/696quasi.htm

Mettetal, Mack @ FACET, 2014

Defining operationalization of variables

Experiments

Mettetal, Mack @ FACET, 2014

Faculty Academy on Excellence in Teaching resources and social media channels