SoTL 101

The first step is to see what "we" already know about your problem/issue/question:

It is possible that the problem you are having has been solved or the question answered. Google Scholar is your friend as are the reference librarians on your campus. Check with your teaching center - they often have resource libraries!

There may be teaching books on your topic of interest:

• Bain, Ken. (2004). What the Best College Teachers Do. Boston: Harvard University Press.
• Fink, L. Dee. (2013). Creating Significant Learning Experiences. Jossey-Bass.
• Nilson, Linda, B. (2010).Teaching at its Best. Jossey-Bass.

Other sources for SoTL research are:
• List of general SoTL journals
• Resources for SoTL in STEM

If you read the literature and don't find an answer, proceed to Step Two!

Once you have read the literature (and not found the answer to your question), you need to determine what research needs to be done to answer the question/solve the problem you are interested in. Then you need to be able to share that thinking process with potential readers.

In other words, outline/draft (whichever process works better for you to clarify your thinking) an argument that answers the questions: What do we know now? What do we need to know? Why does it matter?Scholars often outline the literature review at this point so they can move forward with the research process. Don't get bogged down in the writing process, but do be clear about argument/rationale for your hypotheses/research questions and methods!

ResourcesSources which define, describe and discuss the literature, finding sources, evaluating sources, organizing and synthesizing information
• University of Alaska Anchorage: Overview of literature reviews
• North Carolina State University: Twelve minute video with graphical explanations of literature reviews
• University of California - Davis: Video about finding examples of Literature Reviews
• University of Toronto: Overview and tips on evaluating articles
• Carnegie Mellon University: Overview of lit review - useful examples of integrated topic sentences
• Virginia Commonwealth University: Excellent overview and ways to organize information and synthesize the literature
• North Carolina Agriculture and Technical State University: Developing the review
• Sage Publications (resources for Fundamentals of Research in Criminology and Criminal Justice): Detailed information on finding sources and writing an integrated literature review
• University of Wisconsin-Madison: Basic steps in writing a literature review
• University of Connecticut: Defining, finding sources, and writing a lit review

This should 1) be based on the literature; 2) contain the specific variables you want to explore; and 3) be testable/measureable. In general a hypothesis will predict what you think you will find and is a stronger test of your ideas. A research question is posed when we don't have enough information to make an argument about what the results will likely be. For instance,

RQ: Does increasing the number of teacher immediacy behaviors increase student engagement?

H: The number of teacher immediacy behaviors will be significantly and positively correlated with student engagement.

Resources
• Cresswell, J.R. (2013). Research Design: Qualitative, Quantitative and Mixed Methods Approaches. 4th edition. Thousand Oaks, CA: Sage.
• Chapter 7: Research Questions and Hypotheses: http://www.sagepub.com/upm-data/22782_Chapter_7.pdf
• Cresswell examples in articles: http://www.sagepub.com/creswellstudy/07/pdfarticles.htm
• Trader, R. (2013). Handout for Quantitative Research Methods at McDaniel College – concise definition of RQ vs. H

The three questions below will give you some ideas of the trade-offs of various types of social science research methods. Choosing an answer to each question should help you narrow down appropriate possibilities for methods.

a. What kind of research do you want to do? Explore an issue or problem? Test an idea/hypothesis?

Exploring an issue/problem. If you wrote a research question because there is not enough information to really nail down alternatives/levels of variables, then qualitative/open ended methods that do not require you to define all the potential aspects/variables of the issue are likely to be more useful.

Ex: I’m wondering why students don’t do the reading in my class – asking open ended questions or having them log the “reasons” why they didn’t do the reading over a course of a couple of weeks is likely to yield more usable information than a set of responses based on what you “think” is happening.

Check out qualitative methods such as:

Focus groups
Open ended surveys
Structured or unstructured interviews
Participant observation
Observation
Case studies
Content analysis – thematic
Action research

Testing an idea/hypothesis. If you are testing a known solution/idea/hypothesis, then you can use qualitative methods or quantitative methods that will allow you to more precisely (numerically) represent the results and compare differences (i.e., between teaching methods; before and after)

Ex: Does lecture or cooperative learning yield deeper understanding of primary concepts?

Quantitative methods like:

Scales (tests) of understanding yielding “number correct”
Scales/surveys of perceived understanding
Experiments using above scales directly comparing the two methods

Qualitative methods such as:

Case studies of two classes (like an experiment but only one class in each “experimental” condition)
Direct observations
Interviews
Focus groups
Action research

b. What is more important? Understanding the depth and complexity of a situation? Generalizing your results as widely as possible?

Understanding the depth and complexity of the situation. Are you interested in really digging "deep" into this phenomenon to understand all the potential variables that may influencing the outcome?

Qualitative methods generally allow you to ask open ended questions and to use follow up questions/methods thus getting more depth and/or complexity.

Ex.: Why are particular activities engaging to students?

Qualitative methods are often more appropriate for “why” questions. Such as:

Focus groups
Structured or unstructured interviews
Case studies
Direct observations either in an artificial (laboratory) or naturalistic (classroom) environment
Content analysis (often a way to analyze focus group, interview or open ended survey data)

Generalizing the results as much as possible. Is it more important to be able to generalize the results beyond your sample?

Quantitative methods allow for easier data gathering and analysis which usually means a larger, more diverse sample, thus more ability to generalize beyond the sample.

Ex.: Which specific activities in an online class are related to students report of more engagement?

Surveys/scales
Experiments (often using a scale as the measurement of the key variables)
Content analysis of open ended survey questions

c. Which perspective matters most? Participants'/students' indirect perspective or a more direct perspective (grades, observation of behavior)? Here is where you make choices between issues such as measuring student learning outcomes directly or getting a perspective (i.e., students, colleagues) on the teaching/learning situation.

Participants' (i.e., students') perspective. Sometimes it’s more important to gain the view from participants than to try to gain a more direct view of outcomes.

Ex: Do students feel more confident about their learning when lecture is used or when whole class discussion is used?

Either quantitative or qualitative methods can be used to gain students’/participants’ perspectives

Case studies
Focus groups
Surveys/scale
Structured/unstructured interviews

Outside “objective” perspective. Sometimes a more direct view of learning outcomes or student or teacher behavior is more important than perceptions.

Ex: Do students retain more information when lecture is used or when whole class discussion is used?

Either qualitative or quantitative methods can be used to gain the outsiders’ perspective although people often “trust” the numbers more than qualitative interpretations of data

Scale/test/survey of learning
Content analysis of activity/assignment based on the learning
Experiments – direct comparison of two methods using scale/test/survey of learning or grade based on activity/assignment

Now that you have all of the data, you must analyze it in a way that answers your question and/or tests your hypotheses. Pay attention to the assumptions for statistical tests - a computer program will run whatever you tell it to whether the data is appropriate or not. Thus, the old adage, garbage in - garbage out! If you are using a method/statistic for the first time, reach out to others on your campus who have experience with that method. Asking experts specific questions, almost always elicits helpful answers!

Quantitative and Qualitative Analysis
Explanation of collecting and analyzing qualitative and quantitative data from University of Kansas

University of the West of England research overview including qualitative and quantitative data analysis

Qualitative Analysis
Ethnographic Action Research site talks about coding qualitative data

Statistical Analysis
Adding statistical analysis to your excel package with directions for running some basic stats in Excel

SISA allows you to do statistical analysis directly on the Internet.

Treasury Board of Canada - brief explanation of descriptive and inferential statistics

Quick overview of descriptive statistics with online calculator from the physics department at Saint John's University 

Center for Applied Linguistics chapter on data analysis and presentation: spreadsheets, categorical and nominal data

Graphpad: online statistical calculator for many stats tests 

HyperStat: Online statistical textbook

Presenting your data
Six free web-based tools for analysis and presentation of results by gigaom

1. Don't "overreach" your data: do not draw conclusions that go way beyond the type of data you gather.

2. Do not draw causal conclusions from correlational data.

An excellent powerpoint on the difference between results and discussion (drawing conclusions from results) sections(University of Vermont)

The importance of drawing sound conclusions regardless of your findings

Drawing conclusions in an iterative process with qualitative research methods

The debate about drawing causal conclusions based on qualitative data in ResearchGate

Conclusion drawing and verification of qualitative data from the NSF

Now, you have something to share with other teachers everywhere! You may want to skip forward to the "submit to a conference or publication" page to consider the appropriate outlet for your research. That will almost always give you some ideas on how to start since such outlets provide page minimums and maximums (no need to write a 15 page literature review if the entire article can only be 15 pages!), section names, etc.

General resources:

General guide to the parts of the paper from University of Southern California

Concise overview and explanation of the parts of the paper from Germana Community College

Interesting read from Joseph Pasek, University of Michigan who writes a social science paper which talks about how to write a social science paper!

Explanation of social science paper written as an APA paper from Illinois State University

Sample papers (while many disciplines use APA formatting for SoTL work, not all do- be sure to check the journal/conference requirements before you format):

Annotated APA Paper from WriteSource≠

Annotated APA Paper from APA

Annotated APA Paper from Purdue Owl

From Research Methods Knowledge Base

APA help:

Help from APA

Purdue Owl

Cornell University APA citations

Baker College APA Guides

Free citations formatting: bibme

You can look for disciplinary conferences or divisions of your disciplinary conferences. For instance, the National Communication Association has panels for its Instructional Development Division. The communication discipline also have interest groups such as the Communication Education for Central States Communication Association and Instructional Communication for Eastern States Communication Association.

Likewise, Psychology has the Society for the Teaching of Pscyhology at Midwestern Psychological Association Conference.

So, certainly consider your "normal" disciplinary venues.

There are also conferences devoted just to teaching or SoTL work such as:

The annual FACET Retreat, held in May each year, is an IU wide combination of "retreat" (inspiritational, self-discovery, fun) types of events with interactive workshops and presentations of research related to teaching

Midwest Scholarship of Teaching and Learning Conference at IU South Bend

International Society for the Scholarship of Teaching and Learning Conference (ISSOTL)

Illinois State's list of SoTL related conferences including their own

Lilly Conferences on College and University Teaching are held annually in Oxford, OH; Travis City, MI; Bethesda, MD; Newport Beach, CA; and Austin, TX: these conferences welcome a variety of submissions from posters and presentations of research to interactive workshops and roundtable discussions.

Potential Publications

Journal of Scholarship of Teaching and Learning (FACET, IU)

Journal of Teaching and Learning with Technology (FACET, IU)

Journal of Excellence in College Teaching (Miami University of Ohio)

Teaching and Learning Inquiry (ISSOTL)

Online Learning Journal (Online Learning Consortium - formerly Sloan)

Excellent list of SoTL publications and links to other lists from ISSOTL