SoTL 101

So, you are interested in SoTL research or maybe you just have a teaching/learning question you'd like to answer? How do you start? Welcome to SoTL 101. This webpage is designed for those who have never done SoTL or social science research. So, if it seems too basic, feel free to skip to the parts you need!

Dr. Marcia Dixson (FACET Class of 2006)
Assistant Vice-Chancellor for Teaching and Learning, PFW

Scholarship of Teaching and Learning (SoTL) is: “problem posing about an issue of teaching or learning, study of the problem through methods appropriate to the disciplinary epistemologies, applications of results to practice, communication of results, self-reflection, and peer review” (Cambridge, 2010, p. 8) *

The first step is to see what "we" already know about your problem/issue/question:

It is possible that the problem you are having has been solved or the question answered. Google Scholar is your friend as are the reference librarians on your campus. Check with your teaching center - they often have resource libraries!

There may be teaching books on your topic of interest:

• Bain, Ken. (2004). What the Best College Teachers Do. Boston: Harvard University Press.
• Fink, L. Dee. (2013). Creating Significant Learning Experiences. Jossey-Bass.
• Nilson, Linda, B. (2010).Teaching at its Best. Jossey-Bass.

Other sources for SoTL research are:
List of general SoTL journals
Resources for SoTL in STEM

If you read the literature and don't find an answer, proceed to Step Two!

Once you have read the literature (and not found the answer to your question), you need to determine what research needs to be done to answer the question/solve the problem you are interested in. Then you need to be able to share that thinking process with potential readers.

In other words, outline/draft (whichever process works better for you to clarify your thinking) an argument that answers the questions: What do we know now? What do we need to know? Why does it matter?Scholars often outline the literature review at this point so they can move forward with the research process. Don't get bogged down in the writing process, but do be clear about argument/rationale for your hypotheses/research questions and methods!

ResourcesSources which define, describe and discuss the literature, finding sources, evaluating sources, organizing and synthesizing information
University of Alaska Anchorage: Overview of literature reviews
North Carolina State University: Twelve minute video with graphical explanations of literature reviews
University of California - Davis: Video about finding examples of Literature Reviews
University of Toronto: Overview and tips on evaluating articles
Carnegie Mellon University: Overview of lit review - useful examples of integrated topic sentences
Virginia Commonwealth University: Excellent overview and ways to organize information and synthesize the literature
North Carolina Agriculture and Technical State University: Developing the review
Sage Publications (resources for Fundamentals of Research in Criminology and Criminal Justice): Detailed information on finding sources and writing an integrated literature review
University of Wisconsin-Madison: Basic steps in writing a literature review
University of Connecticut: Defining, finding sources, and writing a lit review

This should 1) be based on the literature; 2) contain the specific variables you want to explore; and 3) be testable/measureable. In general a hypothesis will predict what you think you will find and is a stronger test of your ideas. A research question is posed when we don't have enough information to make an argument about what the results will likely be. For instance,

RQ: Does increasing the number of teacher immediacy behaviors increase student engagement?

H: The number of teacher immediacy behaviors will be significantly and positively correlated with student engagement.

Resources
• Cresswell, J.R. (2013). Research Design: Qualitative, Quantitative and Mixed Methods Approaches. 4th edition. Thousand Oaks, CA: Sage.
Chapter 7: Research Questions and Hypotheses: http://www.sagepub.com/upm-data/22782_Chapter_7.pdf
Cresswell examples in articles: http://www.sagepub.com/creswellstudy/07/pdfarticles.htm
• Trader, R. (2013). Handout for Quantitative Research Methods at McDaniel College – concise definition of RQ vs. H

The three questions below will give you some ideas of the trade-offs of various types of social science research methods. Choosing an answer to each question should help you narrow down appropriate possibilities for methods.

a. What kind of research do you want to do? Explore an issue or problem? Test an idea/hypothesis?

Exploring an issue/problem. If you wrote a research question because there is not enough information to really nail down alternatives/levels of variables, then qualitative/open ended methods that do not require you to define all the potential aspects/variables of the issue are likely to be more useful.

Ex: I’m wondering why students don’t do the reading in my class – asking open ended questions or having them log the “reasons” why they didn’t do the reading over a course of a couple of weeks is likely to yield more usable information than a set of responses based on what you “think” is happening.

Check out qualitative methods such as:

Focus groups
Open ended surveys
Structured or unstructured interviews
Participant observation
Observation
Case studies
Content analysis – thematic
Action research

Testing an idea/hypothesis. If you are testing a known solution/idea/hypothesis, then you can use qualitative methods or quantitative methods that will allow you to more precisely (numerically) represent the results and compare differences (i.e., between teaching methods; before and after)

Ex: Does lecture or cooperative learning yield deeper understanding of primary concepts?

Quantitative methods like:

Scales (tests) of understanding yielding “number correct”
Scales/surveys of perceived understanding
Experiments using above scales directly comparing the two methods

Qualitative methods such as:

Case studies of two classes (like an experiment but only one class in each “experimental” condition)
Direct observations
Interviews
Focus groups
Action research

b. What is more important? Understanding the depth and complexity of a situation? Generalizing your results as widely as possible?

Understanding the depth and complexity of the situation. Are you interested in really digging "deep" into this phenomenon to understand all the potential variables that may influencing the outcome?

Qualitative methods generally allow you to ask open ended questions and to use follow up questions/methods thus getting more depth and/or complexity.

Ex.: Why are particular activities engaging to students?

Qualitative methods are often more appropriate for “why” questions. Such as:

Focus groups
Structured or unstructured interviews
Case studies
Direct observations either in an artificial (laboratory) or naturalistic (classroom) environment
Content analysis (often a way to analyze focus group, interview or open ended survey data)

Generalizing the results as much as possible. Is it more important to be able to generalize the results beyond your sample?

Quantitative methods allow for easier data gathering and analysis which usually means a larger, more diverse sample, thus more ability to generalize beyond the sample.

Ex.: Which specific activities in an online class are related to students report of more engagement?

Surveys/scales
Experiments (often using a scale as the measurement of the key variables)
Content analysis of open ended survey questions

c. Which perspective matters most? Participants'/students' indirect perspective or a more direct perspective (grades, observation of behavior)? Here is where you make choices between issues such as measuring student learning outcomes directly or getting a perspective (i.e., students, colleagues) on the teaching/learning situation.

Participants' (i.e., students') perspective. Sometimes it’s more important to gain the view from participants than to try to gain a more direct view of outcomes.

Ex: Do students feel more confident about their learning when lecture is used or when whole class discussion is used?

Either quantitative or qualitative methods can be used to gain students’/participants’ perspectives

Case studies
Focus groups
Surveys/scale
Structured/unstructured interviews

Outside “objective” perspective. Sometimes a more direct view of learning outcomes or student or teacher behavior is more important than perceptions.

Ex: Do students retain more information when lecture is used or when whole class discussion is used?

Either qualitative or quantitative methods can be used to gain the outsiders’ perspective although people often “trust” the numbers more than qualitative interpretations of data

Scale/test/survey of learning
Content analysis of activity/assignment based on the learning
Experiments – direct comparison of two methods using scale/test/survey of learning or grade based on activity/assignment

Now that you have all of the data, you must analyze it in a way that answers your question and/or tests your hypotheses. Pay attention to the assumptions for statistical tests - a computer program will run whatever you tell it to whether the data is appropriate or not. Thus, the old adage, garbage in - garbage out! If you are using a method/statistic for the first time, reach out to others on your campus who have experience with that method. Asking experts specific questions, almost always elicits helpful answers!

Quantitative and Qualitative Analysis
Explanation of collecting and analyzing qualitative and quantitative data from University of Kansas

University of the West of England research overview including qualitative and quantitative data analysis

Qualitative Analysis
Ethnographic Action Research site talks about coding qualitative data

Statistical Analysis
Adding statistical analysis to your excel package with directions for running some basic stats in Excel

SISA allows you to do statistical analysis directly on the Internet.

Treasury Board of Canada - brief explanation of descriptive and inferential statistics

Quick overview of descriptive statistics with online calculator from the physics department at Saint John's University 

Center for Applied Linguistics chapter on data analysis and presentation: spreadsheets, categorical and nominal data

Graphpad: online statistical calculator for many stats tests 

HyperStat: Online statistical textbook

Presenting your data
Six free web-based tools for analysis and presentation of results by gigaom

Now, you have something to share with other teachers everywhere! You may want to skip forward to the "submit to a conference or publication" page to consider the appropriate outlet for your research. That will almost always give you some ideas on how to start since such outlets provide page minimums and maximums (no need to write a 15 page literature review if the entire article can only be 15 pages!), section names, etc.

General resources:

General guide to the parts of the paper from University of Southern California

Concise overview and explanation of the parts of the paper from Germana Community College

Interesting read from Joseph Pasek, University of Michigan who writes a social science paper which talks about how to write a social science paper!

Explanation of social science paper written as an APA paper from Illinois State University

Sample papers (while many disciplines use APA formatting for SoTL work, not all do- be sure to check the journal/conference requirements before you format):

Annotated APA Paper from WriteSource≠

Annotated APA Paper from APA

Annotated APA Paper from Purdue Owl

From Research Methods Knowledge Base

APA help:

Help from APA

Purdue Owl

Cornell University APA citations

Baker College APA Guides

Free citations formatting: bibme

You can look for disciplinary conferences or divisions of your disciplinary conferences. For instance, the National Communication Association has panels for its Instructional Development Division. The communication discipline also have interest groups such as the Communication Education for Central States Communication Association and Instructional Communication for Eastern States Communication Association.

Likewise, Psychology has the Society for the Teaching of Pscyhology at Midwestern Psychological Association Conference.

So, certainly consider your "normal" disciplinary venues.

There are also conferences devoted just to teaching or SoTL work such as:

The annual FACET Retreat, held in May each year, is an IU wide combination of "retreat" (inspiritational, self-discovery, fun) types of events with interactive workshops and presentations of research related to teaching

Midwest Scholarship of Teaching and Learning Conference at IU South Bend

International Society for the Scholarship of Teaching and Learning Conference (ISSOTL)

Illinois State's list of SoTL related conferences including their own

Lilly Conferences on College and University Teaching are held annually in Oxford, OH; Travis City, MI; Bethesda, MD; Newport Beach, CA; and Austin, TX: these conferences welcome a variety of submissions from posters and presentations of research to interactive workshops and roundtable discussions.

Potential Publications

Journal of Scholarship of Teaching and Learning (FACET, IU)

Journal of Teaching and Learning with Technology (FACET, IU)

Journal of Excellence in College Teaching (Miami University of Ohio)

Teaching and Learning Inquiry (ISSOTL)

Online Learning Journal (Online Learning Consortium - formerly Sloan)

Excellent list of SoTL publications and links to other lists from ISSOTL

The references below explain the "how to's" and issues to consider when undertaking research about teaching and learning. Several of the books provide in-depth discussions of the topics from this website.

Enjoy!

Bass, R. (1999). The scholarship of teaching: What’s the problem? Inventio: Creative Thinking about Learning and Teaching, 1(1), 1–10.

Bishop-Clark, C. & Dietz-Uhler, B. (2012). Engaging in the scholarship of teaching and learning: A guide to the process and how to deveop a project from start to finish. Stylus Publishing.

Boyer, E.L. (1997). Scholarship reconsidered: Priorities of the professoriate. Jossey-Bass.

Cross, K. P., & Steadman, M. H. (1996). Classroom research: Implementing the scholarship of teaching. San Francisco, CA: Jossey-Bass.

Gurung, & Wilson, J. H. (Eds.). (2013). Doing the scholarship of teaching and learning: Measuring systematic changes to teaching and improvements in learning. In New Directions for Teaching and Learning, 136.

Huber, M. T., & Morreale, S. P. (Eds.). (2002). Disciplinary styles in the scholarship of teaching and learning: Exploring common ground. Washington, DC: American Association for Higher Education.

Hutchings, P. (2000). Opening lines: Approaches to the scholarship of teaching and learning. Menlo Park, CA: Carnegie Foundation for the Advancement of Teaching.

Hutchings, P. (2002). Ethics of inquiry: Issues in the scholarship of teaching and learning. Menlo Park, CA: Carnegie Foundation for the Advancement of Teaching.

Illinois University's blog The SoTL Advocate

McKinney, K. & Huber, M.T. (2013). The scholarship of teaching and learning in and across the disciplines. Bloomington: Indiana University Press.

Vanderbilt University's site about scholarship of teaching and learning

*Cambridge, B. (2010). Fostering the scholarship of teaching and learning: Communities of practice. In D. Lieberman & C. Wehlburg, (Eds.). To improve the academy. Bolton, MA: Anker.

Special thanks to FACET members and fellow FACET Retreat workshop presenters: Dr. Julie Saam, IUK; Dr. Irwin Mallin, IPFW; and Dr. Gwendolyn Mettetal, IUSB for their contributions to this website.

Thanks to Dr. Tamul's 2015 Research Methods course at IPFW for their help in finding resources: Brad Affolder, Christie Bowen, JW Kieckheifer, Jordan Littlejohn, Saneta Maiko, Ashley Motia, Niki Reynolds, Israa Samarin, Amanda Seilheimer, Ashley Whitcraft, and Joe Wolter

 

Back to Top