Quantitative Design Considerations

Correlation Overview (2:50)

Correlational designs
- interpretation of strength and direction of correlations (from Pearson's r and scatterplot)
- Issues with causality (reverse-causality; third-variable problem)
- common data collection techniques (surveys, observations)

Overview of Experimental Designs (5:36)

Experimental designs
- independent variable (manipulated cause = treatment vs. control group)
- dependent variable (measured outcome = same for both groups)
- random assignment (to treatment vs. control group)
- experimental control (to control for third variables)

What are the differences between experimental and quasi-experimental design?

Component	Experimental	Quasi-Experimental Design
Random Assignment	Participants are randomly assigned to treatment or control group	Participants are not randomly assigned; they are in naturally occurring groups (e.g., two classes; men vs. women).
Control Group	Experiments require a control group (does not receive the treatment).	Control groups are not mandatory.
SoTL Example	Students are randomly assigned to receive either the treatment manipulation or not.	An instructor compares exam performance between first-generation and continuing-generation students.

How does the research design I select affect the types of analyses I'll do?

Descriptive statistics summarize the characteristics of a data set (e.g., Mean, Standard Deviation, Percent).
Inferential statistics help you come to conclusions and make predictions based on your data (e.g., correlations; t-tests; ANOVA).

Research Design and Analysis Examples

Design Option #1: One-Group, Post-Test Only (Descriptive)
Descriptive Statistics: Mean, Standard Deviation, and Range of test scores
Inferential Statistics Test: One-sample t-test
Research Question: What is the average test score for students who used a Think-Pair-Share technique?

Design Option #2: One-Group, Pre-Test and Post-Test (Quasi-Experimental)
Inferential Statistics Test: Paired-sample t-test
Research Question: Does the use of a Think-Pair-Share technique improve baseline test scores?

Design Option #3: Two-Groups, Post-Test Only (Experimental)
Inferential Statistics Test: Independent-samples t-test
Research Question: Do students who used the Think-Pair-Share technique score higher on a test of content knowledge than those who did not?

Design Option #4: Two-Groups, Pre-Test and Post-Test (Experimental)
Inferential Statistics Test: Repeated-Measures ANOVA
Research Question: Do students who used the Think-Pair-Share technique score higher on a test of content knowledge than those who did not (controlling for baseline level of knowledge)?

Things to Ask Yourself

Are you manipulating something in your study?
- Yes, some students will receive the treatment and some won't --> Experimental (or Quasi-Experimental) Design
- No, I'm just measuring students' experiences, attitudes, beliefs, or personality traits --> Correlational Design
Do you need a comparison or control group?
- Two courses receiving different treatments --> Yes, Quasi-Experimental Design
- Randomly assigning students to receive treatment or not --> Yes, Experimental Design
- Examining the relationship between two concepts or variables --> No, Correlational Design
When will you administer your surveys or collect student data?
- Are you using a "post-test only" design? --> collect at the end of semester or immediately after the intervention
- Are you using a"pre-post test" design? --> collect at the beginning and end of semester or before/after intervention

Additional Resources