Question 11·Medium·Evaluate Statistical Claims: Observational Studies and Experiments
At one large suburban high school, 60 eleventh-grade volunteers were recruited to evaluate a new SAT Math review course. The 60 students were randomly assigned to two groups: 30 students took the review course (treatment), and 30 students did not (control). After the course ended, the treatment group’s average SAT Math score was significantly higher than the control group’s average score.
Which conclusion is best supported by this study?
For experimental-design and statistics interpretation questions, first identify whether the study is an experiment (random assignment to treatment vs control) or an observational study. Then separately check whether there is random sampling from a broad population. Use this to decide: (1) if you can talk about cause-and-effect (needs random assignment), and (2) how far you can generalize the results (needs random sampling). Finally, choose the answer whose wording matches both the correct strength (association vs causation) and the correct population (only participants vs state vs nation).
Hints
Focus on how the groups were formed
Look at the phrase about how the 60 students were split into two groups. Was it random selection from the population, random assignment to groups, both, or neither?
Think about causation vs generalization
Random assignment allows you to talk about cause-and-effect for the participants. Random sampling from a larger population allows you to generalize results to that population. Which of these is present here?
Limit the conclusion to the right group
The students come from one high school and they volunteered. Does that justify a statement about all students in the state or the country, or only about the particular students who were in the study?
Check key words in the answer options
Look carefully for the words that signal causation (like "causes") versus association (like "associated with"), and for words that describe the population ("volunteers in this study," "state," "nationwide"). Match those to what the study design supports.
Step-by-step Explanation
Identify the type of study
Read the setup carefully: 60 eleventh-grade volunteers at one school were randomly assigned to treatment and control groups.
- "Randomly assigned" to treatment vs control means this is a randomized experiment, not just an observational study.
- In a randomized experiment, differences between groups can be attributed to the treatment, as long as the difference in outcomes is real and not due to random chance.
Connect study type to causation vs association
In statistics, two key ideas are:
- Random assignment (experiment) lets us make causal conclusions about the participants in the experiment.
- Random sampling from a larger population would let us generalize results to that larger population.
Here, we do have random assignment (so causation for these volunteers is supported) but we do not have random sampling from all students in the state or country.
Determine the scope of the conclusion
Look at who is in the study:
- Only 60 volunteers from one large suburban high school.
- This group is unlikely to be a representative random sample of all students in the state or nation.
So we cannot reliably claim anything about all eleventh graders in the state or nationwide. Our conclusion must be limited to the volunteers in this study.
Match the correct statistical conclusion to the answer choice
Now match what we know to the options:
- We can say the review course causes higher SAT Math scores, but only for the specific volunteers in this experiment.
- We cannot extend that causal claim to all students in the state or in the nation.
The only option that states a causal conclusion and restricts it to the eleventh-grade volunteers in this study is: “For the eleventh-grade volunteers in this study, taking the review course causes an increase in SAT Math scores.”