Question 48·Hard·Evaluate Statistical Claims: Observational Studies and Experiments
A public health researcher compared the incidence of seasonal flu in two neighboring cities during the same winter. City A aired a televised campaign encouraging residents to get vaccinated, whereas City B aired no such campaign. The researcher observed that City A had a lower flu incidence than City B and concluded that the televised campaign caused the reduction in flu cases.
Which statement best evaluates the validity of the researcher’s conclusion?
For SAT questions about evaluating statistical conclusions, first identify the study type: is it an observational study (researchers just observe existing groups) or a controlled experiment with random assignment? Remember that only well-designed randomized experiments can justify strong causal claims. Then, quickly rule out answers that misunderstand basics (for example, thinking large samples are bad) and look for choices that correctly point out issues like lack of random assignment, possible confounding variables, or biased sampling when someone tries to claim causation from mere correlation.
Hints
Focus on the type of claim
The researcher is saying the campaign caused fewer flu cases. Think about what kind of study design is needed to support a causal claim.
Look at how the groups were formed
Did the researcher assign people or cities to get the televised campaign, or did they just observe what already happened? How does that affect what we can conclude?
Eliminate answers that misunderstand statistics
Ask yourself: Is observing the same time period enough to prove causation? Are large sample sizes usually a problem or an advantage in studies?
Step-by-step Explanation
Understand what the researcher is claiming
The researcher is not just saying that City A had fewer flu cases than City B; they are concluding that the televised vaccination campaign caused the reduction in flu cases. This is a causal claim, not just a description of what was observed.
Identify the type of study
In this situation, the researcher simply observed two neighboring cities: one that chose to air a campaign and one that did not. The researcher did not randomly assign individual residents (or cities) to receive or not receive the televised campaign. That means this is an observational study, not a randomized experiment.
Recall when causal conclusions are justified
Random assignment is what allows us to say that a treatment caused a difference in outcomes, because it balances other factors (like age, income, healthcare access, etc.) between groups. In an observational study like this, many other differences between City A and City B (for example, average age, typical health behaviors, or access to clinics) could be responsible for the difference in flu incidence. So a good evaluation of the conclusion must point out that the study design does not rule out these other explanations.
Match this idea to the correct answer choice
Two answer choices say the conclusion is valid, but a non-randomized observational comparison generally cannot prove causation, so those are not good evaluations. Another choice incorrectly blames “too large” sample sizes, which is not a flaw here—large samples are usually helpful. The remaining choice correctly notes that there was no random assignment to receive or not receive the televised campaign, so the researcher’s causal conclusion may not be valid for that reason.