Question 32·Medium·Evaluate Statistical Claims: Observational Studies and Experiments
A nutritionist wanted to determine whether a newly developed snack bar increases people’s energy levels. She recruited 80 adults who were members of her local gym to participate in the study. Each participant ate one of the snack bars every day for two weeks, and at the end of the study, 68 of the participants reported that they felt “more energetic.”
The nutritionist concluded that eating the snack bar will make most adults feel more energetic. Which aspect of the study design most likely weakens the validity of this conclusion?
For SAT questions about study design and validity, first underline the conclusion (what the researcher is claiming) and note whether it’s about a broad population or just the sample. Then scan the description of the study for how participants were selected and whether they represent that population. When comparing answer choices, pick the flaw that most directly blocks the conclusion (like biased sampling for population claims), rather than secondary issues like imperfect measurement or less-than-ideal duration, unless the question text clearly emphasizes those.
Hints
Focus on the wording of the conclusion
Look closely at the phrase "will make most adults feel more energetic." What kind of group does the conclusion try to describe compared with who was actually in the study?
Think about representativeness
For a study to support a statement about most adults, what should be true about how the participants are chosen?
Compare the answer choices by impact
Ask yourself: Which option affects whether the results can be applied to all adults, not just whether the measurement is perfect or the timing is ideal?
Step-by-step Explanation
Understand the conclusion being made
The nutritionist concluded that eating the snack bar will make most adults feel more energetic. This is a broad generalization about the entire adult population, not just about the people in the study.
Recall what is needed to generalize to a population
To make a valid conclusion about most adults, the participants in the study should be representative of all adults. The best way to get this is by using a random sample from the whole adult population, not a narrow or special group.
Evaluate each aspect of the study design
Now think about which option most directly affects whether the results can apply to most adults:
- Sample size (80 people): Is this automatically “too small,” or could it still give useful information?
- Self-reported energy: Does this mainly affect how accurately energy is measured, or whether we can apply results to all adults?
- Where participants came from: Are gym members similar to all adults, or might they differ in important ways related to energy?
- Study length: Does two weeks completely prevent us from learning anything about energy changes, or is this mainly about long-term effects?
Identify the key flaw that blocks generalization
The biggest problem for a conclusion about most adults is that participants were recruited from a single gym instead of being randomly selected from the adult population. Gym members are likely more health-conscious and possibly more energetic than average adults, so their responses may not represent everyone. This makes that option the aspect of the design that most weakens the validity of the nutritionist’s conclusion.