Question 54·Hard·Evaluate Statistical Claims: Observational Studies and Experiments
A nutrition company compared the average weight loss of 500 customers who purchased its new meal-delivery plan with the average weight loss of 500 people who did not purchase the plan. After 12 weeks, the customers who used the plan lost an average of 8.2 pounds, while the comparison group lost an average of 3.1 pounds. The company advertises that the study proves its plan causes greater weight loss.
Which of the following best explains why this study does not provide strong evidence that the meal-delivery plan itself caused the larger weight loss?
For SAT questions about evaluating statistical claims, first identify what the author is claiming (especially if they say a study "proves" that one thing causes another). Then quickly check the study design: Were participants randomly assigned to groups (a randomized experiment) or did they choose or come pre-grouped (an observational study)? If there is no random assignment, be skeptical of any causal claim and look for answer choices that mention self-selection, confounding variables, or lack of randomization. Eliminate tempting but irrelevant issues like unit choice, sample size being "too large," or time span contradicting the given data.
Hints
Focus on the kind of claim being made
The company says the study proves that the plan causes greater weight loss. Think about what kind of study is needed to convincingly show cause and effect.
Look carefully at how the two groups were formed
Did the researchers randomly assign people to follow the meal plan or not, or did people decide on their own whether to buy the plan?
Think about other differences between the groups
If people decide for themselves whether to buy the plan, what kinds of characteristics (like motivation or lifestyle) might differ between those who buy it and those who don’t, and how could that affect the results?
Test each option against the real issue
Once you’ve identified the main weakness in the study’s design, check which choice describes that weakness, not something about units, sample size, or the existence of weight loss.
Step-by-step Explanation
Understand the company’s claim
The company is saying the study proves that its meal-delivery plan causes more weight loss, because the plan group lost 8.2 pounds on average and the comparison group lost 3.1 pounds on average after 12 weeks.
Recall what is needed to show causation
To give strong evidence that a treatment causes an effect, a study usually must be a randomized experiment:
- Participants are randomly assigned to either the treatment group or a control group.
- Random assignment helps make the groups similar in all other ways, so differences in outcomes are likely due to the treatment, not other factors.
Identify the study design in this problem
Here, the company "compared the average weight loss of 500 customers who purchased its new meal-delivery plan with 500 people who did not purchase the plan."
- That means people chose whether or not to buy the plan.
- This is an observational study with self-selected groups, not a randomized experiment. Because of this, the groups may differ in many ways (motivation, income, time to exercise, other dieting efforts) that also affect weight loss.
Match the flaw to the best answer choice
Now compare each option to the real flaw:
- A) A sample of 1,000 is actually reasonably large; large samples make conclusions more, not less, reliable.
- B) Using pounds instead of kilograms does not change which group lost more weight; it’s just a different unit.
- D) The results show that both groups did lose weight in 12 weeks, so it’s incorrect to say there was not enough time to record any weight loss. The only option that points out the key issue—that participants were not randomly assigned and may differ in important ways—is C) The people who chose to follow the meal-delivery plan were not randomly assigned, so they may differ in important ways from those who did not choose the plan. This explains why the study does not strongly prove that the plan itself caused the larger weight loss.