Question 27·Hard·Evaluate Statistical Claims: Observational Studies and Experiments
Researchers wanted to investigate whether drinking 16 ounces of water immediately before breakfast affects how many calories people consume at breakfast.
• From a city’s census records, the researchers selected a simple random sample of 120 adults. • They randomly assigned 60 of the sampled adults to drink 16 ounces of water 15 minutes before breakfast and the other 60 to drink nothing before breakfast. • All 120 adults then ate a standardized buffet-style breakfast, and the researchers recorded the number of calories each person consumed.
The researchers found that the mean calorie intake of the water group was significantly lower than that of the no-water group (two-sample -test, ).
Which conclusion is best supported by this study?
For questions about conclusions from studies, first identify whether the study is an observational study or a randomized experiment: only randomized experiments can justify causal language like “causes.” Next, check how the sample was chosen to decide the scope—if it is a random sample from a clearly defined population, you can generalize to that population but not beyond (for example, “adults in the city” vs. “adults nationwide”). Finally, use any reported -values or “statistically significant” language to rule out options that deny evidence of a relationship.
Hints
Check whether this is an experiment or an observational study
Look at what the researchers did with the 120 adults. Were they only observed as they naturally behaved, or were they actively assigned into groups?
Think about what random assignment allows you to conclude
If subjects are randomly assigned to treatments, what kind of relationship between treatment and outcome are you allowed to claim when you see a real difference?
Consider who the results apply to
The adults were sampled from one city. Does that support a conclusion about that city, the entire nation, or both?
Use the p-value to eliminate an option
A -value less than 0.01 shows strong evidence of a difference in mean calorie intake between groups. Which option clearly contradicts this idea?
Step-by-step Explanation
Identify the type of study
Look at how the researchers collected and handled the subjects:
- They selected 120 adults using a simple random sample from the city’s census records.
- They then randomly assigned 60 to drink water and 60 to drink nothing.
Because of the random assignment to treatment vs. no treatment, this is a randomized experiment, not just an observational study.
Decide whether causation is allowed
In a randomized experiment, random assignment tends to balance other variables (like usual eating habits, metabolism, etc.) between the two groups. That means any systematic difference in outcomes can reasonably be attributed to the treatment.
So, if the data show a real difference between groups, a randomized experiment can support a cause-and-effect conclusion about the treatment and the outcome, not just an association.
Interpret the p-value and what it says about an effect
The study reports a two-sample -test with for the difference in mean calorie intake.
- A very small -value means the observed difference in means would be very unlikely to occur just by random chance if there were really no difference between the water and no-water groups.
- Therefore, the study provides strong evidence that there is a difference in mean calorie intake between the groups.
So we should reject any choice that says there is no evidence of an association.
Decide how far we can generalize the conclusion
There are two separate questions:
- Causation vs. association: Because this is a randomized experiment and the result is statistically significant, we can support a causal conclusion (the treatment affected calorie intake), not just an association.
- Scope (who does it apply to?): The subjects were a simple random sample of adults in one city, not adults nationwide. That allows us to generalize to adults in that city, but not to all adults in the entire country.
So the best-supported conclusion is the one that states a causal effect of drinking 16 ounces of water before breakfast on calorie intake, and limits that claim to adults in the city: “Drinking 16 ounces of water before breakfast causes adults in the city to consume fewer calories at breakfast.”