Question 9·Hard·Evaluate Statistical Claims: Observational Studies and Experiments
A research team obtained a roster of the 4,800 employees at a large company and randomly selected 500 of them to receive an e-mail invitation to a study about stress management. Of the 500 employees invited, 128 volunteered to participate and were randomly assigned to one of two groups:
- Meditation group: 64 employees attended a 20-minute guided meditation session each workday for 8 weeks.
- Control group: 64 employees were asked to continue their usual routines for 8 weeks.
At the end of the 8 weeks, the researchers measured each participant’s stress score on a standardized scale. The mean stress score of the meditation group was significantly lower than that of the control group.
Which of the following is the most reasonable conclusion based on the study?
For SAT questions about studies and conclusions, always map out the flow: who was selected, who actually participated, and how they were assigned to groups. Then decide (1) whether random assignment allows a cause-and-effect conclusion, and (2) how broadly you can generalize based on who the participants actually represent. Finally, choose the option that matches the observed result and respects these limits—avoid any answer that makes sweeping claims about much larger groups than the study directly supports.
Hints
Focus on who actually participated
Pay close attention to how the group of 4,800 employees was narrowed down. Who actually ended up in the meditation and control groups?
Think about random assignment vs. random selection
Random assignment helps with cause-and-effect, while random selection helps with generalizing to a larger population. Which of these did the researchers fully achieve with the final group of 128 participants?
Watch out for overgeneralizing
Look for answer choices that talk about all employees or all adult workers. Ask yourself: is the study strong enough to support such broad claims?
Step-by-step Explanation
Sort out how the participants were chosen
First, track the groups carefully:
- There are 4,800 total employees.
- From these, 500 were randomly selected to receive an email invitation.
- Of those 500, 128 volunteered to participate.
- Those 128 volunteers were then randomly assigned to the meditation group (64) or the control group (64).
So, the actual experiment is only on the 128 volunteers, not on all 4,800 employees and not even on all 500 who got the email.
Decide what kind of conclusion the design allows
Because the 128 volunteers were randomly assigned to meditation or control, this is a randomized experiment, not just an observational study.
- Random assignment to treatment vs. control lets us make cause-and-effect conclusions within this group of volunteers (if the difference is statistically significant, as stated).
- The result given: the mean stress score in the meditation group was significantly lower than in the control group.
So it’s reasonable to say that daily guided meditation caused lower stress levels for the people in the experiment — but we have to be careful about who that refers to.
Limit how far you can generalize
Even though 500 employees were randomly selected, only the 128 who chose to volunteer actually took part in the experiment. Volunteers may be different from non-volunteers (for example, more interested in stress management), so the final group is not a random sample of all 4,800 employees.
That means:
- We cannot safely say the results apply to all 4,800 employees.
- We also cannot safely say they apply to all adult workers in general.
We must restrict our conclusion to the people who volunteered and completed the study.
Match the reasoning to the answer choice
We want a choice that:
- Describes the actual participants (the volunteers who completed the study), and
- Reflects the observed difference between the meditation and control groups, without overgeneralizing.
Choice B does exactly this: it states that among the employees who volunteered and completed the study, those assigned to daily guided meditation had lower mean stress scores than those who did not meditate. This matches both the design of the experiment and the reported result, without claiming too much about other people.