Question 41·Medium·Evaluate Statistical Claims: Observational Studies and Experiments
A public-health researcher wants to estimate the average number of hours teenagers in her state sleep on school nights. She obtains a roster of all students at her own high school, randomly selects 100 names, and emails each student a questionnaire about their sleep habits.
Which of the following best describes the scope and limitation of this study?
For survey and experiment questions, first separate the population of interest (who we want to know about) from the sample (who was actually studied) and the sampling frame (the list we sampled from). Ask two key questions: (1) Is the sample randomly selected from the entire population, or only from a subset (like one school)? That tells you how far you can generalize the results. (2) Did the researchers assign treatments or control conditions, or just observe/ask questions? That tells you whether you can claim causation or only association. Use these ideas to quickly eliminate options that overgeneralize, claim causation from an observational study, or misunderstand the target population.
Hints
Who do we care about vs. who was actually studied?
Underline the group the researcher wants information about, and separately underline the group from which she actually selected students. Are these the same?
Think about the sampling frame
Even though 100 students were chosen at random, ask: random from what list? Is that list all teenagers in the state, or a smaller group?
Is this an experiment or an observational study?
Did the researcher assign different workloads or sleep schedules to students, or just ask questions? This determines whether the study can support cause-and-effect claims.
Is the study really invalid?
Consider whether the study must include people outside the target population to be valid, or whether it can still tell us something useful about at least some group.
Step-by-step Explanation
Identify the population of interest
The researcher wants to know the average number of hours teenagers in her state sleep on school nights. That means the target population is all teenagers in the state.
Identify the actual group sampled
Look carefully at who was actually surveyed:
- She obtains a roster of all students at her own high school.
- She then randomly selects 100 names from that roster.
So the sampling frame (the list from which the sample is drawn) is only students at her own high school, not all teenagers in the state. The sample is random within that one school, but it is not a random sample from the whole state.
Decide what the study can and cannot generalize
Because the sample comes only from one high school:
- The results can reasonably estimate the average sleep of students at that high school, since the 100 students were randomly chosen from that school’s roster.
- The results cannot safely be generalized to all teenagers in the state, because teenagers at other schools or in other areas might have different sleep patterns.
Also, notice this is just a questionnaire—students report their sleep habits. The researcher did not assign different workloads or control conditions, so this is an observational study, not an experiment. It cannot prove that workload causes less sleep.
Match the reasoning to the best choice
Now check each type of claim:
- Any option saying the study gives an unbiased estimate for all teenagers in the state ignores that the sample came from just one school.
- Any option saying the study establishes that workload causes reduced sleep is wrong, because this is a questionnaire, not a controlled experiment.
- Saying the study is invalid because teenagers outside the state were not surveyed is wrong: the population of interest is teenagers in the state.
The only statement that correctly reflects the scope (students at the researcher’s high school) and the limitation (sampling frame limited to that school, so not all teenagers in the state) is: “The study can estimate average sleep only for students at the researcher’s high school because the sampling frame was limited to that school.”