Question 22·Easy·Evaluate Statistical Claims: Observational Studies and Experiments
A magazine wants to estimate the average daily time teenagers in the country spend on social media. The magazine posted an online poll on its technology news website, and 8,000 readers who identified themselves as teenagers responded.
Which of the following best evaluates the method the magazine used to collect its data?
For SAT questions about surveys and experiments, first identify the population (who we care about) and the sample (who was actually studied). Then focus on the sampling method: Is it random, or is it a convenience or voluntary sample like an online poll? Remember that a large sample size cannot fix a biased sampling method. Look for answer choices that correctly point out issues like self-selection, limited access (only website visitors, only customers, only club members), and groups being systematically excluded, and be skeptical of choices that rely only on sample size or on the idea that “anyone could have responded.”
Hints
Clarify who we care about
Ask yourself: Who does the magazine want to draw conclusions about, and who actually answered the poll?
Think about how participants were selected
Did every teenager in the country have an equal chance of being in the sample, or did only a certain type of teenager tend to see and answer this poll?
Consider the effect of a voluntary online poll
When people choose on their own whether to respond to a poll posted on a specific website, what kind of bias might that introduce into the results?
Check each choice against your reasoning
Look for the option that talks about the quality of the sampling method (who is included and how), not just the number of responses.
Step-by-step Explanation
Identify the population and the sample
The magazine wants information about all teenagers in the country. That is the population. The actual data, however, come from 8,000 readers of a technology news website who chose to respond to the online poll. That group is the sample.
Examine how the sample was chosen
The poll was posted on a technology news website, and only people who chose to respond were included. This means:
- Only teenagers who visit that specific tech site could even see the poll.
- Even among those visitors, only the ones who wanted to respond did so.
This is called a voluntary (self-selected) online sample, not a random sample of all teenagers in the country.
Decide whether the sample is representative
Ask: Are teenagers who read a technology news website and choose to respond to a poll likely to be similar to the average teenager in the country?
- They may be more interested in technology and social media than average.
- Teenagers who do not visit that website (or do not have easy internet access) are not represented at all.
So the sample is likely to be systematically different from the full population, which is exactly what we call sampling bias.
Match this reasoning to the best answer choice
Now compare the choices to what you found:
- A focuses only on the large number of responses, but a large biased sample is still biased.
- C says the sample is unbiased just because any teenager could respond, but in practice most teenagers never see the poll or choose not to respond.
- D suggests fewer responses would be more representative, which does not fix the core problem: who is in the sample.
The only choice that correctly points out that the method is likely biased because it relies on self-selected readers of a technology news website is choice B.