Question 24·Hard·Evaluate Statistical Claims: Observational Studies and Experiments
To gauge public support for building a new dog park, a city council posted an online poll on the city’s website and promoted it on social media for two weeks. A total of 3,200 residents responded, and 68% of the respondents said they favored building the dog park.
Which of the following statements must be true?
For questions about statistical studies, first identify how the data were collected: random sample, voluntary response, online poll, etc. Then decide what conclusions are justified: random, well-designed samples can support estimates about a population; voluntary or convenience samples often cannot. When the question asks what must be true, reject any option that makes strong, absolute claims (like exact percentages or guaranteed changes) that go beyond what the sampling method supports, and look for the more cautious statement that follows logically from the design.
Hints
Focus on how participants were chosen
Look at how people ended up in the sample. Did the council randomly select residents, or did residents choose to participate on their own?
Pay attention to the phrase “must be true”
Ask yourself: Which statement is guaranteed by the way the poll was done, not just something that might be true?
Distinguish between the sample and the whole population
The 3,200 people who responded are a sample. All city residents are the population. Which options jump from sample results to strong claims about the entire population?
Watch for extreme or absolute language
Be cautious of choices that use words like “exactly” or “necessarily,” especially when the sampling method is not carefully controlled.
Step-by-step Explanation
Identify what the question is really asking
The question asks which statement must be true based on how the poll was conducted and the information given. That means the correct choice has to be guaranteed by the setup, not just plausible or likely.
Understand the sampling method
The poll was posted on the city’s website and promoted on social media for two weeks, and residents chose whether to respond. This is a voluntary online poll, not a random sample. In voluntary response samples, people with strong opinions are more likely to participate, and many residents may never see or answer the poll.
Think about what we can and cannot conclude
Because the sample is voluntary and not randomly chosen:
- A large sample size does not by itself guarantee that the sample is representative.
- We cannot know the exact percentage of all city residents who favor or oppose the park.
- We also cannot predict what would have happened if the poll ran longer; the results might change or might stay similar. What we do know for sure is that voluntary polls always carry a risk that the sample is biased and not representative.
Match this reasoning to the answer choices
Only one option matches what must be true: the voluntary nature of the online poll may have produced a biased sample that is not representative of all city residents. The other options make unjustified claims about accuracy, exact percentages, or how results would change over time, none of which are guaranteed by the information given.