Question 6·Hard·Evaluate Statistical Claims: Observational Studies and Experiments
A market research team at a video-streaming service wants to estimate the proportion of all current subscribers who would pay an additional $5 per month for a commercial-free plan. The team randomly selected 20,000 subscribers and emailed each one a questionnaire. A total of 1,900 subscribers completed the questionnaire, and 1,330 of them indicated that they would be willing to pay the extra $5. Based on these results, the team concluded that about 70% of all current subscribers would pay the additional fee.
Which of the following best explains why this conclusion may not accurately represent the sentiments of all current subscribers?
For survey and experiment questions, quickly identify the population (who we care about), the sample (who actually provided data), and how that sample was obtained. Check if every selected person responded or if participation was voluntary, which can create nonresponse or voluntary response bias. Then read each answer choice asking, “Does this describe a real reason the respondents might not represent the whole population?” Eliminate choices about sample size being “too small” when it is clearly large, or about details that don’t affect representativeness, and choose the option that best describes a plausible bias in the sampling or response process.
Hints
Clarify what the 70% represents
First, think about how the 70% was calculated. Is it based on all 20,000 selected subscribers, or only on the 1,900 who actually answered the questionnaire?
Compare intended sample vs actual data
Notice that 20,000 subscribers were contacted, but only 1,900 responded. How could it be a problem if the people who chose to respond are different in some way from those who did not respond?
Focus on representativeness, not just size
Ask yourself: Which answer choice talks about a reason why the group that provided data might not be representative of all current subscribers, even though the original selection was random?
Desmos Guide
Verify the reported proportion numerically
In Desmos, type 1330/1900 to confirm the decimal value. This lets you see that the research team computed the sample percentage correctly, so you can then focus your attention on whether the way the data were collected could make that percentage misleading for all subscribers.
Step-by-step Explanation
Identify the population and what is being estimated
The research team wants to know the proportion of all current subscribers who would pay an extra $5 per month for a commercial-free plan. From the data, they computed the proportion of respondents willing to pay:
- Number who said yes: 1,330
- Number who responded: 1,900
So the sample proportion is , or 70%. The question is not about the calculation itself but whether this 70% is a trustworthy estimate for all subscribers.
Examine how the data were actually collected
The team randomly selected 20,000 subscribers and emailed them a questionnaire. However, only 1,900 subscribers completed the questionnaire.
So there are two levels:
- Initial random sample: 20,000 subscribers
- Actual data used: the 1,900 people who chose to respond
The conclusion about 70% is based on the responders only, not on everyone who was selected.
Recognize the type of potential bias
When many people in a selected sample do not respond, and the people who do respond might be different from those who do not, this is called nonresponse bias (a kind of voluntary response bias).
In this context, people who care more about ads or are more engaged with the service might be more likely to answer the email survey, so the responders might not represent the average subscriber.
Match this issue to the best answer choice
Now compare each option to the problem just described:
- Saying the sample size of 1,900 is too small is not accurate; 1,900 is actually a large sample for estimating a proportion.
- The use of email is reasonable for an online streaming service, and nothing suggests this alone causes a major bias.
- The key issue is that only 1,900 of the 20,000 selected subscribers responded, and the responders might systematically differ from nonresponders (for example, caring more about skipping ads).
- The specific price of $5 does not by itself create bias in estimating willingness to pay at that price.
Therefore, the best explanation is: Not all of the 20,000 selected subscribers responded, and those who did respond may differ systematically from those who did not.