Question 56·Medium·Evaluate Statistical Claims: Observational Studies and Experiments
A city health department surveyed people who were working out at three downtown gyms one Saturday afternoon. Of the 480 people surveyed, 76% said they met federal exercise guidelines for the previous week. The department’s press release states: "About three-quarters of city residents meet the guidelines."
Based on the study design, which statement best evaluates this claim?
For questions about statistical claims, always separate what the claim is about (the population) from who was actually studied (the sample). Check whether the sample was randomly selected and representative of the population, or if it was a convenience or biased sample (like only volunteers, only customers, or only people in a specific place). Do not be distracted by large sample sizes or nicely rounded percentages; focus on whether the study design and sampling method really justify extending the result to the entire population.
Hints
Compare the sample to the population
First, identify who the city is making a claim about (the population) and who was actually surveyed (the sample). Are these two groups the same?
Think about how the sample was chosen
Were the people in the survey selected randomly from all residents, or were they chosen because they were easy to reach in one particular setting?
Consider whether an experiment is necessary
To estimate what fraction of people have a certain characteristic (like meeting exercise guidelines), do you always need an experiment, or can an observational survey be enough if it is designed well?
Look beyond the percentages
Even if is close to three-quarters, ask whether the way the data were collected allows you to extend that number to all city residents.
Desmos Guide
Compare 76% to three-quarters numerically
In Desmos, type 0.76 on one line and 3/4 on another line. Compare the decimal values shown. You will see they are very close, so the numerical part of the statement is reasonable; the real question is whether this percentage from gym-goers can stand for all city residents, which Desmos cannot check—you must judge the sampling method.
Step-by-step Explanation
Identify the population in the claim
The press release says, "About three-quarters of city residents meet the guidelines."
That means the population they want to draw a conclusion about is all city residents, not just people who visit gyms.
Identify who was actually surveyed
The health department surveyed people who were working out at three downtown gyms one Saturday afternoon.
So the sample is:
- Only people who choose to go to a gym
- Only three gyms
- Only at one particular time (one Saturday afternoon)
This is not a random sample of all residents; it is a specific group chosen because they were easy to reach.
Decide if the sample is representative of all residents
Ask yourself: Are people at a gym on a Saturday afternoon likely to have similar exercise habits to all city residents, including those who never go to a gym?
People who are at a gym are probably more active and health-conscious than average. That means the sample is likely biased toward people who exercise more than typical residents.
Because of this, the sample may overestimate the true percentage of all residents who meet the guidelines.
Compare this reasoning to the answer choices
Now evaluate the choices:
- A focuses on sample size and number of locations, but ignores that the sample is not representative.
- B incorrectly says experiments are required to estimate a proportion, which is not true; surveys are fine if samples are good.
- D focuses only on the numerical closeness of to three-quarters, but ignores whether the study design supports generalizing to all residents.
The only choice that correctly points out the problem—that the sample was a convenience sample of gym-goers and may not represent all city residents—is:
C) The claim is not justified because the sample was a convenience sample of gym-goers and may not represent all city residents.