A city maintains an address list of all households in the city. To study whether a message about potential savings affects enrollment in a discounted transit program, researchers randomly selected 2,000 households from the list and mailed each household an eligibility survey. Of the selected households, 1,060 returned the survey. Among the households that returned the survey, 700 were eligible for the program.
The researchers randomly assigned 350 eligible households that returned the survey to receive a standard enrollment letter and the other 350 to receive an enrollment letter that included a comparison of the program's cost with typical transit costs. Of the households receiving the standard letter, 147 enrolled; of the households receiving the comparison letter, 189 enrolled.
A researcher claims that, for all eligible households in the city, including the cost comparison causes the enrollment rate to be 12 percentage points higher. Which choice best evaluates this claim?
For study-design questions, first identify the population of interest and the people actually included in the analysis. Then distinguish random assignment, which supports cause-and-effect conclusions for the studied group, from random sampling, which supports generalizing results to a population. Finally, check for nonresponse or another selection process that can limit generalization.
Hints
Separate the two design features
Identify which part of the study involves random selection and which part involves random assignment.
Consider the analyzed group
The letter comparison was conducted only among eligible households that returned the survey. Ask whether that group must represent every eligible household in the city.
Match each feature to its conclusion
Random assignment helps support a cause-and-effect conclusion within the assigned group. Random sampling and response rates affect how broadly a result can be applied.
Desmos Guide
Use reasoning first
This is mainly a study-design question. Desmos can verify the reported enrollment-rate difference, but it cannot decide whether survey respondents represent the city's eligible households.
Verify the rate difference
In the Desmos expression line, enter and . Subtract the first result from the second to verify that the difference is .
Interpret the design outside Desmos
Use the prompt to identify that random assignment supports causation within the responding eligible households, while the incomplete survey response limits generalization to all eligible city households.
Step-by-step Explanation
Verify the reported difference
The enrollment rates are for the standard letter and for the comparison letter. Their difference is , or percentage points.
Use random assignment correctly
The eligible households that returned the survey were randomly assigned to letter types. Therefore, a difference in their enrollment rates can be attributed to the letter type for that group.
Check whether the result applies to the whole city
Only households that returned the eligibility survey were included in the experiment. Since of the selected households did not return it, the responding households may differ from other eligible households. Thus, the causal result cannot automatically be extended to all eligible households in the city.