Question 14·Medium·Evaluate Statistical Claims: Observational Studies and Experiments
A horticulturist wanted to test whether a new fertilizer increases the mass of tomatoes produced per plant. She selected 60 tomato plants that were all grown from the same seed variety in the same greenhouse and randomly assigned 30 of them to receive the new fertilizer and the other 30 to receive no fertilizer. All other growing conditions were kept identical for every plant. At the end of the season, the 30 plants that received the fertilizer produced a mean of 3.4 kilograms of tomatoes per plant, whereas the 30 plants that received no fertilizer produced a mean of 3.0 kilograms of tomatoes per plant.
The horticulturist concluded that, on average, the new fertilizer will increase the mass of tomatoes produced by any tomato plant by about 0.4 kilogram.
Which of the following is the primary reason this conclusion may not be warranted?
For experimental design and statistics questions, first summarize the setup in your own words: what was tested, how subjects were chosen, and how treatments were assigned. Then identify what the conclusion is really claiming—cause-and-effect, generalization to a larger population, or both. Match that against the rules: random assignment supports cause-and-effect; random selection supports generalization. Finally, eliminate answer choices that complain about things that are not required for this claim (like unnecessary extra groups or automatically assuming a sample is too small) and choose the option that correctly identifies the mismatch between the conclusion’s scope and the study’s design.
Hints
Focus on what was randomized
Look carefully at how the plants were chosen for the study versus how they were assigned to receive fertilizer or no fertilizer. Are those the same thing?
Think about the scope of the conclusion
Does the horticulturist draw a conclusion just about these 60 greenhouse plants, or about a much larger group of plants? What has to be true about the sample to justify that broader claim?
Distinguish cause-and-effect from generalization
Random assignment supports cause-and-effect for the plants in the study. What additional ingredient is needed if you want to claim the result holds for all tomato plants grown in many different ways?
Step-by-step Explanation
Understand the study design
The horticulturist:
- Chose 60 tomato plants of the same seed variety, grown in the same greenhouse.
- Randomly assigned 30 plants to get the new fertilizer and 30 plants to get no fertilizer.
- Kept all other conditions identical.
Random assignment to the two groups is good: it allows us to attribute differences in mean tomato mass between the groups to the fertilizer (for plants like these in this environment).
Identify what the conclusion is claiming
The conclusion says: "on average, the new fertilizer will increase the mass of tomatoes produced by any tomato plant by about 0.4 kilogram."
This is a generalization:
- It is not just about these 60 plants.
- It is about any tomato plant (all varieties, all locations, all growing conditions).
To justify a broad conclusion about a large population, the sample has to represent that population well.
Recall the roles of random assignment vs random selection
Key ideas from statistics:
- Random assignment (how units are put into treatment vs control) allows cause‑and‑effect conclusions for the units in the study and similar ones.
- Random selection (how units are chosen from a population) is what allows generalizing results to a larger population.
In this study:
- We do have random assignment to fertilizer vs no fertilizer.
- We do not have random selection of plants from all tomato plants grown under various conditions. Instead, all plants come from one variety in one greenhouse.
Match the statistical issue to the correct answer choice
Now evaluate the options:
- Choice A: Says the 0.4‑kilogram difference is too small. That is about practical importance, but the question asks about whether the conclusion is warranted statistically.
- Choice B: Claims 60 plants is not enough for any conclusions; 60 can be a reasonable sample size, and nothing in the prompt suggests otherwise.
- Choice C: A no‑fertilizer group is a valid control; a “standard fertilizer” group is not required to test whether the new fertilizer changes yield compared with no fertilizer.
- Choice D: Points out that the plants were not randomly selected from all tomato plants under various conditions, so the result may not apply to all tomato plants.
Because the main flaw is over‑generalizing from a very specific, non‑random set of plants, the correct answer is: D) The plants in the study were not randomly selected from all tomato plants that could be grown under various conditions.