Decide when a sample supports generalization

These questions describe a survey, then ask something like:

• Which population can the result be extended or generalized to?
• Is the sample representative of a certain group?
• Is the survey biased?
• Which sampling method would give reliable information about a population?
• What keeps the claim from reaching a larger group?

Each one comes down to two groups:

• The target population is the whole group the researcher wants to describe.
• The sample is the smaller group that was actually picked and measured.

Before you read the answer choices, fill in this sentence:

The researcher wants to describe ______, but the data came from ______.

If the blanks name different groups, look closely at how the sample was picked.

A random sample uses chance to pick people or objects from a population. In a simple random sample, every member of that population has the same chance of being picked.

Chance doesn’t play favorites, so a random sample helps you get a representative sample: one that doesn’t systematically leave out or overcount any important part of the population it came from.

Here’s the central rule:

The last three words do the work. Picking at random doesn’t make the population any bigger.

Say a museum randomly picks $150$ names from its complete list of annual members. The results can represent all annual members of that museum. They don’t automatically represent:

• all museum visitors
• all residents of the city
• members of other museums

Anyone who isn’t a member of this museum was never on the list, so chance never had a shot at picking them. The list chance actually picks from is called the sampling frame, and it sets the limit. So the key question is: who could have been picked?

Selection bias happens when the way people get into a sample systematically favors some parts of the target population over others. The sample can then lean toward an answer the full population wouldn’t give.

Convenience sample

A convenience sample uses whoever is easy to reach, such as:

• the first customers through one entrance
• people at one event or place
• one class, because it’s nearby
• visitors to a business tied to the survey topic

Easy to reach doesn’t mean representative. A place can draw people who share an interest, a schedule, an age range or a habit.

Volunteer or self-selected sample

In a voluntary-response sample, people decide for themselves whether to answer. That’s why it’s also called a self-selected sample.

People with strong opinions, or a special interest in the topic, may be more likely to answer. And posting a survey link where everyone can see it doesn’t make the respondents random. Everyone could answer, but chance didn’t choose who did.

An organized list isn’t automatically random

Picking the first $100$ names in alphabetical order, or the people at the top of a sign-up sheet, follows a rule, but chance plays no part. The order can be tied to something that matters. The top of a sign-up sheet, for example, holds the people who signed up first, and they may be the most eager. On top of that, not every member had the same chance of being picked.

When a sample wasn’t randomly picked from the target population, don’t swing to the other extreme and decide the data mean nothing. They still describe the people who answered.

Say $73\%$ of $200$ visitors surveyed at a food festival prefer outdoor seating. Then:

• it’s true that $73\%$ of those $200$ visitors gave that answer
• the survey can tell you something useful about those people
• but it doesn’t justify claiming that about $73\%$ of all city residents feel the same way

So there are three cases:

1. If chance picked from a list of the whole target population, like every household in town, the result can represent that whole population.
2. If chance picked from a narrower list, like one museum’s members, the result speaks only for the people on that list.
3. If the sample was convenient or self-selected, like the festival visitors, the result describes the people who answered and goes no further.

How close a sample percentage is likely to land to the population’s, and how sample size affects that, is a different question, the one Estimate populations and margin of error answers.

These two sound alike, but they answer different questions:

• Random sampling decides who gets picked from a population. It sets who the result can speak for.
• Random assignment decides which treatment each participant gets. It matters when a question asks about cause and effect.

For the questions here, you only need to ask one thing:

Was the sample randomly picked from the population the conclusion names?

When a question asks whether a treatment caused an outcome, that’s the job of random assignment, and it comes next in Decide when a study supports causation.

You won’t need Desmos for these. A calculator can’t tell you which list a sample came from, whether it’s representative or who the result can describe. Numbers in the question don’t change that, because the decision is about groups, not arithmetic.

Instead, work through the question in three moves:

1. Underline the group the conclusion or research goal names, like “all households in the town.”
2. Circle the people who could actually have been picked, like “households on the registration list.”
3. Check whether chance did the picking. That tells you which of the three cases above you’re in.

Then read the choices for overreach. Words like all, every and exactly, or a group bigger than the one you circled, are the usual giveaways.