Sampling Bias
Whenever we take a sample from a population, there is the potential of introducing sampling bias. It is important to be aware of potential sources of bias and take steps to minimize the chance that sampling bias is present in the way that we sample.
Here are the four main sources of bias to consider when sampling from a population:
- Undercoverage occurs when some groups of the population are left out of the sampling process and the individuals in these groups do not have an equal chance of being selected for the sample. For example, a sample survey of households in a country may miss people who are homeless, prison inmates, or students living in dorms.
- Non-response bias occurs when an individual chosen for a sample cannot be contacted or decides to not participate in the study or research. This type of bias occurs after the sample has been selected and can create potential bias in the data collected.
- Response bias is defined as a systemic pattern of inaccurate responses to questions. This type of bias can occur when a person does not understand a question or feels influenced to respond to a question in a certain way. Response bias can also occur as a result of the wording of questions that are of a sensitive nature.
- A voluntary response bias is another form of bias because the sample is not random or representative of the population. The people who volunteer for a study or survey may be more inclined to respond to questions or report certain behaviors.
We need to be aware of the potential sources of bias that will prevent us from appropriately generalizing sample results to the population. Work in pairs to answer Questions 2 – 5
question 2
Let’s say the student takes a simple random sample of students from her high school and asks those students about their race/ethnicity. Describe any potential sources of bias in her sampling method.
question 3
Assume the student creates a questionnaire that asks about race/ethnicity and then asks for volunteers at all three high schools to take the questionnaire. Describe any potential sources of bias in her sampling method.
Hopefully you are beginning to understand how to recognize a source of potential sampling bias. By looking at key words and phrases, like volunteer and questionnaire, or by realizing that only one group from a population has been sampled, you are able to identify a method that may tend to produce unrepresentative samples.
As you answer Question 4, refer to the list of sampling methods given in the preview page. Here is a summary of those methods.
- Simple random sampling assigns a number to every member of the population then uses a random number generator to select a sample.
- Systematic sampling assigns a number to every member of the population then choses individuals from the population at regular intervals (e.g. every 4th individual from a randomly selected starting point).
- Stratified sampling divides a population into groups via some criterion then uses simple random selection or systematic selection to collect a sample from each group.
- Convenience sampling selects a sample most accessible to the researcher.
Which of these do you feel would most likely tend to produce random samples from this particular population of three high schools?
question 4
Since there are three public high schools in the student’s county, describe how she might select a representative sample. Explain how the method you chose minimizes bias.
Finally, discuss Question 5 together in pairs and record your responses.
question 5
This student wants to use the results of her study to describe the race/ethnicity of all high school-aged students in her county (ages 14–18). What is the population of interest now? Is the sample of students described in Question 4 appropriate for this population? Explain.
Guidance
[Wrap-up: Did you find it challenging to come up with the best sampling method for the student’s population of three high schools to answer Question 4? You might have attempted to eliminate one or more methods first. Convenience sampling tends to produce unrepresentative samples, so that would be the first method to eliminate. Simple random sampling and systematic sampling both require the enrollment of all three schools to be merged first, then sampled. Both of these methods help to ensure that each individual of the population has an equivalent chance of being selected in a sample. But could these methods tend to blur distinctions of race and ethnicity that occur naturally in each schools unique populations? If we eliminate them, we are left with stratified sampling. What characteristic of this method could make it the most appropriate one to use? These types of questions are necessary to performing as strong a statistical analysis as possible. In some cases, we may find that not matter what method we use, there is little chance that the population of interest we have chosen could be used to generalize results to an even larger population, as you may have discovered in your discussion of Question 5. We’ll be learning about experimental design in the next section. The concepts and skills you’ve obtained during this activity will be helpful as you explore further.]
