What to Know About 2C: Experimental Design

learning GOALS

At the end of this page, you should feel comfortable performing these skills:

  • Identify the factor of interest and response factor in an experiment.
  • Identify the nuisance factors in an experiment.
  • Identify the experimental and control groups in an experiment.
  • Recognize random assignment.
  • Identify which of two methods of replication was used in an experiment.

Experimental design refers to the structure of an experiment (a specific type of research method). How can the experiment be conducted efficiently, effectively, and responsibly, to return sound results? Researchers must carefully consider several key components of experimental design before any work is done to collect data.

In the upcoming activity, you will need to identify the key components of experimental design. We’ll take them one by one in the following questions.

Experimental Design

An experimental study, or experiment, is a specific method of research that is especially useful for determining cause and effect. Typically, a basic experiment design includes a treatment imposed by the researcher, a comparison between one group that receives the treatment and another group that does not, and random assignment of participants into each group. We’ll explore each of these concepts in more detail below.

To help you understand the key components of experimental design, consider this simplified experiment:

[insert an image of a lake? fish eggs? pH strips?]

An ecologist wants to know if acid rain affects the rate of reproduction for fish in a local lake. The ecologist incubates 1,000 fish eggs in healthy lake water (pH = 6.5) and another 1,000 fish eggs in acidic water (pH = 4.5). The eggs have been randomly assigned to either the healthy lake water group or the acidic water group. The water temperature, amount of dissolved oxygen, and amount and timing of light were kept at optimal levels for both groups. After 14 days, the ecologist counted how many eggs from each group had hatched.

Each of the questions below will ask about aspects of the design present in this experiment.

question 1

A good experiment always starts with a good research question. For an experiment, the research question is usually about whether one thing causes another. Fill in the blanks below to summarize the ecologist’s research question.

“Does ______________________ cause ________________________?”

Factors

In an experimental study, the factor of interest (also known as the explanatory variable or independent variable) is the factor that the researcher purposely changes or manipulates to see if it impacts a specific outcome. The treatments are the different levels of the factor of interest (or explanatory variable) you are changing (e.g., no vaccine vs. vaccine).

The response variable (also known as the dependent variable) is an objective measure of the research question that is measured at the end of an experiment and compared across the different levels of the factor of interest (or explanatory variable).

A small flowchart. The first box says "Factor of Interest / Explanatory Variable." There is an arrow beside it labeled "affects" going to the second box, labeled "Response Variable."

The nuisance factors are factors that are kept the same across all levels of the factor or are explicitly controlled in the experimental design. These factors are not of interest in the study but may affect a change in the response variable.

Video Placement

[A video briefly describing factor of interest, treatment, response variable, and nuisance factors. This should be no more than a minute long. ]

Example

Summarize the terminology surrounding factors by answering the questions below.

  1. The factor of interest is also known as the _________________. Fill in the blank.
  2. The response variable is also known as the ________________. Fill in the blank.
  3. What does the researcher purposely change or manipulate to see if it impacts a specific outcome? Choose all that apply
    1. a) experimental factor
    2. b) explanatory variable
    3. c) response factor
    4. d) factor of interest
  4. What are treatments in an experiment?
    1. a) medicine applied to one set of individuals but not the other
    2. b) the different levels of the factor of interest (or explanatory variable) you are changing
  5. What are nuisance factors in an experiment?
    1. a) types of observational units that are particularly bothersome such as loud talkers
    2. b) types of factors that researchers are unable to account for in an experiment
    3. c) factors that are kept the same across all levels of the factor or are explicitly controlled in the experimental design
  6. What is measured at the end of an experiment and compared across the different levels of the factor of interest?

Now you try. Use the definitions given above to answer Question 2. Remember to view the feedback for your answers to assess your understanding.

question 2

Identify the different parts of the experimental design in the previous simplified experiment.

Part A: What is the factor of interest?

  1. a) pH level
  2. b) Temperature
  3. c) Oxygen level
  4. d) Amount of light
  5. e) Timing of light
  6. f) Incubation duration
  7. g) Number of eggs hatched

 

Part B: What is the response variable?

  1. a) pH
  2. b) Temperature
  3. c) Oxygen level
  4. d) Amount of light
  5. e) Timing of light
  6. f)  Incubation duration
  7. g) Number of eggs hatched

 

Part C: What are the nuisance factors in the previous experiment? There may be more than one correct answer.

  1. a) pH
  2. b) Temperature
  3. c) Oxygen level
  4. d) Amount of light
  5. e) Timing of light
  6. f) Incubation duration
  7. g) Number of eggs hatched

Controls

An important aspect of all experiments is a comparison between one or more groups or levels of the factor of interest, called treatments. In a simple experimental design, one group receives the researcher’s treatment and the other group(s) does not. The group that receives the treatment is commonly called the experimental group. The group that does not receive the treatment is usually called the control group. The control group is usually kept under conditions that are considered typical or common for a given situation.

A small flowchart. At the top, there is a box that reads "Available Test Subjects/Experimental Units. As similar as possible." There are two arrows leading to different boxes, both labeled "Randomize." One of the boxes reads "Control Group - Not Manipulated," while the other reads "Experimental Group - Manipulated Design Factor."

It is important to remember that the only difference between the experimental and control groups is the researcher’s manipulation of the factor of interest.

example

Complete the sentence by filling in the blank.

The _______________ is not manipulated, while the _________________ receives the researcher’s treatment.

Now you try answering Question 3 to identify the experimental and control groups in the simplified experiment given above.

question 3

In the previous simplified experiment:

Part A: Which group is the experimental group?

  1. a) 1,000 fish eggs in water with a pH of 6.5
  2. b) 1,000 fish eggs in water with a pH of 4.5

 

Part B: Which group is the control group?

  1. a) 1,000 fish eggs in water with a pH of 6.5
  2. b) 1,000 fish eggs in water with a pH of 4.5

 Random Assignment

To ensure that there are no unexpected differences between the experimental and control groups, good experimental design uses random assignment to determine which participants are in each group. This helps to minimize the impact of other variables that the researcher has no control over. Do not confuse this with random sampling! (In fact, most experiments do not use random sampling to find their sample.)

In our previous fish experiment, random assignment means that no matter how the 2,000 fish eggs were obtained, the ecologist would use some system of randomization to divide the eggs into two equal groups. For example, the researcher can use a random number generator to select 1,000 fish to be in the control group and then assign the remaining 1,000 fish to the experimental group. This would help to minimize the impact of other potential variables, such as the genetics and health of individual eggs.

Video Placement

[Perspective Video with a worked example: a 3-instructor video that explains the difference between random sampling and random assignment. These types of similar terms present a significant challenge to students who don’t yet have a mental landscape of statistics. This explanation should be followed by one or two examples in the style of Question 4]

Now it’s your turn to try identifying random assignment using statistical descriptions.

question 4

Identify the scenario(s) below that describe(s) random assignment (not random sampling). There may be more than one correct answer.

  1. a) A political pollster uses an electronic database to randomly generate a list of telephone numbers to call with questions about an upcoming election.
  2. b) A researcher uses a random number generator to determine which patients will receive an experimental medication and which patients will receive standard treatment.
  3. c) A researcher flips a coin to decide whether to assign a new participant to the experimental or control group.
  4. d) A professor uses a random number generator to decide which students to contact for a course evaluation.

 Replication

The final component of experimental design that we will explore is replication. In addition to random assignment, replication helps ensure that the results of an experiment are truly caused by the change in the factor of interest and not by other hidden factors or natural variation in data. A good experiment always achieves replication by including large numbers of participants, but replication can also be achieved by repeating the entire experiment with new groups of participants. As you will learn in future lessons, certain statistical analyses can only be performed if the sample meets a minimum size requirement.

See the example below for a summary of replication, then answer Question 5.

example

Fill in the blanks to complete these sentences.

Replication helps ensure that the results of an experiment are truly caused by ________________________ and not by ____________________________. A good experiment can achieve replication by ________________________ or by _______________________________.

Now you try it. Use the information you found above to help you answer Question 5.

question 5

Which form of replication is demonstrated in the previous fish experiment?

  1. a) Many participants included in one experiment
  2. b) The experiment repeated with different participants

Looking ahead

In the upcoming activity, you will design an experiment to study the health of coral reefs in our oceans. Use the National Ocean Service website provided here to learn some basic facts about coral, then record your responses to Question 6. Make sure to have your responses available during the activity.

Read the article on this webpage (make sure to read the “Did you know?” section!) then answer Question 6.

https://oceanservice.noaa.gov/facts/coral.html.

Question 6

Answer the following questions. Don’t forget to have your answers available during the upcoming activity!

 

Part A: Is coral a plant? An animal? Both?

 

Part B: What are some factors that impact the health of coral reefs?

 

Part C: What is coral bleaching?

Summary

In this What to Know page, you learned several key components of experimental design. Let’s summarize where each of these terms showed up on the questions.

  • In Question 2, Parts A and B, you identified the factor of interest and response factor in an experiment.
  • In Question 2, Part C, you identified the nuisance factors in an experiment.
  • In Question 3, Parts A and B, you identified the experimental and control groups in an experiment.
  • In Question 4, you recognized random assignment.
  • In Question 5, you identified which of two methods of replication was used in a study.
  • In Question 6, you prepared for the upcoming activity. Don’t forget to have your answer ready when you begin the activity!

There was quite a lot of information in this section. Feel free to return to it as needed in the future to refresh the definitions and your understanding. If you feel comfortable with these ideas, it’s time to practice them in the upcoming activity.