learning GOALS
- Determine whether or not an experiment has been designed well.
- Use a diagram to illustrate the design of a given experiment.
- Determine factors, other than the factor of interest, that could have an effect on the response variable of a given experiment (nuisance factors).
- Identify blocks in a specified experiment.
In the next activity, you will need to identify the basic principles of type of experimental design called the completely randomized block design. You’ll also need to and describe the role blocking plays and how to use a completely randomized block design to achieve a research objective.
Randomized Design
In previous activities, you learned that the design of an experiment can have a major impact on its results. If an experiment is not designed well, its results may be meaningless and misleading.
Good Experimental Design
In Questions 1 – 3 below, you’ll consider a large and important study as your deepen your understanding of the key components of an experiment, and pick up some new terms. Recall the key components of experimental design you learned about in section [2C]: factor of interest, response factor, nuisance factors, treatments, and random assignment. Can you describe what role each of these plays in an experiment?
Recall
Core skill:
Recall that researchers conduct experiments by assigning subjects (sometimes called experimental units) to certain experimental conditions or treatments (also called levels of the factor of interest) and then observing outcomes on the response variable. One experimental condition is the treatment of interest — this is what the researcher changes or manipulates to see if it causes a change in the response variable. You learned that the other group is called the control group. The control group does not receive the treatment of interest and is usually kept under conditions considered typical or common. The control group is essential to the experiment. It shows that, all else being equal, the factor of interest is either making a change in the response variable or it isn’t. There are some particular techniques that researchers use to make sure all the treatment groups are indeed otherwise equivalent.
- A placebo is a type of control treatment. A placebo is a harmless version of the treatment that does not contain any active ingredients (e.g., a sugar pill). It will typically look, taste, and smell like the treatment of interest or mimic it so that the two treatments appear identical to the subjects; this way the subjects don’t know which treatment they are receiving. A phenomenon called the placebo effect can sometimes, though, introduce ambiguity in the assessment of any changes occurring in the response variable. You’ll read more about the use of a placebo and the placebo effect in Question 1 below.
- Researchers will use a technique called blinding, in which the subjects do not know which treatment they received. A double-blind study is one in which neither the subject nor the people collecting data on the subjects know about the treatment assignments. Other blinding levels exist as well, especially in medical drug trials, including ones in which only the top level directors of a study have information about which treatment the patient receives. The patients, data collectors, and even the patients’ doctors and the people analyzing the data don’t know who received the treatment and who got the placebo.
Look for the terms placebo, placebo effect, blind, and double-blind in the questions and examples below.
Video Placement
[Perspective Video: A summary of the key components: factors (interest, response, nuisance), treatments (experimental, control, placebo, placebo effect), treatments (aka levels[and why “levels”? e.g. different dosages of a drug], random assignment, blinding, double-blinding) — it would be beneficial to include a diagram like the one in Question 3]
[A choose-your-study example could be placed here to mimic Question 1 – 3]
Now it’s your turn to explore a large and influential experiment as you answer Questions 1 – 3. Keep your eye out for the key components and terms you’ll seen so far.
Question 1
In the 1950’s, a large-scale experiment was designed to test the effectiveness of the Salk vaccine in preventing polio, which had killed or paralyzed thousands of children. Children were placed in either the experimental or control group through a process of random assignment. Suppose that [latex]200,745[/latex] children were given a treatment consisting of Salk vaccine injections, while a control group of [latex]201,229[/latex] children were injected with a placebo that contained no drug. The children being injected did not know whether they were getting the Salk vaccine or the placebo. Among the children given the Salk vaccine, [latex]33[/latex] later developed paralytic polio, but among the children given the placebo, [latex]115[/latex] later developed paralytic polio.
Part A: A good experiment always starts with a good research question. What is the research objective or question in this experiment?
Part B: What is the factor of interest in this experiment? What is the response variable in this experiment?
Part C: In an experiment, subjects are often referred to as experimental units. This name emphasizes that the objects measured do not have to be human beings. They could, for example, be government buildings, department stores, animals, or computer chips. When people are the experimental units, we call them subjects, participants, or patients, depending upon the context of the study. What are the experimental units in this experiment?
Part D: What are the treatments or levels of the factor of interest?
An experiment will compare the recorded values of the response variable across multiple levels of the factor of interest. Often this includes the treatment of interest, but it should also have a second baseline level treatment for comparison to help with analyzing the effectiveness. The experimental units are split into two groups—one group receives the treatment of interest (this is usually called the experimental group) and the other group does not. The group that does not receive the treatment of interest is the control group.
Part E: Identify the experimental units that are in the experimental group and the experimental units that are in the control group.
A second method of defining the control group is through the use of a placebo. The placebo effect is a positive response that people who believe they are receiving treatment for a condition have, even if what they are actually receiving is a placebo. It is a fascinating connection between mind and body that is still not completely understood. The placebo effect can make it difficult to gauge the effects on the response variable.
Part F: Why is it important to use a placebo in this study?
In medicine, the randomized experiment (clinical trial) has become the gold standard for the evaluation of new medical treatments because of its effectiveness and ability to objectively support conclusions. The Cochrane Collaboration is an organization devoted to synthesizing evidence from medical studies all over the world. According to this organization, there have been hundreds of thousands of randomized experiments comparing medical treatments. In most countries, a company can’t get a new drug approved for sale unless it has been tested in a well-designed experiment. A good experiment will randomly assign the experimental units to the treatments.
Part G: Why is random assignment important?
In an experiment, it is important that each group be treated the same way. It is also important that the subjects do not adjust their behavior because of the treatment they are receiving. For this reason, many experiments use a technique called blinding. Blinding refers to nondisclosure of the treatment an experimental unit is receiving. Whoever has contact with the subjects during the experiment, including the data collectors who record the experimental units’ response outcomes, should be blind to the treatment information. When neither the subject nor those having contact with the subject know the treatment assignment, the study is called double-blind.
Part H: Why do you think blinding is important? Would it be more ideal for a study to be a blind or double-blind study? Was blinding or double-blinding used in the experiment involving the Salk vaccine?
Part I: Replication of studies increases confidence in the conclusion. Was replication achieved in the experiment involving the Salk vaccine? Explain.
Now, compile a list of the key components that you identified in Question 1. Use them to answer Question 2.
question 2
In Question 1, you identified the key components that make up a well-designed experiment in Parts C, D, E, G, H, and I. Review them and use them to answer Part A below.
Part A: List the components that should be included in a well-designed experiment.
Part B: Do you believe the experiment in Question 1 involving the Salk vaccine is an experiment that was designed well? Explain.
Part C: Would you trust any conclusions drawn from this experiment or would you be skeptical? Explain.
In Question 3, you’ll fill in the blanks to describe how many participants were assigned into each group and what was compared at the end of the experiment so that conclusions might be drawn.
question 3
3) Complete the diagram below to illustrate the design of the Salk vaccine experiment.
The Salk vaccine experiment was one of the largest health experiments ever conducted. It was an experiment because subjects were given treatments, but it’s not always possible to conduct experiments. Ethical issues, cost, time, and other considerations sometimes prohibit the use of an experiment. For example, it would be unethical to conduct an experiment on the effects of texting while driving with two treatment groups: those who text while driving and those who don’t. We would need to request half the subjects to engage in a behavior that we already suspect might cause injury or fatality. In this case, an observational study would be more appropriate, in which previously recorded vehicle collision data is observed to understand the effects of texting while driving.
The Salk Vaccine experiment used a completely randomized design because the experimental units (subjects) were randomly assigned to one of two (or more) treatments. Another type of experimental design called block design, or blocking, can be added to a completely randomized design to strengthen the results. Let’s explore this type of design now.
