What to Know About 2E — (don’t use this one)

learning GOALS

At the end of this page, you should feel comfortable performing these skills:

  • Identify the key components of an experiment.
  • Determine whether or not an experiment has been designed well.
  • Use a diagram to illustrate the design of a given experiment.
  • Determine factors, other than the factor of interest, that could have an effect on the response variable of a given experiment (nuisance factors).
  • Identify blocks in a specified experiment.

In the next activity, you will need to identify blinding, placebos, blocks, and nuisance factors as components of good and completely randomized block designs. A randomized block design is a type of experimental design that places similar participants into groups called blocks. The individuals in each block are then randomly assigned to treatments. We’ll learn that, in addition to control and experimental groups, a placebo can be assigned as a treatment.

Randomized Design

In previous activities, you learned that the design of an experiment can have a major impact on its results. If an experiment is not designed well, its results may be meaningless and misleading. Let’s review the key components of an experiment, along with the role that each component plays in its design. This will help you to better understand how the design of experimental studies can affect how we infer conclusions from analyses of data.

Recall that researchers conduct experiments by assigning subjects to certain experimental conditions or treatments and then observing outcomes on the response variable. A placebo is one of the treatments. It is a harmless version of the treatment that does not contain any active ingredients (e.g., a sugar pill). It will typically look, taste, and smell like the treatment of interest or mimic it so that the two treatments appear identical to the subjects; this way the subjects don’t know which treatment they are receiving.

In Questions 1 – 3, you’ll consider a famously important study as your refresh the key components of experimental design.

The Salk Vaccine Trials

In the 1950’s, a large-scale experiment was designed to test the effectiveness of the Salk vaccine in preventing polio, which had killed or paralyzed thousands of children. Children were assigned to either the experimental or control group through a process of random assignment. Suppose that 200,745 children were given a treatment consisting of Salk vaccine injections, while a control group of 201,229 children were injected with a placebo that contained no drug. The children being injected did not know whether they were getting the Salk vaccine or the placebo. Among the children given the Salk vaccine, 33 later developed paralytic polio, but among the children given the placebo, 115 later developed paralytic polio.

Before you begin Question 1, assess your understanding of each of the following terms. You saw these in 2C: Experimental Design. Refer to your notes from that section as necessary.

  • Factor of Interest
  • Response Variable
  • Treatments
  • Random Assignment

You’ll also see some new terminology, including the following. Watch for these and make a note of their definitions, which will be given as the terms appear in Question 1

  • Experimental Units (participants or subjects)
  • Placebo, and the Placebo Effect
  • Blinding
  • Double-Blind

Question 1

 

Part A: A good experiment always starts with a good research question. What is the research objective or question in this experiment?

 

Part B: What is the factor of interest in this experiment? What is the response variable in this experiment?

 

Part C: In an experiment, subjects are often referred to as experimental units. This name emphasizes that the objects measured do not have to be human beings. They could, for example, be government buildings, department stores, animals, or computer chips. What are the experimental units in this experiment?

 

Part D: What are the treatments or levels of the factor of interest?

 

An experiment will compare the recorded values of the response variable across multiple levels of the factor of interest. Often this includes the treatment of interest, but it should also have a second baseline level treatment for comparison to help with analyzing the effectiveness. The experimental units are split into two groups—one group receives the treatment of interest (this is usually called the experimental group) and the other group does not. The group that does not receive the treatment of interest is the control group.

 

Part E: Identify the experimental units that are in the experimental group and the experimental units that are in the control group.

 

A second method of defining the control group is through the use of a placebo. The placebo effect is a positive response that people who believe they are receiving treatment for a condition have, even if what they are actually receiving is a placebo. It is a fascinating connection between mind and body that is still not completely understood. The placebo effect can make it difficult to gauge the effects on the response variable.

 

Part F: Why is it important to use a placebo in this study?

 

In medicine, the randomized experiment (clinical trial) has become the gold standard for the evaluation of new medical treatments because of its effectiveness and ability to objectively support conclusions. The Cochrane Collaboration is an organization devoted to synthesizing evidence from medical studies all over the world. According to this organization, there have been hundreds of thousands of randomized experiments comparing medical treatments. In most countries, a company can’t get a new drug approved for sale unless it has been tested in a well-designed experiment. A good experiment will randomly assign the experimental units to the treatments.

 

Part G: Why is random assignment important?

In an experiment, it is important that each group be treated the same way. It is also important that the subjects do not adjust their behavior because of the treatment they are receiving. For this reason, many experiments use a technique called blinding. Blinding refers to nondisclosure of the treatment an experimental unit is receiving. Whoever has contact with the subjects during the experiment, including the data collectors who record the experimental units’ response outcomes, should be blind to the treatment information. When neither the subject nor those having contact with the subject know the treatment assignment, the study is called double-blind.

 

Part H: Why do you think blinding is important? Would it be more ideal for a study to be a blind or double-blind study? Was blinding or double-blinding used in the experiment involving the Salk vaccine?

 

Part I: Replication of studies increases confidence in the conclusion. Was replication achieved in the experiment involving the Salk vaccine? Explain.

question 2

In Question 1, you identified the key components that make up a well-designed experiment in Parts C, D, E, G, H, and I. Review them and use them to answer Part A below.

 

Part A: List the components that should be included in a well-designed experiment.

 

Part B: Do you believe the experiment in Question 1 involving the Salk vaccine is an experiment that was designed well? Explain.

 

Part C: Would you trust any conclusions drawn from this experiment or would you be skeptical? Explain.

question 3

3) Complete the diagram below to illustrate the design of the Salk vaccine experiment.

A flowchart. The first box reads "Experimental units: 401,974 subjects." It has two arrows coming out of it, both arrows are labeled "randomly assigned." One of the arrows goes to a box with text reading "Experimental Group" and the other to another box reading "Control Group." Both of these have arrows leading to another box that has nothing written in it.

The Salk vaccine experiment was one of the largest health experiments ever conducted. It was an experiment because subjects were given treatments, but ethical issues, cost, time, and other considerations sometimes prohibit the use of an experiment. For example, it would be unethical to conduct a texting-while-driving experiment in which we ask subjects to text while driving because of the risk of injury or fatality. An observational study would be more appropriate in which previously recorded vehicle collision data could be observed to understand the effects of texting while driving.

The experiment described in Question 1 used a completely randomized design because the experimental units (subjects) were randomly assigned to one of the treatments. Another type of experimental design called block design (blocking) can be added to a completely randomized design to strengthen the results. Let’s explore this type of design now.

[above 2 paragraphs rewritten from the original]

Completely Randomized Block Design

In a completely randomized block design, the experimental units (participants) are divided into homogenous (similar) groups called blocks. The blocks help to minimize the potential effects of a nuisance factor–a factor that can cause variation in the response factor but also that the researcher is not interested it. Nuisance factors that are well known and often controlled for include environmental factors like temperature, light, time of day, or physiological factors in living subjects like weight, sex, or age.

To use a block design, the researcher will chose a factor to control, such as age. All subjects exhibiting a similar characteristic of this factor, say they are close in age, are grouped together in similar sized blocks. Then they are randomly assigned to the treatment groups. This will ensure that any variability due to the nuisance factor is evenly distributed across the treatments, and any variability in the response variable attributed to the nuisance factor is isolated and minimized.

To summarize, in a completely randomized block design, the experimental units are first grouped into homogenous (similar) blocks, then randomly assigned to treatments within each block.

See the video below for a demonstration of how blocking works, then answer Question 4.

[I rewrote this section and replaced all the text below with the above]

Recall that in an experiment we want to know what effects the factor of interest has on the response variable. Sometimes a nuisance factor may also affect the response factor, even though we are not interested in this factor (e.g., the specific operator who prepared the treatment, the time of day the experiment was run, the room temperature). All experiments have nuisance factors, so the experimenter will typically need to spend some time deciding which nuisance factors are important enough to keep track of or control.

Sometimes the nuisance factors can be directly controlled in the experiment using a completely randomized block design. This design is used when the experimental units are divided into homogeneous groups called blocks. A block is a group of subjects that are similar, but blocks differ in ways that might affect the outcome of the experiment. For example, if your nuisance factor is known and controllable, it can be added to your experimental design. We use the term blocking to describe the grouping together of homogeneous (similar) experimental units, followed by the random assignment of the experimental units within each group to a treatment.

In a completely randomized block design, we do not wish to determine whether the differences between blocks result in any difference in the value of the response variable. Our goal is to remove any variability in the response variable that may be attributable to the block. Therefore, the advantage of this design is that blocking will help to minimize the effects of nuisance factors.

The basic principles of the completely randomized block design are blocking and randomization. The general rule is to “block what you can and randomize what you cannot.” So, blocking is used to remove the effects of a few of the most important nuisance factors. Randomization is then used to create comparable groups to reduce contaminating effects from the removal of the nuisance factors.

Video Placement

[Perspective Video: a 3-instructor video that explains/demonstrates how to use blocking to control a nuisance factor (i.e., how blocking minimizes the effects of nuisance factors — how random assignment within blocks ensures the effect of a nuisance factor is more equitably distributed across the treatments — how it helps to isolate the factor of interest)

Now you try answering Question 4, about nuisance factors.

question 4

Consider the Salk vaccine experiment again. Suppose there was previous research to suggest that the male children in the experiment would respond more favorably to the Salk vaccine injections than the female children.

Part A: Would gender be considered a nuisance factor? Explain.

Part B: What are some other possible factors in the experiment, other than the factor of interest, that would be considered nuisance factors?

Video Placement

[worked example Video: a 3-instructor worked example in the style of Question 5, demonstrating how to identify if an experiment has used completely randomized block design.]

Now you try it. Assess your understanding of blocking by answering Question 5.

question 5

A marketing research firm wants to determine the most effective method of advertising: print, radio, or television. They recruit 300 volunteers in the study. The chief researcher believes that level of education plays a role in the effectiveness of advertising, so she segments the volunteers by level of education. Of the 300 volunteers, 120 have a high school education, 120 have college degrees, and 60 have advanced degrees. The 120 volunteers with high school diplomas are randomly assigned to either the print advertising group, the radio group, or the television group. The same procedure is followed for the college graduates and advanced degree volunteers. Each group is exposed to the advertising. After one hour, a recall exam is given and the number of correct answers is recorded.

 

Part A: Identify the factor of interest.

 

Part B: Does this experiment have blocks?

  1. a) No, there are no blocks within the experiment.
  2. b) Yes, the method of advertising and the level of education both serve as the block.
  3. c) Yes, the method of advertising serves as the block.
  4. d) Yes, the level of education serves as the block.

 

Part C: Can a completely randomized block design be used to help the marketing firm to determine the most effective method of advertising?

  1. a) No, because there are no blocks within this design.
  2. b) Yes, because the treatments are randomly assigned to the experimental units within each block.
  3. c) No, because there needs to be more than two treatments.
  4. d) Yes, because the treatments are not randomly assigned to the units within each block.

Summary

In this What to Know page, you learned some new terminology and explored a new experiment type: completely randomized block design. You also learned about the placebo effect and nuisance factors.  Let’s summarize where in the text these new terms and design appeared.

  • In Question 1, you identified the key components of an experiment.
  • In Question 2, you determined whether or not an experiment has been designed well.
  • In Question 3, you used a diagram to explain the design of a given experiment.
  • In Question 4, you determined factors other than the explanatory variable that could have an effect on the response variable of a given experiment (nuisance factors)
  • In Question 5, you identified blocks in a specified experiment.

If you feel comfortable with these ideas, it’s time to move on to Forming Connections.