Forming Connections in 2E: Advanced Experimental Design

Objectives for this activity

During this activity, you will:

  • Describe the role that blocking plays in an experiment.
  • Identify the basic principles of the completely randomized block design.
  • Describe how to use a completely randomized block design for a given research objective.

You’ve seen experiments and observational studies and compared their advantages and disadvantages in use as tools for research. In the preview What to Know page for this activity, you learned what completely randomized design and blocking are in experiments. In this activity, you’ll add to your understanding of the key components necessary for a well-designed experiment by learning how to design an experiment using the completely randomized block design. Along the way, you’ll understand the purpose of blocking and see that a well-designed experiment helps to determine whether the appropriate conclusion is made.

 Advanced Experimental Design

In the last few activities, you have been exploring key components of well-designed experiments. Another key component of well-designed experiments is randomization.

*image*

Let’s revisit the polio experiment from Preview Assignment 2.E. Recall that in the 1950s, a large-scale experiment was designed to test the effectiveness of the Salk vaccine in preventing polio. Children were assigned to either the experimental or the control group through random assignment. Suppose that the experimental group of 200,745 children received Salk vaccine injections, while a control group of 201,229 children were injected with a placebo. None of the children knew whether or not they had received the vaccine or the placebo. Among the children given the Salk vaccine, 33 later developed paralytic polio, but among the children given a placebo, 115 later developed paralytic polio.

Think individually about the role that randomization played in this experiment, then share your thoughts with a partner and answer Question 1.

question 1

What role did randomization play in this experiment?

Guidance

[Intro: The basic principles of the completely randomized block design are blocking and randomization.  Randomization allows the researcher to create comparable groups–that is, groups that have similar distributions of the other factors. In other words, the researcher is trying to ensure that the factors in one group are similar to the factors in the other group. Imagine that the health conditions of a subject in this experiment could play a role in how subjects respond to a treatment (in this case, the Salk vaccine injection or the placebo injection). Random assignment helps to ensure that one group won’t be much “healthier” than the other group. In this activity, you’ll explore the role that blocking plays in an experiment and use it to describe how completely randomized block design can be used for a given research objective.  ]

Work together to complete the diagram in Question 2. Some questions to ask as you do so include:

  • How do we determine which experimental units form the blocks?
  • What is the purpose of blocking?
  • How are the experimental units assigned to the experimental treatments?
  • Is the purpose of a completely randomized block design to determine whether differences between blocks result in any differences in the value of the response factor?

Question 2

Another way a researcher might conduct an experiment is by using a completely randomized block design. This design is used when the experimental units are divided into homogeneous groups called blocks. Within each block, the experimental units are randomly assigned to treatments. Complete the following diagram to illustrate this design.

A flowchart. On the far left is "Experimental Units," which has two arrows leading from it to boxes labeled "Block 1" and "Block 2," respectively. "Block 1" has an arrow leading to a box labeled "Experimental Group" and another to a box labeled "Control Group." "Experimental Group" and "Control Group" each have an arrow leading from them to the same blank box. From "Block 2," there is an arrow leading to a different box labeled "Experimental Group" and another box labeled "Treatment 2" Both of these boxes have an arrow leading to a different blank box. The two blank boxes have a two-way arrow between them.

Guidance

[Sub-summary: How did you do with Question 2? It can be difficult sometimes to understand what the question is asking you to do. Just keep in mind that the idea behind completely randomized block design is that blocking holds nuisance factors constant while the factor of interest is allowed to vary. For example, if it is known that the hearts of men and women respond differently to medicine, an experiment that tests the effectiveness of aspirin on heart disease might create a block of men and a block of women to hold the factor of gender constant across the treatment and placebo groups. The key to using completely randomized block design is to first create the blocks, then randomly assign participants within each block to the treatment groups. ]

Work in small groups to answer Questions 3 – 6.

question 3

A school psychologist wants to test the effectiveness of a new method for teaching reading. She recruits 80 first-grade students in District Code 75104 and randomly divides them into two groups. Group 1 is taught by means of the new method, while Group 2 is taught by the traditional method. The same teacher is assigned to teach both groups. At the end of the year, an achievement test is administered and the results are compared.

 

Part A: What is the factor of interest? What is the response variable?

 

Part B: What are some factors, other than the factor of interest, in the study that could affect test scores? How are they controlled?

 

Part C: What are the treatment groups?

 

Part D: What are the experimental units (or subjects)?

 

Part E: Are there any factors that cannot be controlled that might affect the test results? How can we minimize the effect that this variable may have on the outcome?

 

Part F: Previous research indicates that girls and boys may react differently to the two methods of instruction (there are an even number of girls and boys).  How can you design an experiment that will control the effects due to gender on the scores on the achievement test (the response factor)? Complete the following template to illustrate your design.A flowchart. On the far left is "Experimental Units," which has two arrows leading from it to boxes labeled "Block 1" and "Block 2," respectively. "Block 1" has an arrow leading to a box labeled "Experimental Group" and another to a box labeled "Control Group." "Experimental Group" and "Control Group" each have an arrow leading from them to the same blank box. From "Block 2," there is an arrow leading to a different box labeled "Experimental Group" and another box labeled "Treatment 2" Both of these boxes have an arrow leading to a different blank box. The two blank boxes have a two-way arrow between them.

Guidance

[Sub-summary: How are you doing so far? Are you starting to develop a mental landscape for completely randomized block design? If so, try answering these questions in your notes:

  • First, recall the key components of good experimental design.

  • What are the basic principles of the completely randomized block design?

  • What role does each principle play in the design?

Continue to use this scenario first given in Question 3 to answer Questions 4 – 6.

A school psychologist wants to test the effectiveness of a new method for teaching reading. She recruits 80 first-grade students in District Code 75104 and randomly divides them into two groups. Group 1 is taught by means of the new method, while Group 2 is taught by the traditional method. The same teacher is assigned to teach both groups. At the end of the year, an achievement test is administered and the results are compared.

question 4

What are some advantages of using the completely randomized block design for the education researcher in the scenario given in Question 3?

question 5

Again, referring to the scenario given in Question 3, would you consider this experiment to be well-designed?

question 6

Would you trust the results from the conclusions of this experiment or would you be skeptical?

Guidance

[Wrap-up: The key components of a well-designed experiment include treatment, factor of interest (also known as the explanatory variable or independent variable), response variable (also known as the dependent variable), nuisance factors, random assignment, and replication.

The completely randomized block design is an additional method we can use to control for a known difference that affects the response factor.

The basic principles of a completely randomized block design are blocking and randomization.

Blocking helps to minimize the effects of nuisance factors. Randomization is used to assign subjects within the blocks to the treatment groups, which helps ensure that the groups have similar distributions of factors other than the factor of interest.

We can trust the conclusions of a well-designed experiment. The results of a well-designed and implemented study are meaningful. ]