{"id":800,"date":"2017-06-13T23:16:13","date_gmt":"2017-06-13T23:16:13","guid":{"rendered":"https:\/\/courses.lumenlearning.com\/suny-hccc-ma-124-1\/?post_type=chapter&#038;p=800"},"modified":"2018-08-14T20:31:11","modified_gmt":"2018-08-14T20:31:11","slug":"sampling-and-experimentation","status":"publish","type":"chapter","link":"https:\/\/courses.lumenlearning.com\/suny-hccc-ma-124-1\/chapter\/sampling-and-experimentation\/","title":{"raw":"Sampling and Experimentation","rendered":"Sampling and Experimentation"},"content":{"raw":"As we mentioned previously, the first thing we should do before conducting a survey is to identify the population that we want to study. In this lesson, we will show you examples of how to identify the population in a study, and determine whether or not the study actually represents the intended population. We will discuss different techniques\u00a0for <em>random sampling<\/em> that are intended to ensure a population is well represented in a sample.\r\n\r\nWe will also identify the difference between an observational study and an experiment, and ways experiments can be conducted. By the end of this lesson, we hope that you will also be confident in identifying when an experiment may have\u00a0been affected by confounding or the placebo effect, and the methods that are employed to avoid them.\r\n<div class=\"textbox learning-objectives\">\r\n<h3>Learning Objectives<\/h3>\r\n<ul>\r\n \t<li>Identify\u00a0methods for obtaining a random sample of the intended population of a study<\/li>\r\n \t<li>Identify ineffective ways of obtaining a random sample from a population<\/li>\r\n \t<li>Identify types of sample bias<\/li>\r\n \t<li>Identify the differences between observational study and an experiment<\/li>\r\n \t<li>Identify the treatment in an experiment<\/li>\r\n \t<li>Determine whether an experiment may have been influenced by confounding<\/li>\r\n<\/ul>\r\n<\/div>\r\n<h2>Sampling Methods and Bias<\/h2>\r\n<h3>Selecting a Population<\/h3>\r\nSuppose we are hired by a politician to determine the amount of support he has among the electorate should he decide to run for another term.\u00a0 What population should we study? Every person in the district? Not every person is eligible to vote, and regardless of how strongly someone likes or dislikes the candidate, they don't have much to do with him being re-elected if they are not able to vote.\r\n\r\nWhat about eligible voters in the district? That might be better, but if someone is eligible to vote but does not\u00a0register by the deadline, they won't have any say in the election either. What about registered voters? Many people are registered but choose not to vote. What about \"likely voters?\"\r\n\r\nThis is the criteria used in much political polling, but it is sometimes difficult to define a \"likely voter.\" Is it someone who voted in the last election? In the last general election? In the last presidential election? Should we consider someone who just turned 18 a \"likely voter?\" They weren't eligible to vote in the past, so how do we judge the likelihood that they will vote in the next election?\r\n\r\n<a href=\"https:\/\/s3-us-west-2.amazonaws.com\/courses-images\/wp-content\/uploads\/sites\/1141\/2016\/12\/29220533\/crowd-1699137_640.jpg\"><img class=\"aligncenter size-full wp-image-1026\" src=\"https:\/\/s3-us-west-2.amazonaws.com\/courses-images\/wp-content\/uploads\/sites\/1141\/2016\/12\/29220533\/crowd-1699137_640.jpg\" alt=\"rows of Lego figurines\" width=\"640\" height=\"360\" \/><\/a>\r\n\r\n&nbsp;\r\n\r\nIn November 1998, former professional wrestler Jesse \"The Body\" Ventura was elected governor of Minnesota. Up until right before the election, most polls showed he had little chance of winning. There were several contributing factors to the polls not reflecting the actual intent of the electorate:\r\n<ul>\r\n \t<li>Ventura was running on a third-party ticket and most polling methods are better suited to a two-candidate race.<\/li>\r\n \t<li>Many respondents to polls may have been embarrassed to tell pollsters that they were planning to vote for a professional wrestler.<\/li>\r\n \t<li>The mere fact that the polls showed Ventura had little chance of winning might have prompted some people to vote for him in protest to send a message to the major-party candidates.<\/li>\r\n<\/ul>\r\nBut one of the major contributing factors was that Ventura recruited a substantial amount of support from young people, particularly college students, who had never voted before and who registered specifically to vote in the gubernatorial election. The polls did not deem these young people likely voters (since in most cases young people have a lower rate of voter registration and a turnout rate for elections) and so the polling samples were subject to <strong>sampling bias<\/strong>: they omitted a portion of the electorate that was weighted in favor of the winning candidate.\r\n<div>\r\n<div class=\"textbox\">\r\n<div>\r\n<h3>Sampling bias<\/h3>\r\n<\/div>\r\n<div>\r\n\r\nA sampling method is biased if every member of the population doesn\u2019t have equal likelihood of being in the sample.\r\n\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\nSo even identifying the population can be a difficult job, but once we have identified the population, how do we choose an appropriate sample? Remember, although we would prefer to survey all members of the population, this is usually impractical unless the population is very small, so we choose a sample. There are many ways to sample a population, but there is one goal we need to keep in mind: we would like the sample to be <em>representative of the population<\/em>.\r\n\r\nReturning to our hypothetical job as a political pollster, we would not anticipate very accurate results if we drew all of our samples from among the customers at a Starbucks, nor would we expect that a sample drawn entirely from the membership list of the local Elks club would provide a useful picture of district-wide support for our candidate.\r\n\r\nOne way to ensure that the sample has a reasonable chance of mirroring the population is to employ <em>randomness<\/em>. The most basic random method is simple random sampling.\r\n<div>\r\n<div class=\"textbox\">\r\n<div>\r\n<h3>Simple random sample<\/h3>\r\n<\/div>\r\n<div>\r\n\r\nA <strong>random sample<\/strong> is one in which each member of the population has an equal probability of being chosen. A <strong>simple random sample<\/strong> is one in which every member of the population and any group of members has an equal probability of being chosen.\r\n\r\n<\/div>\r\n<\/div>\r\n<div class=\"textbox exercises\">\r\n<h3>example<\/h3>\r\n<div>\r\n\r\nIf we could somehow identify all likely voters in the state, put each of their names on a piece of paper, toss the slips into a (very large) hat and draw 1000 slips out of the hat, we would have a simple random sample.\r\n\r\n<\/div>\r\nIn practice, computers are better suited for this sort of endeavor than millions of slips of paper and extremely large headgear.\r\n\r\n<\/div>\r\n<\/div>\r\nIt is always possible, however, that even a random sample might end up not being totally representative of the population.\u00a0 If we repeatedly take samples of 1000 people from among the population of likely voters in the state of Washington, some of these samples might tend to have a slightly higher percentage of Democrats (or Republicans) than does the general population; some samples might include more older people and some samples might include more younger people; etc. In most cases, this <strong>sampling variability<\/strong> is not significant.\r\n<div>\r\n<div class=\"textbox\">\r\n<div>\r\n<h3>Sampling variability<\/h3>\r\n<\/div>\r\n<div>\r\n\r\nThe natural variation of samples is called <strong>sampling variability<\/strong>.\r\n\r\nThis is unavoidable and expected in random sampling, and in most cases is not an issue.\r\n\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\nTo help account for variability, pollsters might instead use a <strong>stratified sample<\/strong>.\r\n<div>\r\n<div class=\"textbox\">\r\n<div>\r\n<h3>Stratified sampling<\/h3>\r\n<\/div>\r\n<div>\r\n\r\nIn <strong>stratified sampling<\/strong>, a population is divided into a number of subgroups (or strata). Random samples are then taken from each subgroup with sample sizes proportional to the size of the subgroup in the population.\r\n\r\n<\/div>\r\n<\/div>\r\n<div class=\"textbox exercises\">\r\n<h3>example<\/h3>\r\n<div>\r\n\r\nSuppose in a particular state that previous data indicated that the electorate was comprised of 39% Democrats, 37% Republicans and 24% independents.\u00a0 In a sample of 1000 people, they would then expect to get about 390 Democrats, 370 Republicans and 240 independents.\u00a0 To accomplish this, they could randomly select 390 people from among those voters known to be Democrats, 370 from those known to be Republicans, and 240 from those with no party affiliation.\r\n\r\n<\/div>\r\n<\/div>\r\nStratified sampling can also be used to select a sample with people in desired age groups, a specified mix ratio of males and females, etc. A variation on this technique is called <strong>quota sampling<\/strong>.\r\n\r\n<\/div>\r\n<div>\r\n<div class=\"textbox\">\r\n<div>\r\n<h3>Quota sampling<\/h3>\r\n<\/div>\r\n<div>\r\n\r\n<strong>Quota sampling<\/strong> is a variation on stratified sampling, wherein samples are collected in each subgroup until the desired quota is met.\r\n\r\n<\/div>\r\n<\/div>\r\n<div class=\"textbox exercises\">\r\n<h3>example<\/h3>\r\n<div>\r\n\r\nSuppose the pollsters call people at random, but once they have met their quota of 390 Democrats, they only gather people who do not identify themselves as a Democrat.\r\n\r\n<\/div>\r\nYou may have had the experience of being called by a telephone pollster who started by asking you your age, income, etc. and then thanked you for your time and hung up before asking any \"real\" questions. Most likely, they already had contacted enough people in your demographic group and were looking for people who were older or younger, richer or poorer, etc. Quota sampling is usually a bit easier than stratified sampling, but also does not ensure the same level of randomness.\r\n\r\n<\/div>\r\n<\/div>\r\n<div>\r\n\r\nAnother sampling method is <strong>cluster sampling<\/strong>, in which the population is divided into groups, and one or more groups are randomly selected to be in the sample.\r\n\r\n<\/div>\r\n<div>\r\n<div class=\"textbox\">\r\n<div>\r\n<h3>Cluster sampling<\/h3>\r\n<\/div>\r\n<div>\r\n\r\nIn <strong>cluster sampling<\/strong>, the population is divided into subgroups (clusters), and a set of subgroups are selected to be in the sample.\r\n\r\n<\/div>\r\n<\/div>\r\n<div class=\"textbox exercises\">\r\n<h3>example<\/h3>\r\nIf the college wanted to survey students, since students are already divided into classes, they could randomly select 10 classes and give the survey to all the students in those classes. This would be cluster sampling.\r\n\r\n<\/div>\r\n<\/div>\r\nOther sampling methods include <strong>systematic sampling<\/strong>.\r\n<div>\r\n<div class=\"textbox\">\r\n<div>\r\n<h3>Systematic sampling<\/h3>\r\n<\/div>\r\n<div>\r\n\r\nIn <strong>systematic sampling<\/strong>, every <em>n<sup>th<\/sup><\/em> member of the population is selected to be in the sample.\r\n\r\n<\/div>\r\n<\/div>\r\n<div class=\"textbox exercises\">\r\n<h3>example<\/h3>\r\nTo select a sample using systematic sampling, a pollster calls every 100th name in the phone book.\r\n\r\nSystematic sampling is not as random as a simple random sample (if your name is Albert Aardvark and your sister Alexis Aardvark is right after you in the phone book, there is no way you could both end up in the sample) but it can yield acceptable samples.\r\n\r\n<\/div>\r\n<\/div>\r\n<h3>The Worst Way to Sample<\/h3>\r\nPerhaps the worst types of sampling methods are <strong>convenience samples<\/strong> and <strong>voluntary response samples<\/strong>.\r\n<div>\r\n<div class=\"textbox\">\r\n<div>\r\n<h3>Convenience sampling and voluntary response sampling<\/h3>\r\n<\/div>\r\n<div>\r\n\r\n<strong>Convenience sampling<\/strong> is the practice of samples chosen by selecting whoever is convenient.\r\n\r\n<strong>Voluntary response sampling<\/strong> is allowing the sample to volunteer.\r\n\r\n<\/div>\r\n<\/div>\r\n<div class=\"textbox exercises\">\r\n<h3>example<\/h3>\r\nA pollster stands on a street corner and interviews the first 100 people who agree to speak to him. Which sampling method is represented by this scenario?\r\n[reveal-answer q=\"386299\"]Show Answer[\/reveal-answer]\r\n[hidden-answer a=\"386299\"]This is a convenience sample.[\/hidden-answer]\r\n<div>\r\n\r\n<hr \/>\r\n\r\nA website has a survey asking readers to give their opinion on a tax proposal. Which sampling method is represented?\r\n[reveal-answer q=\"503402\"]Show Answer[\/reveal-answer]\r\n[hidden-answer a=\"503402\"]\r\n<div>\r\n\r\nThis is a self-selected sample, or voluntary response sample, in which respondents volunteer to participate.\r\n\r\n<\/div>\r\nUsually voluntary response samples are skewed towards people who have a particularly strong opinion about the subject of the survey or who just have way too much time on their hands and enjoy taking surveys.\r\n\r\n[\/hidden-answer]\r\n\r\n<\/div>\r\n<\/div>\r\nWatch the following video for an overview of all the sampling methods discussed so far.\r\n\r\nhttps:\/\/youtu.be\/sk4TDJU7QbY\r\n<div class=\"textbox key-takeaways\">\r\n<h3>Try It Now<\/h3>\r\nIn each case, indicate what sampling method was used\r\n\r\na. Every 4th person in the class was selected\r\n\r\nb. A sample was selected to contain 25 men and 35 women\r\n\r\nc. Viewers of a new show are asked to vote on the show\u2019s website\r\n\r\nd. A website randomly selects 50 of their customers to send a satisfaction survey to\r\n\r\ne. To survey voters in a town, a polling company randomly selects 10 city blocks, and interviews everyone who lives on those blocks.\r\n\r\n<iframe id=\"mom2\" class=\"resizable\" src=\"https:\/\/www.myopenmath.com\/multiembedq.php?id=6733&amp;theme=oea&amp;iframe_resize_id=mom2\" width=\"100%\" height=\"350\"><\/iframe>\r\n\r\n<\/div>\r\n<h3>Problematic Sampling and Surveying<\/h3>\r\nThere are number of ways that a study can be ruined before you even start collecting data. The first we have already explored \u2013 <strong>sampling <\/strong>or<strong> selection bias<\/strong>, which is when the sample is not representative of the population. One example of this is <strong>voluntary response bias<\/strong>, which is bias introduced by only collecting data from those who volunteer to participate. This is not the only potential source of bias.\r\n<div class=\"textbox\">\r\n<h3>Sources of bias<\/h3>\r\n<ul>\r\n \t<li><strong>Sampling bias<\/strong> \u2013 when the sample is not representative of the population<\/li>\r\n \t<li><strong>Voluntary response bias<\/strong> \u2013 the sampling bias that often occurs when the sample is volunteers<\/li>\r\n \t<li><strong>Self-interest study<\/strong> \u2013 bias that can occur when the researchers have an interest in the outcome<\/li>\r\n \t<li><strong>Response bias<\/strong> \u2013 when the responder gives inaccurate responses for any reason<\/li>\r\n \t<li><strong>Perceived lack of anonymity<\/strong> \u2013 when the responder fears giving an honest answer might negatively affect them<\/li>\r\n \t<li><strong>Loaded questions<\/strong> \u2013 when the question wording influences the responses<\/li>\r\n \t<li><strong>Non-response bias<\/strong> \u2013 when people refusing to participate in the study can influence the validity of the outcome<\/li>\r\n<\/ul>\r\n<\/div>\r\n<\/div>\r\n<div class=\"textbox exercises\">\r\n<h3>examples<\/h3>\r\nConsider a recent study which found that chewing gum may raise math grades in teenagers[footnote]Reuters. <a href=\"http:\/\/news.yahoo.com\/s\/nm\/20090423\/od_uk_nm\/oukoe_uk_gum_learning\" target=\"_blank\">http:\/\/news.yahoo.com\/s\/nm\/20090423\/od_uk_nm\/oukoe_uk_gum_learning<\/a>. Retrieved 4\/27\/09[\/footnote]. This study was conducted by the Wrigley Science Institute, a branch of the Wrigley chewing gum company. Identify the type of sampling bias found in this example.\r\n[reveal-answer q=\"561139\"]Show Answer[\/reveal-answer]\r\n[hidden-answer a=\"561139\"]This is an example of a <strong>self-interest study<\/strong>; one in which the researchers have a vested interest in the outcome of the study. While this does not necessarily ensure that the study was biased, it certainly suggests that we should subject the study to extra scrutiny.[\/hidden-answer]\r\n<div>\r\n\r\n<hr \/>\r\n\r\nA survey asks people \u201cwhen was the last time you visited your doctor?\u201d What type of sampling bias might this lead to?\r\n[reveal-answer q=\"965739\"]Show Answer[\/reveal-answer]\r\n[hidden-answer a=\"965739\"]\r\n<div>\r\n\r\nThis might suffer from <strong>response bias<\/strong>, since many people might not remember exactly when they last saw a doctor and give inaccurate responses.\r\n\r\n<\/div>\r\nSources of response bias may be innocent, such as bad memory, or as intentional as pressuring by the pollster. Consider, for example, how many voting initiative petitions people sign without even reading them.\r\n\r\n[\/hidden-answer]\r\n\r\n<\/div>\r\n\r\n<hr \/>\r\n\r\nA survey asks participants a question about their interactions with members of other races. Which sampling bias might occur for this survey strategy?\r\n[reveal-answer q=\"610809\"]Show Answer[\/reveal-answer]\r\n[hidden-answer a=\"610809\"]Here, a <strong>perceived lack of anonymity<\/strong> could influence the outcome. The respondent might not want to be perceived as racist even if they are, and give an untruthful answer.[\/hidden-answer]\r\n\r\n<hr \/>\r\n\r\nAn employer puts out a survey asking their employees if they have a drug abuse problem and need treatment help. Which sampling bias may occur in this scenario?\r\n[reveal-answer q=\"801450\"]Show Answer[\/reveal-answer]\r\n[hidden-answer a=\"801450\"]Here, answering truthfully might have consequences; responses might not be accurate if the employees do not feel their responses are anonymous or fear retribution from their employer. This survey has the\u00a0potential for\u00a0<strong>perceived lack of anonymity<\/strong>.[\/hidden-answer]\r\n<div>\r\n\r\n<hr \/>\r\n\r\nA survey asks \u201cdo you support funding research of alternative energy sources to reduce our reliance on high-polluting fossil fuels?\u201d Which sampling bias may result from this survey?\r\n[reveal-answer q=\"464953\"]Show Answer[\/reveal-answer]\r\n[hidden-answer a=\"464953\"]\r\n<div>\r\n\r\nThis is an example of a <strong>loaded <\/strong>or<strong> leading question<\/strong> \u2013 questions whose wording leads the respondent towards an answer.\r\n\r\n<\/div>\r\nLoaded questions can occur intentionally by pollsters with an agenda, or accidentally through poor question wording. Also a concern is <strong>question order<\/strong>, where the order of questions changes the results. A psychology researcher provides an example[footnote]Swartz, Norbert. <a href=\"http:\/\/www.umich.edu\/~newsinfo\/MT\/01\/Fal01\/mt6f01.html\" target=\"_blank\">http:\/\/www.umich.edu\/~newsinfo\/MT\/01\/Fal01\/mt6f01.html<\/a>. Retrieved 3\/31\/2009[\/footnote]:\r\n<blockquote>\u201cMy favorite finding is this: we did a study where we asked students, 'How satisfied are you with your life? How often do you have a date?' The two answers were not statistically related - you would conclude that there is no relationship between dating frequency and life satisfaction. But when we reversed the order and asked, 'How often do you have a date? How satisfied are you with your life?' the statistical relationship was a strong one. You would now conclude that there is nothing as important in a student's life as dating frequency.\u201d<\/blockquote>\r\n[\/hidden-answer]\r\n\r\n<\/div>\r\n\r\n<hr \/>\r\n\r\nA telephone poll asks the question \u201cDo you often have time to relax and read a book?\u201d, and 50% of the people called refused to answer the survey. Which sampling bias is represented by this survey?\r\n[reveal-answer q=\"782411\"]Show Answer[\/reveal-answer]\r\n[hidden-answer a=\"782411\"]It is unlikely that the results will be representative of the entire population. This is an example of <strong>non-response bias<\/strong>, introduced by people refusing to participate in a study or dropping out of an experiment. When people refuse to participate, we can no longer be so certain that our sample is representative of the population.[\/hidden-answer]\r\n\r\n<\/div>\r\nThese problematic scenarios for statistics gathering are discussed further in the following video.\r\n\r\nhttps:\/\/youtu.be\/GXAi06PiVsk\r\n<div class=\"textbox key-takeaways\">\r\n<h3>Try It Now<\/h3>\r\nIn each situation, identify a potential source of bias\r\n\r\na. A survey asks how many sexual partners a person has had in the last year\r\n\r\nb. A radio station asks readers to phone in their choice in a daily poll.\r\n\r\nc. A substitute teacher wants to know how students in the class did on their last test. The teacher asks the 10 students sitting in the front row to state their latest test score.\r\n\r\nd. High school students are asked if they have consumed alcohol in the last two weeks.\r\n\r\ne. The Beef Council releases a study stating that consuming red meat poses little cardiovascular risk.\r\n\r\nf. A poll asks \u201cDo you support a new transportation tax, or would you prefer to see our public transportation system fall apart?\u201d\r\n\r\n<iframe id=\"mom5\" class=\"resizable\" src=\"https:\/\/www.myopenmath.com\/multiembedq.php?id=6727&amp;theme=oea&amp;iframe_resize_id=mom5\" width=\"100%\" height=\"350\"><\/iframe>\r\n\r\n<\/div>\r\n<h2>Experiments<\/h2>\r\n<h3>Observing vs. Acting<\/h3>\r\nSo far, we have primarily discussed <strong>observational studies<\/strong> \u2013 studies in which conclusions would be drawn from observations of a sample or the population. In some cases these observations might be unsolicited, such as studying the percentage of cars that turn right at a red light even when there is a \u201cno turn on red\u201d sign. In other cases the observations are solicited, like in a survey or a poll.\r\n\r\nIn contrast, it is common to use <strong>experiments<\/strong> when exploring how subjects react to an outside influence. In an experiment, some kind of <strong>treatment<\/strong> is applied to the subjects and the results are measured and recorded.\r\n\r\n<a href=\"https:\/\/s3-us-west-2.amazonaws.com\/courses-images\/wp-content\/uploads\/sites\/1141\/2016\/12\/30181148\/lab-217041_640.jpg\"><img class=\"aligncenter size-full wp-image-1033\" src=\"https:\/\/s3-us-west-2.amazonaws.com\/courses-images\/wp-content\/uploads\/sites\/1141\/2016\/12\/30181148\/lab-217041_640.jpg\" alt=\"Collection of clear glass scientific measuring equipment, with clear liquid in them\" width=\"640\" height=\"423\" \/><\/a>\r\n<div>\r\n<div class=\"textbox\">\r\n<div>\r\n<h3>Observational studies and experiments<\/h3>\r\n<\/div>\r\n<div>\r\n<ul>\r\n \t<li>An <strong>observational study<\/strong> is a study based on observations or measurements<\/li>\r\n \t<li>An <strong>experiment<\/strong> is a study in which the effects of a <strong>treatment<\/strong> are measured<\/li>\r\n<\/ul>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<div class=\"textbox exercises\">\r\n<h3>Examples<\/h3>\r\nHere are some examples of experiments:\r\n\r\nA pharmaceutical company tests a new medicine for treating Alzheimer\u2019s disease by administering the drug to 50 elderly patients with recent diagnoses. The treatment here is the new drug.\r\n\r\n<hr \/>\r\n\r\nA gym tests out a new weight loss program by enlisting 30 volunteers to try out the program. The treatment here is the new program.\r\n\r\n<hr \/>\r\n\r\nYou test a new kitchen cleaner by buying a bottle and cleaning your kitchen. The new cleaner is the treatment.\r\n\r\n<hr \/>\r\n\r\nA psychology researcher explores the effect of music on temperament by measuring people\u2019s temperament while listening to different types of music. The music is the treatment.\r\n\r\n<\/div>\r\n&nbsp;\r\n\r\nThese examples are discussed further in the following video.\r\n\r\nhttps:\/\/youtu.be\/HSTTKzsdHEw\r\n<div class=\"textbox key-takeaways\">\r\n<h3>Try It Now<\/h3>\r\nIs each scenario describing an observational study or an experiment?\r\n\r\na. The weights of 30 randomly selected people are measured\r\n\r\nb. Subjects are asked to do 20 jumping jacks, and then their heart rates are measured\r\n\r\nc. Twenty coffee drinkers and twenty tea drinkers are given a concentration test\r\n[reveal-answer q=\"713031\"]Show Answer[\/reveal-answer]\r\n[hidden-answer a=\"713031\"]\r\n\r\na. Observational study\r\n<div>\r\n\r\nb. Experiment; the treatment is the jumping jacks\r\n\r\nc. Experiment; the treatments are coffee and tea\r\n\r\n<\/div>\r\n[\/hidden-answer]\r\n\r\n<iframe id=\"mom1\" class=\"resizable\" src=\"https:\/\/www.myopenmath.com\/multiembedq.php?id=6914&amp;theme=oea&amp;iframe_resize_id=mom1\" width=\"100%\" height=\"250\"><\/iframe>\r\n\r\n<\/div>\r\nWhen conducting experiments, it is essential to isolate the treatment being tested.\r\n<div class=\"textbox exercises\">\r\n<h3>example<\/h3>\r\nSuppose a middle school (junior high) finds that their students are not scoring well on the state\u2019s standardized math test. They decide to run an experiment to see if an alternate curriculum would improve scores. To run the test, they hire a math specialist to come in and teach a class using the new curriculum. To their delight, they see an improvement in test scores.\r\n\r\n<\/div>\r\nThe difficulty with this scenario is that it is not clear whether the curriculum is responsible for the improvement, or whether the improvement is due to a math specialist teaching the class. This is called <strong>confounding<\/strong> \u2013 when it is not clear which factor or factors caused the observed effect. Confounding is the downfall of many experiments, though sometimes it is hidden.\r\n<div class=\"textbox\">\r\n<h3>Confounding<\/h3>\r\n<strong>Confounding<\/strong> occurs when there are two potential variables that could have caused the outcome and it is not possible to determine which actually caused the result.\r\n\r\n<\/div>\r\n<div class=\"textbox exercises\">\r\n<h3>examples<\/h3>\r\nA drug company study about a weight loss pill might report that people lost an average of 8 pounds while using their new drug. However, in the fine print you find a statement saying that participants were encouraged to also diet and exercise. It is not clear in this case whether the weight loss is due to the pill, to diet and exercise, or a combination of both. In this case confounding has occurred.\r\n\r\n&nbsp;\r\n\r\n<\/div>\r\n<div class=\"textbox exercises\">\r\n<h3>Example<\/h3>\r\nResearchers conduct an experiment to determine whether students will perform better on an arithmetic test if they listen to music during the test. They first give the student a test without music, then give a similar test while the student listens to music. In this case, the student might perform better on the second test, regardless of the music, simply because it was the second test and they were warmed up.\r\n\r\n<\/div>\r\nView the following for additional discussion of these examples.\r\n\r\nhttps:\/\/youtu.be\/SrCm12HZay0\r\n<div class=\"textbox key-takeaways\">\r\n<h3>Try It Now<\/h3>\r\n<iframe id=\"mom10\" class=\"resizable\" src=\"https:\/\/www.myopenmath.com\/multiembedq.php?id=6736&amp;theme=oea&amp;iframe_resize_id=mom10\" width=\"100%\" height=\"250\"><\/iframe>\r\n\r\n<\/div>\r\n<div>There are a number of measures that can be introduced to help reduce the likelihood of confounding. The primary measure is to use a <strong>control group<\/strong>.<\/div>\r\n<div class=\"textbox\">\r\n<h3>Control group<\/h3>\r\nWhen using a control group, the participants are divided into two or more groups, typically a <strong>control group<\/strong> and a treatment group. The treatment group receives the treatment being tested; the control group does not receive the treatment.\r\n\r\n<\/div>\r\nIdeally, the groups are otherwise as similar as possible, isolating the treatment as the only potential source of difference between the groups. For this reason, the method of dividing groups is important. Some researchers attempt to ensure that the groups have similar characteristics (same number of females, same number of people over 50, etc.), but it is nearly impossible to control for every characteristic. Because of this, random assignment is very commonly used.\r\n<div class=\"textbox exercises\">\r\n<h3>examples<\/h3>\r\nTo determine if a two day prep course would help high school students improve their scores on the SAT test, a group of students was randomly divided into two subgroups. The first group, the treatment group, was given a two day prep course. The second group, the control group, was not given the prep course. Afterwards, both groups were given the SAT.\r\n\r\n<hr \/>\r\n\r\nA company testing a new plant food grows two crops of plants in adjacent fields, the treatment group receiving the new plant food and the control group not. The crop yield would then be compared. By growing them at the same time in adjacent fields, they are controlling for weather and other confounding factors.\r\n\r\n<\/div>\r\nSometimes not giving the control group anything does not completely control for confounding variables. For example, suppose a medicine study is testing a new headache pill by giving the treatment group the pill and the control group nothing. If the treatment group showed improvement, we would not know whether it was due to the medicine in the pill, or a response to have taken any pill. This is called a <strong>placebo effect<\/strong>.\r\n<div>\r\n<div class=\"textbox\">\r\n<div>\r\n<h3>Placebo effect<\/h3>\r\n<\/div>\r\n<div>\r\n\r\nThe <strong>placebo effect<\/strong> is when the effectiveness of a treatment is influenced by the patient\u2019s perception of how effective they think the treatment will be, so a result might be seen even if the treatment is ineffectual.\r\n\r\n<\/div>\r\n<\/div>\r\n<div class=\"textbox exercises\">\r\n<h3>example<\/h3>\r\nA study found that when doing painful dental tooth extractions, patients told they were receiving a strong painkiller while actually receiving a saltwater injection found as much pain relief as patients receiving a dose of morphine.[footnote]Levine JD, Gordon NC, Smith R, Fields HL. (1981) Analgesic responses to morphine and placebo in individuals with postoperative pain. Pain. 10:379-89.[\/footnote]\r\n\r\n<\/div>\r\n<\/div>\r\nTo control for the placebo effect, a <strong>placebo<\/strong>, or dummy treatment, is often given to the control group. This way, both groups are truly identical except for the specific treatment given.\r\n<div>\r\n<div class=\"textbox\">\r\n<div>\r\n<h3>Placebo and Placebo controlled experiments<\/h3>\r\n<\/div>\r\n<div>\r\n<ul>\r\n \t<li>A <strong>placebo<\/strong> is a dummy treatment given to control for the placebo effect.<\/li>\r\n \t<li>An experiment that gives the control group a placebo is called a <strong>placebo controlled experiment<\/strong>.<\/li>\r\n<\/ul>\r\n<\/div>\r\n<\/div>\r\n<div class=\"textbox exercises\">\r\n<h3>examples<\/h3>\r\nIn a study for a new medicine that is dispensed in a pill form, a sugar pill could be used as a placebo.\r\n\r\n<hr \/>\r\n\r\nIn a study on the effect of alcohol on memory, a non-alcoholic beer might be given to the control group as a placebo.\r\n\r\n<hr \/>\r\n\r\nIn a study of a frozen meal diet plan, the treatment group would receive the diet food, and the control could be given standard frozen meals stripped of their original packaging.\r\n\r\nThe following video walks through the controlled experiment scenarios, including the ones using placebos.\r\n\r\nhttps:\/\/youtu.be\/UkCHUeqMb5Y\r\n\r\n<\/div>\r\nIn some cases, it is more appropriate to compare to a conventional treatment than a placebo. For example, in a cancer research study, it would not be ethical to deny any treatment to the control group or to give a placebo treatment. In this case, the currently acceptable medicine would be given to the second group, called a <strong>comparison group<\/strong> in this case. In our SAT test example, the non-treatment group would most likely be encouraged to study on their own, rather than be asked to not study at all, to provide a meaningful comparison.\r\n\r\n<\/div>\r\nWhen using a placebo, it would defeat the purpose if the participant knew they were receiving the placebo.\r\n<div>\r\n<div class=\"textbox\">\r\n<div>\r\n<h3>Blind studies<\/h3>\r\n<\/div>\r\n<div>\r\n<ul>\r\n \t<li>A <strong>blind study<\/strong> is one in which the participant does not know whether or not they are receiving the treatment or a placebo.<\/li>\r\n \t<li>A <strong>double-blind study<\/strong> is one in which those interacting with the participants don\u2019t know who is in the treatment group and who is in the control group.<\/li>\r\n<\/ul>\r\n<\/div>\r\n<\/div>\r\n<div class=\"textbox exercises\">\r\n<h3>examples<\/h3>\r\n<div>\r\n\r\nIn a study about anti-depression medicine, you would not want the psychological evaluator to know whether the patient is in the treatment or control group either, as it might influence their evaluation, so the experiment should be conducted as a double-blind study.\r\n\r\n<\/div>\r\n\r\n<hr \/>\r\n\r\nIt should be noted that not every experiment needs a control group.\r\n\r\nIf a researcher is testing whether a new fabric can withstand fire, she simply needs to torch multiple samples of the fabric \u2013 there is no need for a control group.\r\n\r\nThese examples are demonstrated in the following video.\r\n\r\nhttps:\/\/youtu.be\/7BFZVGCxeYc\r\n\r\n<\/div>\r\n<div class=\"textbox key-takeaways\">\r\n<h3>Try It now<\/h3>\r\nTo test a new lie detector, two groups of subjects are given the new test. One group is asked to answer all the questions truthfully, and the second group is asked to lie on one set of questions. The person administering the lie detector test does not know what group each subject is in.\r\n\r\nDoes this experiment have a control group? Is it blind, double-blind, or neither?\r\n[reveal-answer q=\"514145\"]Show Answer[\/reveal-answer]\r\n[hidden-answer a=\"514145\"]\r\n&lt;div&gt;\r\n\r\nThe truth-telling group could be considered the control group, but really both groups are treatment groups here, since it is important for the lie detector to be able to correctly identify lies, and also not identify truth telling as lying. This study is blind, since the person running the test does not know what group each subject is in.\r\n[\/hidden-answer]\r\n\r\n<iframe id=\"mom50\" class=\"resizable\" src=\"https:\/\/www.myopenmath.com\/multiembedq.php?id=6728&amp;theme=oea&amp;iframe_resize_id=mom50\" width=\"100%\" height=\"250\"><\/iframe>\r\n\r\n<\/div>\r\n<\/div>","rendered":"<p>As we mentioned previously, the first thing we should do before conducting a survey is to identify the population that we want to study. In this lesson, we will show you examples of how to identify the population in a study, and determine whether or not the study actually represents the intended population. We will discuss different techniques\u00a0for <em>random sampling<\/em> that are intended to ensure a population is well represented in a sample.<\/p>\n<p>We will also identify the difference between an observational study and an experiment, and ways experiments can be conducted. By the end of this lesson, we hope that you will also be confident in identifying when an experiment may have\u00a0been affected by confounding or the placebo effect, and the methods that are employed to avoid them.<\/p>\n<div class=\"textbox learning-objectives\">\n<h3>Learning Objectives<\/h3>\n<ul>\n<li>Identify\u00a0methods for obtaining a random sample of the intended population of a study<\/li>\n<li>Identify ineffective ways of obtaining a random sample from a population<\/li>\n<li>Identify types of sample bias<\/li>\n<li>Identify the differences between observational study and an experiment<\/li>\n<li>Identify the treatment in an experiment<\/li>\n<li>Determine whether an experiment may have been influenced by confounding<\/li>\n<\/ul>\n<\/div>\n<h2>Sampling Methods and Bias<\/h2>\n<h3>Selecting a Population<\/h3>\n<p>Suppose we are hired by a politician to determine the amount of support he has among the electorate should he decide to run for another term.\u00a0 What population should we study? Every person in the district? Not every person is eligible to vote, and regardless of how strongly someone likes or dislikes the candidate, they don&#8217;t have much to do with him being re-elected if they are not able to vote.<\/p>\n<p>What about eligible voters in the district? That might be better, but if someone is eligible to vote but does not\u00a0register by the deadline, they won&#8217;t have any say in the election either. What about registered voters? Many people are registered but choose not to vote. What about &#8220;likely voters?&#8221;<\/p>\n<p>This is the criteria used in much political polling, but it is sometimes difficult to define a &#8220;likely voter.&#8221; Is it someone who voted in the last election? In the last general election? In the last presidential election? Should we consider someone who just turned 18 a &#8220;likely voter?&#8221; They weren&#8217;t eligible to vote in the past, so how do we judge the likelihood that they will vote in the next election?<\/p>\n<p><a href=\"https:\/\/s3-us-west-2.amazonaws.com\/courses-images\/wp-content\/uploads\/sites\/1141\/2016\/12\/29220533\/crowd-1699137_640.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-1026\" src=\"https:\/\/s3-us-west-2.amazonaws.com\/courses-images\/wp-content\/uploads\/sites\/1141\/2016\/12\/29220533\/crowd-1699137_640.jpg\" alt=\"rows of Lego figurines\" width=\"640\" height=\"360\" \/><\/a><\/p>\n<p>&nbsp;<\/p>\n<p>In November 1998, former professional wrestler Jesse &#8220;The Body&#8221; Ventura was elected governor of Minnesota. Up until right before the election, most polls showed he had little chance of winning. There were several contributing factors to the polls not reflecting the actual intent of the electorate:<\/p>\n<ul>\n<li>Ventura was running on a third-party ticket and most polling methods are better suited to a two-candidate race.<\/li>\n<li>Many respondents to polls may have been embarrassed to tell pollsters that they were planning to vote for a professional wrestler.<\/li>\n<li>The mere fact that the polls showed Ventura had little chance of winning might have prompted some people to vote for him in protest to send a message to the major-party candidates.<\/li>\n<\/ul>\n<p>But one of the major contributing factors was that Ventura recruited a substantial amount of support from young people, particularly college students, who had never voted before and who registered specifically to vote in the gubernatorial election. The polls did not deem these young people likely voters (since in most cases young people have a lower rate of voter registration and a turnout rate for elections) and so the polling samples were subject to <strong>sampling bias<\/strong>: they omitted a portion of the electorate that was weighted in favor of the winning candidate.<\/p>\n<div>\n<div class=\"textbox\">\n<div>\n<h3>Sampling bias<\/h3>\n<\/div>\n<div>\n<p>A sampling method is biased if every member of the population doesn\u2019t have equal likelihood of being in the sample.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<p>So even identifying the population can be a difficult job, but once we have identified the population, how do we choose an appropriate sample? Remember, although we would prefer to survey all members of the population, this is usually impractical unless the population is very small, so we choose a sample. There are many ways to sample a population, but there is one goal we need to keep in mind: we would like the sample to be <em>representative of the population<\/em>.<\/p>\n<p>Returning to our hypothetical job as a political pollster, we would not anticipate very accurate results if we drew all of our samples from among the customers at a Starbucks, nor would we expect that a sample drawn entirely from the membership list of the local Elks club would provide a useful picture of district-wide support for our candidate.<\/p>\n<p>One way to ensure that the sample has a reasonable chance of mirroring the population is to employ <em>randomness<\/em>. The most basic random method is simple random sampling.<\/p>\n<div>\n<div class=\"textbox\">\n<div>\n<h3>Simple random sample<\/h3>\n<\/div>\n<div>\n<p>A <strong>random sample<\/strong> is one in which each member of the population has an equal probability of being chosen. A <strong>simple random sample<\/strong> is one in which every member of the population and any group of members has an equal probability of being chosen.<\/p>\n<\/div>\n<\/div>\n<div class=\"textbox exercises\">\n<h3>example<\/h3>\n<div>\n<p>If we could somehow identify all likely voters in the state, put each of their names on a piece of paper, toss the slips into a (very large) hat and draw 1000 slips out of the hat, we would have a simple random sample.<\/p>\n<\/div>\n<p>In practice, computers are better suited for this sort of endeavor than millions of slips of paper and extremely large headgear.<\/p>\n<\/div>\n<\/div>\n<p>It is always possible, however, that even a random sample might end up not being totally representative of the population.\u00a0 If we repeatedly take samples of 1000 people from among the population of likely voters in the state of Washington, some of these samples might tend to have a slightly higher percentage of Democrats (or Republicans) than does the general population; some samples might include more older people and some samples might include more younger people; etc. In most cases, this <strong>sampling variability<\/strong> is not significant.<\/p>\n<div>\n<div class=\"textbox\">\n<div>\n<h3>Sampling variability<\/h3>\n<\/div>\n<div>\n<p>The natural variation of samples is called <strong>sampling variability<\/strong>.<\/p>\n<p>This is unavoidable and expected in random sampling, and in most cases is not an issue.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<p>To help account for variability, pollsters might instead use a <strong>stratified sample<\/strong>.<\/p>\n<div>\n<div class=\"textbox\">\n<div>\n<h3>Stratified sampling<\/h3>\n<\/div>\n<div>\n<p>In <strong>stratified sampling<\/strong>, a population is divided into a number of subgroups (or strata). Random samples are then taken from each subgroup with sample sizes proportional to the size of the subgroup in the population.<\/p>\n<\/div>\n<\/div>\n<div class=\"textbox exercises\">\n<h3>example<\/h3>\n<div>\n<p>Suppose in a particular state that previous data indicated that the electorate was comprised of 39% Democrats, 37% Republicans and 24% independents.\u00a0 In a sample of 1000 people, they would then expect to get about 390 Democrats, 370 Republicans and 240 independents.\u00a0 To accomplish this, they could randomly select 390 people from among those voters known to be Democrats, 370 from those known to be Republicans, and 240 from those with no party affiliation.<\/p>\n<\/div>\n<\/div>\n<p>Stratified sampling can also be used to select a sample with people in desired age groups, a specified mix ratio of males and females, etc. A variation on this technique is called <strong>quota sampling<\/strong>.<\/p>\n<\/div>\n<div>\n<div class=\"textbox\">\n<div>\n<h3>Quota sampling<\/h3>\n<\/div>\n<div>\n<p><strong>Quota sampling<\/strong> is a variation on stratified sampling, wherein samples are collected in each subgroup until the desired quota is met.<\/p>\n<\/div>\n<\/div>\n<div class=\"textbox exercises\">\n<h3>example<\/h3>\n<div>\n<p>Suppose the pollsters call people at random, but once they have met their quota of 390 Democrats, they only gather people who do not identify themselves as a Democrat.<\/p>\n<\/div>\n<p>You may have had the experience of being called by a telephone pollster who started by asking you your age, income, etc. and then thanked you for your time and hung up before asking any &#8220;real&#8221; questions. Most likely, they already had contacted enough people in your demographic group and were looking for people who were older or younger, richer or poorer, etc. Quota sampling is usually a bit easier than stratified sampling, but also does not ensure the same level of randomness.<\/p>\n<\/div>\n<\/div>\n<div>\n<p>Another sampling method is <strong>cluster sampling<\/strong>, in which the population is divided into groups, and one or more groups are randomly selected to be in the sample.<\/p>\n<\/div>\n<div>\n<div class=\"textbox\">\n<div>\n<h3>Cluster sampling<\/h3>\n<\/div>\n<div>\n<p>In <strong>cluster sampling<\/strong>, the population is divided into subgroups (clusters), and a set of subgroups are selected to be in the sample.<\/p>\n<\/div>\n<\/div>\n<div class=\"textbox exercises\">\n<h3>example<\/h3>\n<p>If the college wanted to survey students, since students are already divided into classes, they could randomly select 10 classes and give the survey to all the students in those classes. This would be cluster sampling.<\/p>\n<\/div>\n<\/div>\n<p>Other sampling methods include <strong>systematic sampling<\/strong>.<\/p>\n<div>\n<div class=\"textbox\">\n<div>\n<h3>Systematic sampling<\/h3>\n<\/div>\n<div>\n<p>In <strong>systematic sampling<\/strong>, every <em>n<sup>th<\/sup><\/em> member of the population is selected to be in the sample.<\/p>\n<\/div>\n<\/div>\n<div class=\"textbox exercises\">\n<h3>example<\/h3>\n<p>To select a sample using systematic sampling, a pollster calls every 100th name in the phone book.<\/p>\n<p>Systematic sampling is not as random as a simple random sample (if your name is Albert Aardvark and your sister Alexis Aardvark is right after you in the phone book, there is no way you could both end up in the sample) but it can yield acceptable samples.<\/p>\n<\/div>\n<\/div>\n<h3>The Worst Way to Sample<\/h3>\n<p>Perhaps the worst types of sampling methods are <strong>convenience samples<\/strong> and <strong>voluntary response samples<\/strong>.<\/p>\n<div>\n<div class=\"textbox\">\n<div>\n<h3>Convenience sampling and voluntary response sampling<\/h3>\n<\/div>\n<div>\n<p><strong>Convenience sampling<\/strong> is the practice of samples chosen by selecting whoever is convenient.<\/p>\n<p><strong>Voluntary response sampling<\/strong> is allowing the sample to volunteer.<\/p>\n<\/div>\n<\/div>\n<div class=\"textbox exercises\">\n<h3>example<\/h3>\n<p>A pollster stands on a street corner and interviews the first 100 people who agree to speak to him. Which sampling method is represented by this scenario?<\/p>\n<div class=\"qa-wrapper\" style=\"display: block\"><span class=\"show-answer collapsed\" style=\"cursor: pointer\" data-target=\"q386299\">Show Answer<\/span><\/p>\n<div id=\"q386299\" class=\"hidden-answer\" style=\"display: none\">This is a convenience sample.<\/div>\n<\/div>\n<div>\n<hr \/>\n<p>A website has a survey asking readers to give their opinion on a tax proposal. Which sampling method is represented?<\/p>\n<div class=\"qa-wrapper\" style=\"display: block\"><span class=\"show-answer collapsed\" style=\"cursor: pointer\" data-target=\"q503402\">Show Answer<\/span><\/p>\n<div id=\"q503402\" class=\"hidden-answer\" style=\"display: none\">\n<div>\n<p>This is a self-selected sample, or voluntary response sample, in which respondents volunteer to participate.<\/p>\n<\/div>\n<p>Usually voluntary response samples are skewed towards people who have a particularly strong opinion about the subject of the survey or who just have way too much time on their hands and enjoy taking surveys.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<p>Watch the following video for an overview of all the sampling methods discussed so far.<\/p>\n<p><iframe loading=\"lazy\" id=\"oembed-1\" title=\"Sampling methods\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/sk4TDJU7QbY?feature=oembed&#38;rel=0\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<div class=\"textbox key-takeaways\">\n<h3>Try It Now<\/h3>\n<p>In each case, indicate what sampling method was used<\/p>\n<p>a. Every 4th person in the class was selected<\/p>\n<p>b. A sample was selected to contain 25 men and 35 women<\/p>\n<p>c. Viewers of a new show are asked to vote on the show\u2019s website<\/p>\n<p>d. A website randomly selects 50 of their customers to send a satisfaction survey to<\/p>\n<p>e. To survey voters in a town, a polling company randomly selects 10 city blocks, and interviews everyone who lives on those blocks.<\/p>\n<p><iframe loading=\"lazy\" id=\"mom2\" class=\"resizable\" src=\"https:\/\/www.myopenmath.com\/multiembedq.php?id=6733&amp;theme=oea&amp;iframe_resize_id=mom2\" width=\"100%\" height=\"350\"><\/iframe><\/p>\n<\/div>\n<h3>Problematic Sampling and Surveying<\/h3>\n<p>There are number of ways that a study can be ruined before you even start collecting data. The first we have already explored \u2013 <strong>sampling <\/strong>or<strong> selection bias<\/strong>, which is when the sample is not representative of the population. One example of this is <strong>voluntary response bias<\/strong>, which is bias introduced by only collecting data from those who volunteer to participate. This is not the only potential source of bias.<\/p>\n<div class=\"textbox\">\n<h3>Sources of bias<\/h3>\n<ul>\n<li><strong>Sampling bias<\/strong> \u2013 when the sample is not representative of the population<\/li>\n<li><strong>Voluntary response bias<\/strong> \u2013 the sampling bias that often occurs when the sample is volunteers<\/li>\n<li><strong>Self-interest study<\/strong> \u2013 bias that can occur when the researchers have an interest in the outcome<\/li>\n<li><strong>Response bias<\/strong> \u2013 when the responder gives inaccurate responses for any reason<\/li>\n<li><strong>Perceived lack of anonymity<\/strong> \u2013 when the responder fears giving an honest answer might negatively affect them<\/li>\n<li><strong>Loaded questions<\/strong> \u2013 when the question wording influences the responses<\/li>\n<li><strong>Non-response bias<\/strong> \u2013 when people refusing to participate in the study can influence the validity of the outcome<\/li>\n<\/ul>\n<\/div>\n<\/div>\n<div class=\"textbox exercises\">\n<h3>examples<\/h3>\n<p>Consider a recent study which found that chewing gum may raise math grades in teenagers<a class=\"footnote\" title=\"Reuters. http:\/\/news.yahoo.com\/s\/nm\/20090423\/od_uk_nm\/oukoe_uk_gum_learning. Retrieved 4\/27\/09\" id=\"return-footnote-800-1\" href=\"#footnote-800-1\" aria-label=\"Footnote 1\"><sup class=\"footnote\">[1]<\/sup><\/a>. This study was conducted by the Wrigley Science Institute, a branch of the Wrigley chewing gum company. Identify the type of sampling bias found in this example.<\/p>\n<div class=\"qa-wrapper\" style=\"display: block\"><span class=\"show-answer collapsed\" style=\"cursor: pointer\" data-target=\"q561139\">Show Answer<\/span><\/p>\n<div id=\"q561139\" class=\"hidden-answer\" style=\"display: none\">This is an example of a <strong>self-interest study<\/strong>; one in which the researchers have a vested interest in the outcome of the study. While this does not necessarily ensure that the study was biased, it certainly suggests that we should subject the study to extra scrutiny.<\/div>\n<\/div>\n<div>\n<hr \/>\n<p>A survey asks people \u201cwhen was the last time you visited your doctor?\u201d What type of sampling bias might this lead to?<\/p>\n<div class=\"qa-wrapper\" style=\"display: block\"><span class=\"show-answer collapsed\" style=\"cursor: pointer\" data-target=\"q965739\">Show Answer<\/span><\/p>\n<div id=\"q965739\" class=\"hidden-answer\" style=\"display: none\">\n<div>\n<p>This might suffer from <strong>response bias<\/strong>, since many people might not remember exactly when they last saw a doctor and give inaccurate responses.<\/p>\n<\/div>\n<p>Sources of response bias may be innocent, such as bad memory, or as intentional as pressuring by the pollster. Consider, for example, how many voting initiative petitions people sign without even reading them.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<hr \/>\n<p>A survey asks participants a question about their interactions with members of other races. Which sampling bias might occur for this survey strategy?<\/p>\n<div class=\"qa-wrapper\" style=\"display: block\"><span class=\"show-answer collapsed\" style=\"cursor: pointer\" data-target=\"q610809\">Show Answer<\/span><\/p>\n<div id=\"q610809\" class=\"hidden-answer\" style=\"display: none\">Here, a <strong>perceived lack of anonymity<\/strong> could influence the outcome. The respondent might not want to be perceived as racist even if they are, and give an untruthful answer.<\/div>\n<\/div>\n<hr \/>\n<p>An employer puts out a survey asking their employees if they have a drug abuse problem and need treatment help. Which sampling bias may occur in this scenario?<\/p>\n<div class=\"qa-wrapper\" style=\"display: block\"><span class=\"show-answer collapsed\" style=\"cursor: pointer\" data-target=\"q801450\">Show Answer<\/span><\/p>\n<div id=\"q801450\" class=\"hidden-answer\" style=\"display: none\">Here, answering truthfully might have consequences; responses might not be accurate if the employees do not feel their responses are anonymous or fear retribution from their employer. This survey has the\u00a0potential for\u00a0<strong>perceived lack of anonymity<\/strong>.<\/div>\n<\/div>\n<div>\n<hr \/>\n<p>A survey asks \u201cdo you support funding research of alternative energy sources to reduce our reliance on high-polluting fossil fuels?\u201d Which sampling bias may result from this survey?<\/p>\n<div class=\"qa-wrapper\" style=\"display: block\"><span class=\"show-answer collapsed\" style=\"cursor: pointer\" data-target=\"q464953\">Show Answer<\/span><\/p>\n<div id=\"q464953\" class=\"hidden-answer\" style=\"display: none\">\n<div>\n<p>This is an example of a <strong>loaded <\/strong>or<strong> leading question<\/strong> \u2013 questions whose wording leads the respondent towards an answer.<\/p>\n<\/div>\n<p>Loaded questions can occur intentionally by pollsters with an agenda, or accidentally through poor question wording. Also a concern is <strong>question order<\/strong>, where the order of questions changes the results. A psychology researcher provides an example<a class=\"footnote\" title=\"Swartz, Norbert. http:\/\/www.umich.edu\/~newsinfo\/MT\/01\/Fal01\/mt6f01.html. Retrieved 3\/31\/2009\" id=\"return-footnote-800-2\" href=\"#footnote-800-2\" aria-label=\"Footnote 2\"><sup class=\"footnote\">[2]<\/sup><\/a>:<\/p>\n<blockquote><p>\u201cMy favorite finding is this: we did a study where we asked students, &#8216;How satisfied are you with your life? How often do you have a date?&#8217; The two answers were not statistically related &#8211; you would conclude that there is no relationship between dating frequency and life satisfaction. But when we reversed the order and asked, &#8216;How often do you have a date? How satisfied are you with your life?&#8217; the statistical relationship was a strong one. You would now conclude that there is nothing as important in a student&#8217;s life as dating frequency.\u201d<\/p><\/blockquote>\n<\/div>\n<\/div>\n<\/div>\n<hr \/>\n<p>A telephone poll asks the question \u201cDo you often have time to relax and read a book?\u201d, and 50% of the people called refused to answer the survey. Which sampling bias is represented by this survey?<\/p>\n<div class=\"qa-wrapper\" style=\"display: block\"><span class=\"show-answer collapsed\" style=\"cursor: pointer\" data-target=\"q782411\">Show Answer<\/span><\/p>\n<div id=\"q782411\" class=\"hidden-answer\" style=\"display: none\">It is unlikely that the results will be representative of the entire population. This is an example of <strong>non-response bias<\/strong>, introduced by people refusing to participate in a study or dropping out of an experiment. When people refuse to participate, we can no longer be so certain that our sample is representative of the population.<\/div>\n<\/div>\n<\/div>\n<p>These problematic scenarios for statistics gathering are discussed further in the following video.<\/p>\n<p><iframe loading=\"lazy\" id=\"oembed-2\" title=\"Bias\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/GXAi06PiVsk?feature=oembed&#38;rel=0\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<div class=\"textbox key-takeaways\">\n<h3>Try It Now<\/h3>\n<p>In each situation, identify a potential source of bias<\/p>\n<p>a. A survey asks how many sexual partners a person has had in the last year<\/p>\n<p>b. A radio station asks readers to phone in their choice in a daily poll.<\/p>\n<p>c. A substitute teacher wants to know how students in the class did on their last test. The teacher asks the 10 students sitting in the front row to state their latest test score.<\/p>\n<p>d. High school students are asked if they have consumed alcohol in the last two weeks.<\/p>\n<p>e. The Beef Council releases a study stating that consuming red meat poses little cardiovascular risk.<\/p>\n<p>f. A poll asks \u201cDo you support a new transportation tax, or would you prefer to see our public transportation system fall apart?\u201d<\/p>\n<p><iframe loading=\"lazy\" id=\"mom5\" class=\"resizable\" src=\"https:\/\/www.myopenmath.com\/multiembedq.php?id=6727&amp;theme=oea&amp;iframe_resize_id=mom5\" width=\"100%\" height=\"350\"><\/iframe><\/p>\n<\/div>\n<h2>Experiments<\/h2>\n<h3>Observing vs. Acting<\/h3>\n<p>So far, we have primarily discussed <strong>observational studies<\/strong> \u2013 studies in which conclusions would be drawn from observations of a sample or the population. In some cases these observations might be unsolicited, such as studying the percentage of cars that turn right at a red light even when there is a \u201cno turn on red\u201d sign. In other cases the observations are solicited, like in a survey or a poll.<\/p>\n<p>In contrast, it is common to use <strong>experiments<\/strong> when exploring how subjects react to an outside influence. In an experiment, some kind of <strong>treatment<\/strong> is applied to the subjects and the results are measured and recorded.<\/p>\n<p><a href=\"https:\/\/s3-us-west-2.amazonaws.com\/courses-images\/wp-content\/uploads\/sites\/1141\/2016\/12\/30181148\/lab-217041_640.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-1033\" src=\"https:\/\/s3-us-west-2.amazonaws.com\/courses-images\/wp-content\/uploads\/sites\/1141\/2016\/12\/30181148\/lab-217041_640.jpg\" alt=\"Collection of clear glass scientific measuring equipment, with clear liquid in them\" width=\"640\" height=\"423\" \/><\/a><\/p>\n<div>\n<div class=\"textbox\">\n<div>\n<h3>Observational studies and experiments<\/h3>\n<\/div>\n<div>\n<ul>\n<li>An <strong>observational study<\/strong> is a study based on observations or measurements<\/li>\n<li>An <strong>experiment<\/strong> is a study in which the effects of a <strong>treatment<\/strong> are measured<\/li>\n<\/ul>\n<\/div>\n<\/div>\n<\/div>\n<div class=\"textbox exercises\">\n<h3>Examples<\/h3>\n<p>Here are some examples of experiments:<\/p>\n<p>A pharmaceutical company tests a new medicine for treating Alzheimer\u2019s disease by administering the drug to 50 elderly patients with recent diagnoses. The treatment here is the new drug.<\/p>\n<hr \/>\n<p>A gym tests out a new weight loss program by enlisting 30 volunteers to try out the program. The treatment here is the new program.<\/p>\n<hr \/>\n<p>You test a new kitchen cleaner by buying a bottle and cleaning your kitchen. The new cleaner is the treatment.<\/p>\n<hr \/>\n<p>A psychology researcher explores the effect of music on temperament by measuring people\u2019s temperament while listening to different types of music. The music is the treatment.<\/p>\n<\/div>\n<p>&nbsp;<\/p>\n<p>These examples are discussed further in the following video.<\/p>\n<p><iframe loading=\"lazy\" id=\"oembed-3\" title=\"Experiments\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/HSTTKzsdHEw?feature=oembed&#38;rel=0\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<div class=\"textbox key-takeaways\">\n<h3>Try It Now<\/h3>\n<p>Is each scenario describing an observational study or an experiment?<\/p>\n<p>a. The weights of 30 randomly selected people are measured<\/p>\n<p>b. Subjects are asked to do 20 jumping jacks, and then their heart rates are measured<\/p>\n<p>c. Twenty coffee drinkers and twenty tea drinkers are given a concentration test<\/p>\n<div class=\"qa-wrapper\" style=\"display: block\"><span class=\"show-answer collapsed\" style=\"cursor: pointer\" data-target=\"q713031\">Show Answer<\/span><\/p>\n<div id=\"q713031\" class=\"hidden-answer\" style=\"display: none\">\n<p>a. Observational study<\/p>\n<div>\n<p>b. Experiment; the treatment is the jumping jacks<\/p>\n<p>c. Experiment; the treatments are coffee and tea<\/p>\n<\/div>\n<\/div>\n<\/div>\n<p><iframe loading=\"lazy\" id=\"mom1\" class=\"resizable\" src=\"https:\/\/www.myopenmath.com\/multiembedq.php?id=6914&amp;theme=oea&amp;iframe_resize_id=mom1\" width=\"100%\" height=\"250\"><\/iframe><\/p>\n<\/div>\n<p>When conducting experiments, it is essential to isolate the treatment being tested.<\/p>\n<div class=\"textbox exercises\">\n<h3>example<\/h3>\n<p>Suppose a middle school (junior high) finds that their students are not scoring well on the state\u2019s standardized math test. They decide to run an experiment to see if an alternate curriculum would improve scores. To run the test, they hire a math specialist to come in and teach a class using the new curriculum. To their delight, they see an improvement in test scores.<\/p>\n<\/div>\n<p>The difficulty with this scenario is that it is not clear whether the curriculum is responsible for the improvement, or whether the improvement is due to a math specialist teaching the class. This is called <strong>confounding<\/strong> \u2013 when it is not clear which factor or factors caused the observed effect. Confounding is the downfall of many experiments, though sometimes it is hidden.<\/p>\n<div class=\"textbox\">\n<h3>Confounding<\/h3>\n<p><strong>Confounding<\/strong> occurs when there are two potential variables that could have caused the outcome and it is not possible to determine which actually caused the result.<\/p>\n<\/div>\n<div class=\"textbox exercises\">\n<h3>examples<\/h3>\n<p>A drug company study about a weight loss pill might report that people lost an average of 8 pounds while using their new drug. However, in the fine print you find a statement saying that participants were encouraged to also diet and exercise. It is not clear in this case whether the weight loss is due to the pill, to diet and exercise, or a combination of both. In this case confounding has occurred.<\/p>\n<p>&nbsp;<\/p>\n<\/div>\n<div class=\"textbox exercises\">\n<h3>Example<\/h3>\n<p>Researchers conduct an experiment to determine whether students will perform better on an arithmetic test if they listen to music during the test. They first give the student a test without music, then give a similar test while the student listens to music. In this case, the student might perform better on the second test, regardless of the music, simply because it was the second test and they were warmed up.<\/p>\n<\/div>\n<p>View the following for additional discussion of these examples.<\/p>\n<p><iframe loading=\"lazy\" id=\"oembed-4\" title=\"Confounding\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/SrCm12HZay0?feature=oembed&#38;rel=0\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<div class=\"textbox key-takeaways\">\n<h3>Try It Now<\/h3>\n<p><iframe loading=\"lazy\" id=\"mom10\" class=\"resizable\" src=\"https:\/\/www.myopenmath.com\/multiembedq.php?id=6736&amp;theme=oea&amp;iframe_resize_id=mom10\" width=\"100%\" height=\"250\"><\/iframe><\/p>\n<\/div>\n<div>There are a number of measures that can be introduced to help reduce the likelihood of confounding. The primary measure is to use a <strong>control group<\/strong>.<\/div>\n<div class=\"textbox\">\n<h3>Control group<\/h3>\n<p>When using a control group, the participants are divided into two or more groups, typically a <strong>control group<\/strong> and a treatment group. The treatment group receives the treatment being tested; the control group does not receive the treatment.<\/p>\n<\/div>\n<p>Ideally, the groups are otherwise as similar as possible, isolating the treatment as the only potential source of difference between the groups. For this reason, the method of dividing groups is important. Some researchers attempt to ensure that the groups have similar characteristics (same number of females, same number of people over 50, etc.), but it is nearly impossible to control for every characteristic. Because of this, random assignment is very commonly used.<\/p>\n<div class=\"textbox exercises\">\n<h3>examples<\/h3>\n<p>To determine if a two day prep course would help high school students improve their scores on the SAT test, a group of students was randomly divided into two subgroups. The first group, the treatment group, was given a two day prep course. The second group, the control group, was not given the prep course. Afterwards, both groups were given the SAT.<\/p>\n<hr \/>\n<p>A company testing a new plant food grows two crops of plants in adjacent fields, the treatment group receiving the new plant food and the control group not. The crop yield would then be compared. By growing them at the same time in adjacent fields, they are controlling for weather and other confounding factors.<\/p>\n<\/div>\n<p>Sometimes not giving the control group anything does not completely control for confounding variables. For example, suppose a medicine study is testing a new headache pill by giving the treatment group the pill and the control group nothing. If the treatment group showed improvement, we would not know whether it was due to the medicine in the pill, or a response to have taken any pill. This is called a <strong>placebo effect<\/strong>.<\/p>\n<div>\n<div class=\"textbox\">\n<div>\n<h3>Placebo effect<\/h3>\n<\/div>\n<div>\n<p>The <strong>placebo effect<\/strong> is when the effectiveness of a treatment is influenced by the patient\u2019s perception of how effective they think the treatment will be, so a result might be seen even if the treatment is ineffectual.<\/p>\n<\/div>\n<\/div>\n<div class=\"textbox exercises\">\n<h3>example<\/h3>\n<p>A study found that when doing painful dental tooth extractions, patients told they were receiving a strong painkiller while actually receiving a saltwater injection found as much pain relief as patients receiving a dose of morphine.<a class=\"footnote\" title=\"Levine JD, Gordon NC, Smith R, Fields HL. (1981) Analgesic responses to morphine and placebo in individuals with postoperative pain. Pain. 10:379-89.\" id=\"return-footnote-800-3\" href=\"#footnote-800-3\" aria-label=\"Footnote 3\"><sup class=\"footnote\">[3]<\/sup><\/a><\/p>\n<\/div>\n<\/div>\n<p>To control for the placebo effect, a <strong>placebo<\/strong>, or dummy treatment, is often given to the control group. This way, both groups are truly identical except for the specific treatment given.<\/p>\n<div>\n<div class=\"textbox\">\n<div>\n<h3>Placebo and Placebo controlled experiments<\/h3>\n<\/div>\n<div>\n<ul>\n<li>A <strong>placebo<\/strong> is a dummy treatment given to control for the placebo effect.<\/li>\n<li>An experiment that gives the control group a placebo is called a <strong>placebo controlled experiment<\/strong>.<\/li>\n<\/ul>\n<\/div>\n<\/div>\n<div class=\"textbox exercises\">\n<h3>examples<\/h3>\n<p>In a study for a new medicine that is dispensed in a pill form, a sugar pill could be used as a placebo.<\/p>\n<hr \/>\n<p>In a study on the effect of alcohol on memory, a non-alcoholic beer might be given to the control group as a placebo.<\/p>\n<hr \/>\n<p>In a study of a frozen meal diet plan, the treatment group would receive the diet food, and the control could be given standard frozen meals stripped of their original packaging.<\/p>\n<p>The following video walks through the controlled experiment scenarios, including the ones using placebos.<\/p>\n<p><iframe loading=\"lazy\" id=\"oembed-5\" title=\"Controlled Experiments\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/UkCHUeqMb5Y?feature=oembed&#38;rel=0\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<\/div>\n<p>In some cases, it is more appropriate to compare to a conventional treatment than a placebo. For example, in a cancer research study, it would not be ethical to deny any treatment to the control group or to give a placebo treatment. In this case, the currently acceptable medicine would be given to the second group, called a <strong>comparison group<\/strong> in this case. In our SAT test example, the non-treatment group would most likely be encouraged to study on their own, rather than be asked to not study at all, to provide a meaningful comparison.<\/p>\n<\/div>\n<p>When using a placebo, it would defeat the purpose if the participant knew they were receiving the placebo.<\/p>\n<div>\n<div class=\"textbox\">\n<div>\n<h3>Blind studies<\/h3>\n<\/div>\n<div>\n<ul>\n<li>A <strong>blind study<\/strong> is one in which the participant does not know whether or not they are receiving the treatment or a placebo.<\/li>\n<li>A <strong>double-blind study<\/strong> is one in which those interacting with the participants don\u2019t know who is in the treatment group and who is in the control group.<\/li>\n<\/ul>\n<\/div>\n<\/div>\n<div class=\"textbox exercises\">\n<h3>examples<\/h3>\n<div>\n<p>In a study about anti-depression medicine, you would not want the psychological evaluator to know whether the patient is in the treatment or control group either, as it might influence their evaluation, so the experiment should be conducted as a double-blind study.<\/p>\n<\/div>\n<hr \/>\n<p>It should be noted that not every experiment needs a control group.<\/p>\n<p>If a researcher is testing whether a new fabric can withstand fire, she simply needs to torch multiple samples of the fabric \u2013 there is no need for a control group.<\/p>\n<p>These examples are demonstrated in the following video.<\/p>\n<p><iframe loading=\"lazy\" id=\"oembed-6\" title=\"Blind Experiments\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/7BFZVGCxeYc?feature=oembed&#38;rel=0\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<\/div>\n<div class=\"textbox key-takeaways\">\n<h3>Try It now<\/h3>\n<p>To test a new lie detector, two groups of subjects are given the new test. One group is asked to answer all the questions truthfully, and the second group is asked to lie on one set of questions. The person administering the lie detector test does not know what group each subject is in.<\/p>\n<p>Does this experiment have a control group? Is it blind, double-blind, or neither?<\/p>\n<div class=\"qa-wrapper\" style=\"display: block\"><span class=\"show-answer collapsed\" style=\"cursor: pointer\" data-target=\"q514145\">Show Answer<\/span><\/p>\n<div id=\"q514145\" class=\"hidden-answer\" style=\"display: none\">\n&lt;div&gt;<\/p>\n<p>The truth-telling group could be considered the control group, but really both groups are treatment groups here, since it is important for the lie detector to be able to correctly identify lies, and also not identify truth telling as lying. This study is blind, since the person running the test does not know what group each subject is in.\n<\/p><\/div>\n<\/div>\n<p><iframe loading=\"lazy\" id=\"mom50\" class=\"resizable\" src=\"https:\/\/www.myopenmath.com\/multiembedq.php?id=6728&amp;theme=oea&amp;iframe_resize_id=mom50\" width=\"100%\" height=\"250\"><\/iframe><\/p>\n<\/div>\n<\/div>\n\n\t\t\t <section class=\"citations-section\" role=\"contentinfo\">\n\t\t\t <h3>Candela Citations<\/h3>\n\t\t\t\t\t <div>\n\t\t\t\t\t\t <div id=\"citation-list-800\">\n\t\t\t\t\t\t\t <div class=\"licensing\"><div class=\"license-attribution-dropdown-subheading\">CC licensed content, Original<\/div><ul class=\"citation-list\"><li>Introduction and Learning Objectives. <strong>Provided by<\/strong>: Lumen Learning. <strong>License<\/strong>: <em><a target=\"_blank\" rel=\"license\" href=\"https:\/\/creativecommons.org\/licenses\/by\/4.0\/\">CC BY: Attribution<\/a><\/em><\/li><li>Revision and Adaptation. <strong>Provided by<\/strong>: Lumen Learning. <strong>License<\/strong>: <em><a target=\"_blank\" rel=\"license\" href=\"https:\/\/creativecommons.org\/licenses\/by\/4.0\/\">CC BY: Attribution<\/a><\/em><\/li><\/ul><div class=\"license-attribution-dropdown-subheading\">CC licensed content, Shared previously<\/div><ul class=\"citation-list\"><li>Math in Society. <strong>Authored by<\/strong>: David Lippman. <strong>Located at<\/strong>: <a target=\"_blank\" href=\"http:\/\/www.opentextbookstore.com\/mathinsociety\/\">http:\/\/www.opentextbookstore.com\/mathinsociety\/<\/a>. <strong>License<\/strong>: <em><a target=\"_blank\" rel=\"license\" href=\"https:\/\/creativecommons.org\/licenses\/by-sa\/4.0\/\">CC BY-SA: Attribution-ShareAlike<\/a><\/em><\/li><li>crowd-lego-staff-choice-selector. <strong>Authored by<\/strong>: mwewering. <strong>Located at<\/strong>: <a target=\"_blank\" href=\"https:\/\/pixabay.com\/en\/crowd-lego-staff-choice-selector-1699137\/\">https:\/\/pixabay.com\/en\/crowd-lego-staff-choice-selector-1699137\/<\/a>. <strong>License<\/strong>: <em><a target=\"_blank\" rel=\"license\" href=\"https:\/\/creativecommons.org\/about\/cc0\">CC0: No Rights Reserved<\/a><\/em><\/li><li>Sampling methods. <strong>Authored by<\/strong>: OCLPhase2&#039;s channel. <strong>Located at<\/strong>: <a target=\"_blank\" href=\"https:\/\/youtu.be\/sk4TDJU7QbY\">https:\/\/youtu.be\/sk4TDJU7QbY<\/a>. <strong>License<\/strong>: <em><a target=\"_blank\" rel=\"license\" href=\"https:\/\/creativecommons.org\/licenses\/by\/4.0\/\">CC BY: Attribution<\/a><\/em><\/li><li>Question ID 6733, 6727. <strong>Authored by<\/strong>: Lippman, David. <strong>License<\/strong>: <em><a target=\"_blank\" rel=\"license\" href=\"https:\/\/creativecommons.org\/licenses\/by\/4.0\/\">CC BY: Attribution<\/a><\/em>. <strong>License Terms<\/strong>: IMathAS Community License CC-BY +GPL<\/li><li>lab-research-chemistry-test. <strong>Authored by<\/strong>: PublicDomainPictures. <strong>Located at<\/strong>: <a target=\"_blank\" href=\"https:\/\/pixabay.com\/en\/lab-research-chemistry-test-217041\/\">https:\/\/pixabay.com\/en\/lab-research-chemistry-test-217041\/<\/a>. <strong>License<\/strong>: <em><a target=\"_blank\" rel=\"license\" href=\"https:\/\/creativecommons.org\/about\/cc0\">CC0: No Rights Reserved<\/a><\/em><\/li><li>Experiments. <strong>Authored by<\/strong>: OCLPhase2&#039;s channel. <strong>Located at<\/strong>: <a target=\"_blank\" href=\"https:\/\/youtu.be\/HSTTKzsdHEw\">https:\/\/youtu.be\/HSTTKzsdHEw<\/a>. <strong>License<\/strong>: <em><a target=\"_blank\" rel=\"license\" href=\"https:\/\/creativecommons.org\/licenses\/by\/4.0\/\">CC BY: Attribution<\/a><\/em><\/li><li>Confounding. <strong>Authored by<\/strong>: OCLPhase2&#039;s channel. <strong>Located at<\/strong>: <a target=\"_blank\" href=\"https:\/\/youtu.be\/SrCm12HZay0\">https:\/\/youtu.be\/SrCm12HZay0<\/a>. <strong>License<\/strong>: <em><a target=\"_blank\" rel=\"license\" href=\"https:\/\/creativecommons.org\/licenses\/by\/4.0\/\">CC BY: Attribution<\/a><\/em><\/li><li>Controlled Experiments. <strong>Authored by<\/strong>: OCLPhase2&#039;s channel. <strong>Located at<\/strong>: <a target=\"_blank\" href=\"https:\/\/youtu.be\/UkCHUeqMb5Y\">https:\/\/youtu.be\/UkCHUeqMb5Y<\/a>. <strong>License<\/strong>: <em><a target=\"_blank\" rel=\"license\" href=\"https:\/\/creativecommons.org\/licenses\/by\/4.0\/\">CC BY: Attribution<\/a><\/em><\/li><li>Blind Experiments. <strong>Authored by<\/strong>: OCLPhase2&#039;s channel. <strong>Provided by<\/strong>: https:\/\/youtu.be\/7BFZVGCxeYc. <strong>License<\/strong>: <em><a target=\"_blank\" rel=\"license\" href=\"https:\/\/creativecommons.org\/licenses\/by\/4.0\/\">CC BY: Attribution<\/a><\/em><\/li><li>Question ID 6736, 6728, 6914. <strong>Authored by<\/strong>: Lippman, David. <strong>License<\/strong>: <em><a target=\"_blank\" rel=\"license\" href=\"https:\/\/creativecommons.org\/licenses\/by\/4.0\/\">CC BY: Attribution<\/a><\/em>. <strong>License Terms<\/strong>: IMathAS Community License CC-BY + GPL<\/li><\/ul><\/div>\n\t\t\t\t\t\t <\/div>\n\t\t\t\t\t <\/div>\n\t\t\t <\/section><hr class=\"before-footnotes clear\" \/><div class=\"footnotes\"><ol><li id=\"footnote-800-1\">Reuters. <a href=\"http:\/\/news.yahoo.com\/s\/nm\/20090423\/od_uk_nm\/oukoe_uk_gum_learning\" target=\"_blank\">http:\/\/news.yahoo.com\/s\/nm\/20090423\/od_uk_nm\/oukoe_uk_gum_learning<\/a>. Retrieved 4\/27\/09 <a href=\"#return-footnote-800-1\" class=\"return-footnote\" aria-label=\"Return to footnote 1\">&crarr;<\/a><\/li><li id=\"footnote-800-2\">Swartz, Norbert. <a href=\"http:\/\/www.umich.edu\/~newsinfo\/MT\/01\/Fal01\/mt6f01.html\" target=\"_blank\">http:\/\/www.umich.edu\/~newsinfo\/MT\/01\/Fal01\/mt6f01.html<\/a>. Retrieved 3\/31\/2009 <a href=\"#return-footnote-800-2\" class=\"return-footnote\" aria-label=\"Return to footnote 2\">&crarr;<\/a><\/li><li id=\"footnote-800-3\">Levine JD, Gordon NC, Smith R, Fields HL. (1981) Analgesic responses to morphine and placebo in individuals with postoperative pain. Pain. 10:379-89. <a href=\"#return-footnote-800-3\" class=\"return-footnote\" aria-label=\"Return to footnote 3\">&crarr;<\/a><\/li><\/ol><\/div>","protected":false},"author":21,"menu_order":3,"template":"","meta":{"_candela_citation":"[{\"type\":\"original\",\"description\":\"Introduction and Learning Objectives\",\"author\":\"\",\"organization\":\"Lumen Learning\",\"url\":\"\",\"project\":\"\",\"license\":\"cc-by\",\"license_terms\":\"\"},{\"type\":\"cc\",\"description\":\"Math in Society\",\"author\":\"David Lippman\",\"organization\":\"\",\"url\":\"http:\/\/www.opentextbookstore.com\/mathinsociety\/\",\"project\":\"\",\"license\":\"cc-by-sa\",\"license_terms\":\"\"},{\"type\":\"cc\",\"description\":\"crowd-lego-staff-choice-selector\",\"author\":\"mwewering\",\"organization\":\"\",\"url\":\"https:\/\/pixabay.com\/en\/crowd-lego-staff-choice-selector-1699137\/\",\"project\":\"\",\"license\":\"cc0\",\"license_terms\":\"\"},{\"type\":\"cc\",\"description\":\"Sampling methods\",\"author\":\"OCLPhase2\\'s channel\",\"organization\":\"\",\"url\":\"https:\/\/youtu.be\/sk4TDJU7QbY\",\"project\":\"\",\"license\":\"cc-by\",\"license_terms\":\"\"},{\"type\":\"cc\",\"description\":\"Question ID 6733, 6727\",\"author\":\"Lippman, David\",\"organization\":\"\",\"url\":\"\",\"project\":\"\",\"license\":\"cc-by\",\"license_terms\":\"IMathAS Community License CC-BY +GPL\"},{\"type\":\"original\",\"description\":\"Revision and Adaptation\",\"author\":\"\",\"organization\":\"Lumen Learning\",\"url\":\"\",\"project\":\"\",\"license\":\"cc-by\",\"license_terms\":\"\"},{\"type\":\"cc\",\"description\":\"lab-research-chemistry-test\",\"author\":\"PublicDomainPictures\",\"organization\":\"\",\"url\":\"https:\/\/pixabay.com\/en\/lab-research-chemistry-test-217041\/\",\"project\":\"\",\"license\":\"cc0\",\"license_terms\":\"\"},{\"type\":\"cc\",\"description\":\"Experiments\",\"author\":\"OCLPhase2\\'s channel\",\"organization\":\"\",\"url\":\"https:\/\/youtu.be\/HSTTKzsdHEw\",\"project\":\"\",\"license\":\"cc-by\",\"license_terms\":\"\"},{\"type\":\"cc\",\"description\":\"Confounding\",\"author\":\"OCLPhase2\\'s channel\",\"organization\":\"\",\"url\":\"https:\/\/youtu.be\/SrCm12HZay0\",\"project\":\"\",\"license\":\"cc-by\",\"license_terms\":\"\"},{\"type\":\"cc\",\"description\":\"Controlled Experiments\",\"author\":\"OCLPhase2\\'s channel\",\"organization\":\"\",\"url\":\"https:\/\/youtu.be\/UkCHUeqMb5Y\",\"project\":\"\",\"license\":\"cc-by\",\"license_terms\":\"\"},{\"type\":\"cc\",\"description\":\"Blind Experiments\",\"author\":\"OCLPhase2\\'s channel\",\"organization\":\"https:\/\/youtu.be\/7BFZVGCxeYc\",\"url\":\"\",\"project\":\"\",\"license\":\"cc-by\",\"license_terms\":\"\"},{\"type\":\"cc\",\"description\":\"Question ID 6736, 6728, 6914\",\"author\":\"Lippman, David\",\"organization\":\"\",\"url\":\"\",\"project\":\"\",\"license\":\"cc-by\",\"license_terms\":\"IMathAS Community License CC-BY + GPL\"}]","CANDELA_OUTCOMES_GUID":"","pb_show_title":"on","pb_short_title":"","pb_subtitle":"","pb_authors":[],"pb_section_license":""},"chapter-type":[],"contributor":[],"license":[],"class_list":["post-800","chapter","type-chapter","status-publish","hentry"],"part":784,"_links":{"self":[{"href":"https:\/\/courses.lumenlearning.com\/suny-hccc-ma-124-1\/wp-json\/pressbooks\/v2\/chapters\/800","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/courses.lumenlearning.com\/suny-hccc-ma-124-1\/wp-json\/pressbooks\/v2\/chapters"}],"about":[{"href":"https:\/\/courses.lumenlearning.com\/suny-hccc-ma-124-1\/wp-json\/wp\/v2\/types\/chapter"}],"author":[{"embeddable":true,"href":"https:\/\/courses.lumenlearning.com\/suny-hccc-ma-124-1\/wp-json\/wp\/v2\/users\/21"}],"version-history":[{"count":2,"href":"https:\/\/courses.lumenlearning.com\/suny-hccc-ma-124-1\/wp-json\/pressbooks\/v2\/chapters\/800\/revisions"}],"predecessor-version":[{"id":1140,"href":"https:\/\/courses.lumenlearning.com\/suny-hccc-ma-124-1\/wp-json\/pressbooks\/v2\/chapters\/800\/revisions\/1140"}],"part":[{"href":"https:\/\/courses.lumenlearning.com\/suny-hccc-ma-124-1\/wp-json\/pressbooks\/v2\/parts\/784"}],"metadata":[{"href":"https:\/\/courses.lumenlearning.com\/suny-hccc-ma-124-1\/wp-json\/pressbooks\/v2\/chapters\/800\/metadata\/"}],"wp:attachment":[{"href":"https:\/\/courses.lumenlearning.com\/suny-hccc-ma-124-1\/wp-json\/wp\/v2\/media?parent=800"}],"wp:term":[{"taxonomy":"chapter-type","embeddable":true,"href":"https:\/\/courses.lumenlearning.com\/suny-hccc-ma-124-1\/wp-json\/pressbooks\/v2\/chapter-type?post=800"},{"taxonomy":"contributor","embeddable":true,"href":"https:\/\/courses.lumenlearning.com\/suny-hccc-ma-124-1\/wp-json\/wp\/v2\/contributor?post=800"},{"taxonomy":"license","embeddable":true,"href":"https:\/\/courses.lumenlearning.com\/suny-hccc-ma-124-1\/wp-json\/wp\/v2\/license?post=800"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}