{"id":936,"date":"2017-05-11T17:18:24","date_gmt":"2017-05-11T17:18:24","guid":{"rendered":"https:\/\/courses.lumenlearning.com\/suny-natural-resources-biometrics\/chapter\/chapter-8-multiple-linear-regression\/"},"modified":"2017-05-11T18:31:38","modified_gmt":"2017-05-11T18:31:38","slug":"chapter-8-multiple-linear-regression","status":"publish","type":"chapter","link":"https:\/\/courses.lumenlearning.com\/suny-natural-resources-biometrics\/chapter\/chapter-8-multiple-linear-regression\/","title":{"raw":"Chapter 8: Multiple Linear Regression","rendered":"Chapter 8: Multiple Linear Regression"},"content":{"raw":"
\r\n\r\nIt frequently happens that a dependent variable (y<\/em>) in which we are interested is related to more than one independent variable. If this relationship can be estimated, it may enable us to make more precise predictions of the dependent variable than would be possible by a simple linear regression. Regressions based on more than one independent variable are called multiple regressions<\/strong>.\r\n\r\nMultiple linear regression is an extension of simple linear regression and many of the ideas we examined in simple linear regression carry over to the multiple regression setting. For example, scatterplots, correlation, and least squares method are still essential components for a multiple regression.\r\n\r\nFor example, a habitat suitability index (used to evaluate the impact on wildlife habitat from land use changes) for ruffed grouse might be related to three factors:\r\n

x<\/strong>1<\/span><\/sub> = stem density\r\nx<\/strong>2<\/span><\/sub> = percent of conifers\r\nx<\/strong>3<\/span><\/sub> = amount of understory herbaceous matter<\/p>\r\nA researcher would collect data on these variables and use the sample data to construct a regression equation relating these three variables to the response. The researcher will have questions about his model similar to a simple linear regression model.\r\n

    \r\n \t
  • How strong is the relationship between y and the three predictor variables?<\/li>\r\n \t
  • How well does the model fit?<\/li>\r\n \t
  • Have any important assumptions been violated?<\/li>\r\n \t
  • How good are the estimates and predictions?<\/li>\r\n<\/ul>\r\nThe general linear regression model takes the form of\r\n

    \"13323.png\"<\/span>,<\/p>\r\nwith the mean value of y given as\r\n

    \"13331.png\"<\/span>,<\/p>\r\nwhere:\r\n

      \r\n \t
    • y is the random response variable and \u03bc<\/span>y<\/span> is the mean value of y,<\/em><\/li>\r\n \t
    • \u03b2<\/span>0<\/span>, \u03b2<\/span>1<\/span>, \u03b2<\/span>2<\/span>, and \u03b2<\/span>k<\/span> are the parameters to be estimated based on the sample data,<\/li>\r\n \t
    • x<\/em>1<\/span>, x<\/em>2<\/span>,\u2026, x<\/em>k<\/span> are the predictor variables that are assumed to be non-random or fixed and measured without error, and k is the number of predictor variable,<\/li>\r\n \t
    • and \u03b5<\/span> is the random error, which allows each response to deviate from the average value of y<\/em>. The errors are assumed to be independent, have a mean of zero and a common variance (\u03c3<\/span>2<\/span>), and are normally distributed.<\/li>\r\n<\/ul>\r\nAs you can see, the multiple regression model and assumptions are very similar to those for a simple linear regression model with one predictor variable. Examining residual plots and normal probability plots for the residuals is key to verifying the assumptions.\r\n

      Correlation<\/h2>\r\nAs with simple linear regression, we should always begin with a scatterplot of the response variable versus each predictor variable. Linear correlation coefficients for each pair should also be computed. Instead of computing the correlation of each pair individually, we can create a correlation matrix, which shows the linear correlation between each pair of variables under consideration in a multiple linear regression model.\r\n\r\n[caption id=\"\" align=\"aligncenter\" width=\"511\"]\"13236.png\" Table 1. A correlation matrix.[\/caption]\r\n

      In this matrix, the upper value is the linear correlation coefficient and the lower value is the p-value for testing the null hypothesis that a correlation coefficient is equal to zero. This matrix allows us to see the strength and direction of the linear relationship between each predictor variable and the response variable, but also the relationship between the predictor variables. For example, y<\/em> and x1<\/em> have a strong, positive linear relationship with r = 0.816, which is statistically significant because p = 0.000. We can also see that predictor variables x1<\/em> and x3<\/em> have a moderately strong positive linear relationship (r = 0.588) that is significant (p = 0.001).<\/p>\r\nThere are many different reasons for selecting which explanatory variables to include in our model (see Model Development and Selection), however, we frequently choose the ones that have a high linear correlation with the response variable, but we must be careful. We do not want to include explanatory variables that are highly correlated among themselves. We need to be aware of any multicollinearity between predictor variables.\r\n

      Multicollinearity<\/strong> exists between two explanatory variables if they have a strong linear relationship.<\/span><\/p>\r\nFor example, if we are trying to predict a person\u2019s blood pressure, one predictor variable would be weight and another predictor variable would be diet. Both predictor variables are highly correlated with blood pressure (as weight increases blood pressure typically increases, and as diet increases blood pressure also increases). But, both predictor variables are also highly correlated with each other. Both of these predictor variables are conveying essentially the same information when it comes to explaining blood pressure. Including both in the model may lead to problems when estimating the coefficients, as multicollinearity increases the standard errors of the coefficients. This means that coefficients for some variables may be found not<\/strong> to be significantly different from zero, whereas without multicollinearity and with lower standard errors, the same coefficients might have been found significant. Ways to test for multicollinearity are not covered in this text, however a general rule of thumb is to be wary of a linear correlation of less than -0.7 and greater than 0.7 between two predictor variables. Always examine the correlation matrix for relationships between predictor variables to avoid multicollinearity issues.\r\n

      Estimation<\/h2>\r\nEstimation and inference procedures are also very similar to simple linear regression. Just as we used our sample data to estimate \u03b2<\/span>0<\/span> and \u03b2<\/span>1<\/span> for our simple linear regression model, we are going to extend this process to estimate all the coefficients for our multiple regression models.\r\n\r\nWith the simpler population model\r\n

      \"13364.png\"<\/span>x<\/em><\/p>\r\n\u03b2<\/span>1<\/span> is the slope and tells the user what the change in the response would be as the predictor variable changes. With multiple predictor variables, and therefore multiple parameters to estimate, the coefficients \u03b2<\/span>1,<\/span> \u03b2<\/span>2<\/span>,<\/em> \u03b2<\/span>3<\/span> and so on are called partial slopes or partial regression coefficients. The partial slope \u03b2<\/span>i<\/span> measures the change in y<\/em> for a one-unit change in x<\/em>i<\/span> when all other independent variables are held constant.<\/strong> These regression coefficients must be estimated from the sample data in order to obtain the general form of the estimated multiple regression equation\r\n

      \"13373.png\"<\/p>\r\nand the population model\r\n

      \"13386.png\"<\/p>\r\nwhere k<\/em> = the number of independent variables (also called predictor variables)\r\n

      y\u0302<\/em><\/span> = the predicted value of the dependent variable (computed by using the multiple regression equation)<\/p>\r\n

      x<\/em>1<\/span>, x<\/em>2<\/span>, \u2026, x<\/em>k<\/span> = the independent variables<\/p>\r\n

      \u03b2<\/span>0<\/span> is the y-intercept (the value of y when all the predictor variables equal 0)<\/p>\r\n

      b<\/em>0<\/span> is the estimate of \u03b2<\/span>0<\/span> based on that sample data<\/p>\r\n

      \u03b2<\/span>1<\/span>,<\/em> \u03b2<\/span>2<\/span>,<\/em> \u03b2<\/span>3<\/span>,\u2026<\/em>\u03b2<\/span>k<\/span> are the coefficients of the independent variables x<\/em>1<\/span>, x<\/em>2<\/span>, \u2026, x<\/em>k<\/span><\/p>\r\n

      b<\/em>1<\/span>, b<\/em>2<\/span>, b<\/em>3<\/span>, \u2026, b<\/em>k<\/span> are the sample estimates of the coefficients \u03b2<\/span>1<\/span>,<\/em> \u03b2<\/span>2<\/span>,<\/em> \u03b2<\/span>3<\/span>,\u2026<\/em>\u03b2<\/span>k<\/span><\/p>\r\nThe method of least-squares is still used to fit the model to the data. Remember that this method minimizes the sum of the squared deviations of the observed and predicted values (SSE).\r\n\r\nThe analysis of variance table for multiple regression has a similar appearance to that of a simple linear regression.\r\n\r\n[caption id=\"\" align=\"aligncenter\" width=\"901\"]\"13226.png\" Table 2. ANOVA table.[\/caption]\r\n\r\nWhere k is the number of predictor variables and n is the number of observations.\r\n\r\nThe best estimate of the random variation \u03c3<\/span>2<\/span>\u2014the variation that is unexplained by the predictor variables\u2014is still s2<\/span>, the MSE. The regression standard error, s, is the square root of the MSE.\r\n\r\nA new column in the ANOVA table for multiple linear regression shows a decomposition of SSR, in which the conditional contribution of each predictor variable given the variables already entered into the model<\/em> is shown for the order of entry that you specify in your regression. These conditional or sequential sums of squares<\/strong> each account for 1 regression degree of freedom, and allow the user to see the contribution of each predictor variable to the total variation explained by the regression model by using the ratio:\r\n

      \"13413.png\"<\/span><\/p>\r\n\r\n

      Adjusted R2<\/span><\/h2>\r\nIn simple linear regression, we used the relationship between the explained and total variation as a measure of model fit:\r\n

      \"13422.png\"<\/p>\r\nNotice from this definition that the value of the coefficient of determination can never decrease with the addition of more variables into the regression model. Hence, R2<\/span> can be artificially inflated as more variables (significant or not) are included in the model. An alternative measure of strength of the regression model is adjusted for degrees of freedom by using mean squares rather than sums of squares:\r\n

      \"13429.png\"<\/p>\r\nThe adjusted R2<\/span> value represents the percentage of variation in the response variable explained by the independent variables, corrected for degrees of freedom. Unlike R2<\/span>, the adjusted R2<\/span> will not tend to increase as variables are added and it will tend to stabilize around some upper limit as variables are added.\r\n

      Tests of Significance<\/h2>\r\nRecall in the previous chapter we tested to see if y<\/em> and x<\/em> were linearly related by testing\r\n<\/colgroup>\r\n\r\n\r\n
      \r\n

      H0<\/span>: \u03b2<\/span>1<\/span> = 0<\/p>\r\n<\/td>\r\n

      		\r\n

      H1<\/span>: \u03b2<\/span>1<\/span> \u2260 0<\/p>\r\n<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\nwith the t-test (or the equivalent F-test). In multiple linear regression, there are several partial slopes and the t-test and F-test are no longer equivalent. Our question changes: Is the regression equation that uses information provided by the predictor variables x1<\/span>, x2<\/span>, x3<\/span>, \u2026, xk<\/span>, better than the simple predictor \"13615.png\"<\/span>(the mean response value), which does not rely on any of these independent variables?\r\n

      H0<\/span>: \u03b2<\/span>1<\/span> = \u03b2<\/span>2<\/span> = \u03b2<\/span>3<\/span> = \u2026=\u03b2<\/span>k<\/span> = 0<\/p>\r\n

      H1<\/span>: At least one of \u03b2<\/span>1<\/span>, \u03b2<\/span>2<\/span> , \u03b2<\/span>3<\/span> , \u2026\u03b2<\/span>k<\/span> \u2260 0<\/p>\r\nThe F-test statistic is used to answer this question and is found in the ANOVA table.\r\n

      \"13437.png\"<\/p>\r\nThis test statistic follows the F-distribution with df1<\/span> = k and df2<\/span> = (n-k-1). Since the exact p-value is given in the output, you can use the Decision Rule to answer the question.\r\n

      If the p-value is less than the level of significance, reject the null hypothesis.<\/span><\/p>\r\nRejecting the null hypothesis supports the claim that at least one of the predictor variables has a significant linear relationship with the response variable. The next step is to determine which predictor variables add important information for prediction in the presence of other predictors already in the model. To test the significance of the partial regression coefficients, you need to examine each relationship separately using individual t-tests.\r\n<\/colgroup>\r\n\r\n\r\n
      \r\n

      H0<\/span>: \u03b2<\/span>i<\/span> = 0<\/p>\r\n<\/td>\r\n

      		\r\n

      H1<\/span>: \u03b2<\/span>i<\/span> \u2260 0<\/p>\r\n<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n

      \r\n\"13447.png\"<\/span> with df = (n-k-1)<\/p>\r\nwhere SE(bi<\/sub>)<\/em> is the standard error of bi<\/span><\/sub><\/em>. Exact p-values are also given for these tests. Examining specific p-values for each predictor variable will allow you to decide which variables are significantly related to the response variable. Typically, any insignificant variables are removed from the model, but remember these tests are done with other variables in the model. A good procedure is to remove the least significant variable and then refit the model with the reduced data set. With each new model, always check the regression standard error (lower is better), the adjusted R2<\/span><\/sup> (higher is better), the p-values for all predictor variables, and the residual and normal probability plots.\r\n\r\nBecause of the complexity of the calculations, we will rely on software to fit the model and give us the regression coefficients. Don\u2019t forget\u2026 you always begin with scatterplots. Strong relationships between predictor and response variables make for a good model.\r\n

      \r\n

      Example 1<\/h3>\r\nA researcher collected data in a project to predict the annual growth per acre of upland boreal forests in southern Canada. They hypothesized that cubic foot volume growth (y<\/em>) is a function of stand basal area per acre (x<\/em>1<\/span>), the percentage of that basal area in black spruce (x<\/em>2<\/span>), and the stand\u2019s site index for black spruce (x<\/em>3<\/span>). \u03b1<\/span> = 0.05.\r\n\r\n<\/div>\r\n\r\n[caption id=\"\" align=\"aligncenter\" width=\"901\"]\"132151.png\" Table 3. Observed data for cubic feet, stand basal area, percent basal area in black spruce, and site index.[\/caption]\r\n\r\nScatterplots of the response variable versus each predictor variable were created along with a correlation matrix.\r\n\r\n[caption id=\"\" align=\"aligncenter\" width=\"901\"]\"13205.png\" Figure 1. Scatterplots of cubic feet versus basal area, percent basal area in black spruce, and site index.[\/caption]\r\n\r\n[caption id=\"\" align=\"aligncenter\" width=\"589\"]\"13195.png\" Table 4. Correlation matrix.[\/caption]\r\n\r\nAs you can see from the scatterplots and the correlation matrix, BA\/ac has the strongest linear relationship with CuFt volume (r = 0.816) and %BA in black spruce has the weakest linear relationship (r = 0.413). Also of note is the moderately strong correlation between the two predictor variables, BA\/ac and SI (r = 0.588). All three predictor variables have significant linear relationships with the response variable (volume) so we will begin by using all variables in our multiple linear regression model. The Minitab output is given below.\r\n\r\nWe begin by testing the following null and alternative hypotheses:\r\n

      H0<\/span><\/sub>: \u03b2<\/span>1<\/span><\/sub> = \u03b2<\/span>2<\/span><\/sub> = \u03b2<\/span>3<\/span><\/sub> = 0<\/p>\r\n

      H1<\/span><\/sub>: At least one of \u03b2<\/span>1<\/span><\/sub>, \u03b2<\/span>2<\/span><\/sub> , \u03b2<\/span>3<\/span><\/sub> \u2260 0<\/p>\r\n\r\n

      General Regression Analysis: CuFt versus BA\/ac, SI, %BA Bspruce<\/h4>\r\n<\/colgroup>\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n
      \r\n

      Regression Equation<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      CuFt = -19.3858 + 0.591004 BA\/ac + 0.0899883 SI + 0.489441 %BA Bspruce<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      Coefficients<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      Term<\/p>\r\n<\/td>\r\n

      		\r\n

      Coef<\/p>\r\n<\/td>\r\n

      		\r\n

      SE Coef<\/p>\r\n<\/td>\r\n

      		\r\n

      T<\/p>\r\n<\/td>\r\n

      		\r\n

      P<\/p>\r\n<\/td>\r\n

      		\r\n

      95% CI<\/p>\r\n<\/td>\r\n

      		<\/td>\r\n<\/tr>\r\n
      \r\n

      Constant<\/p>\r\n<\/td>\r\n

      		\r\n

      -19.3858<\/p>\r\n<\/td>\r\n

      		\r\n

      4.15332<\/p>\r\n<\/td>\r\n

      		\r\n

      -4.6675<\/p>\r\n<\/td>\r\n

      		\r\n

      0.000<\/p>\r\n<\/td>\r\n

      		\r\n

      (-27.9578, -10.8137)<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      BA\/ac<\/p>\r\n<\/td>\r\n

      		\r\n

      0.5910<\/p>\r\n<\/td>\r\n

      		\r\n

      0.04294<\/p>\r\n<\/td>\r\n

      		\r\n

      13.7647<\/p>\r\n<\/td>\r\n

      		\r\n

      0.000<\/p>\r\n<\/td>\r\n

      		\r\n

      (0.5024, 0.6796)<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      SI<\/p>\r\n<\/td>\r\n

      		\r\n

      0.0900<\/p>\r\n<\/td>\r\n

      		\r\n

      0.11262<\/p>\r\n<\/td>\r\n

      		\r\n

      0.7991<\/p>\r\n<\/td>\r\n

      		\r\n

      0.432<\/p>\r\n<\/td>\r\n

      		\r\n

      (-0.1424, 0.3224)<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      %BA Bspruce<\/p>\r\n<\/td>\r\n

      		\r\n

      0.4894<\/p>\r\n<\/td>\r\n

      		\r\n

      0.05245<\/p>\r\n<\/td>\r\n

      		\r\n

      9.3311<\/p>\r\n<\/td>\r\n

      		\r\n

      0.000<\/p>\r\n<\/td>\r\n

      		\r\n

      (0.3812, 0.5977)<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      Summary of Model<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      S = 3.17736<\/p>\r\n<\/td>\r\n

      		\r\n

      R-Sq = 95.53%<\/p>\r\n<\/td>\r\n

      		\r\n

      R-Sq(adj) = 94.97%<\/p>\r\n<\/td>\r\n

      		<\/td>\r\n<\/tr>\r\n
      \r\n

      PRESS = 322.279<\/p>\r\n<\/td>\r\n

      		\r\n

      R-Sq(pred) = 94.05%<\/p>\r\n<\/td>\r\n

      		<\/td>\r\n<\/td>\r\n<\/td>\r\n<\/tr>\r\n
      \r\n

      Analysis of Variance<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      Source<\/p>\r\n<\/td>\r\n

      		\r\n

      DF<\/p>\r\n<\/td>\r\n

      		\r\n

      Seq SS<\/p>\r\n<\/td>\r\n

      		\r\n

      Adj SS<\/p>\r\n<\/td>\r\n

      		\r\n

      Adj MS<\/p>\r\n<\/td>\r\n

      		\r\n

      F<\/p>\r\n<\/td>\r\n

      		\r\n

      P<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      Regression<\/p>\r\n<\/td>\r\n

      		\r\n

      3<\/p>\r\n<\/td>\r\n

      		\r\n

      5176.56<\/p>\r\n<\/td>\r\n

      		\r\n

      5176.56<\/p>\r\n<\/td>\r\n

      		\r\n

      1725.52<\/p>\r\n<\/td>\r\n

      		\r\n

      170.918<\/strong><\/p>\r\n<\/td>\r\n

      		\r\n

      0.000000<\/strong><\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      BA\/ac<\/p>\r\n<\/td>\r\n

      		\r\n

      1<\/p>\r\n<\/td>\r\n

      		\r\n

      3611.17<\/p>\r\n<\/td>\r\n

      		\r\n

      1912.79<\/p>\r\n<\/td>\r\n

      		\r\n

      1912.79<\/p>\r\n<\/td>\r\n

      		\r\n

      189.467<\/p>\r\n<\/td>\r\n

      		\r\n

      0.000000<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      SI<\/p>\r\n<\/td>\r\n

      		\r\n

      1<\/p>\r\n<\/td>\r\n

      		\r\n

      686.37<\/p>\r\n<\/td>\r\n

      		\r\n

      6.45<\/p>\r\n<\/td>\r\n

      		\r\n

      6.45<\/p>\r\n<\/td>\r\n

      		\r\n

      0.638<\/p>\r\n<\/td>\r\n

      		\r\n

      0.432094<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      %BA Bspruce<\/p>\r\n<\/td>\r\n

      		\r\n

      1<\/p>\r\n<\/td>\r\n

      		\r\n

      879.02<\/p>\r\n<\/td>\r\n

      		\r\n

      879.02<\/p>\r\n<\/td>\r\n

      		\r\n

      879.02<\/p>\r\n<\/td>\r\n

      		\r\n

      87.069<\/p>\r\n<\/td>\r\n

      		\r\n

      0.000000<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      Error<\/p>\r\n<\/td>\r\n

      		\r\n

      24<\/p>\r\n<\/td>\r\n

      		\r\n

      242.30<\/p>\r\n<\/td>\r\n

      		\r\n

      242.30<\/p>\r\n<\/td>\r\n

      		\r\n

      10.10<\/p>\r\n<\/td>\r\n

      		<\/td>\r\n<\/td>\r\n<\/tr>\r\n
      \r\n

      Total<\/p>\r\n<\/td>\r\n

      		\r\n

      27<\/p>\r\n<\/td>\r\n

      		\r\n

      5418.86<\/p>\r\n<\/td>\r\n

      		<\/td>\r\n<\/td>\r\n<\/td>\r\n<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\nThe F-test statistic (and associated p-value) is used to answer this question and is found in the ANOVA table. For this example, F = 170.918 with a p-value of 0.00000. The p-value is smaller than our level of significance (0.0000<0.05) so we will reject the null hypothesis. At least one of the predictor variables significantly contributes to the prediction of volume.\r\n\r\nThe coefficients for the three predictor variables are all positive indicating that as they increase cubic foot volume will also increase. For example, if we hold values of SI and %BA Bspruce constant, this equation tells us that as basal area increases by 1 sq. ft., volume will increase an additional 0.591004 cu. ft. The signs of these coefficients are logical, and what we would expect. The adjusted R2<\/span><\/sup> is also very high at 94.97%.\r\n\r\nThe next step is to examine the individual t-tests for each predictor variable. The test statistics and associated p-values are found in the Minitab output and repeated below:\r\n<\/colgroup>\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n
      \r\n

      Coefficients<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      Term<\/p>\r\n<\/td>\r\n

      		\r\n

      Coef<\/p>\r\n<\/td>\r\n

      		\r\n

      SE Coef<\/p>\r\n<\/td>\r\n

      		\r\n

      T<\/p>\r\n<\/td>\r\n

      		\r\n

      P<\/p>\r\n<\/td>\r\n

      		\r\n

      95% CI<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      Constant<\/p>\r\n<\/td>\r\n

      		\r\n

      -19.3858<\/p>\r\n<\/td>\r\n

      		\r\n

      4.15332<\/p>\r\n<\/td>\r\n

      		\r\n

      -4.6675<\/p>\r\n<\/td>\r\n

      		\r\n

      0.000<\/p>\r\n<\/td>\r\n

      		\r\n

      (-27.9578, -10.8137)<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      BA\/ac<\/p>\r\n<\/td>\r\n

      		\r\n

      0.5910<\/p>\r\n<\/td>\r\n

      		\r\n

      0.04294<\/p>\r\n<\/td>\r\n

      		\r\n

      13.7647<\/strong><\/p>\r\n<\/td>\r\n

      		\r\n

      0.000<\/strong><\/p>\r\n<\/td>\r\n

      		\r\n

      ( 0.5024, 0.6796)<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      SI<\/p>\r\n<\/td>\r\n

      		\r\n

      0.0900<\/p>\r\n<\/td>\r\n

      		\r\n

      0.11262<\/p>\r\n<\/td>\r\n

      		\r\n

      0.7991<\/strong><\/p>\r\n<\/td>\r\n

      		\r\n

      0.432<\/strong><\/p>\r\n<\/td>\r\n

      		\r\n

      ( -0.1424, 0.3224)<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      %BA Bspruce<\/p>\r\n<\/td>\r\n

      		\r\n

      0.4894<\/p>\r\n<\/td>\r\n

      		\r\n

      0.05245<\/p>\r\n<\/td>\r\n

      		\r\n

      9.3311<\/strong><\/p>\r\n<\/td>\r\n

      		\r\n

      0.000<\/strong><\/p>\r\n<\/td>\r\n

      		\r\n

      ( 0.3812, 0.5977)<\/p>\r\n<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\nThe predictor variables BA\/ac and %BA Bspruce have t-statistics of 13.7647 and 9.3311 and p-values of 0.0000, indicating that both are significantly contributing to the prediction of volume. However, SI has a t-statistic of 0.7991 with a p-value of 0.432. This variable does not significantly contribute to the prediction of cubic foot volume.\r\n\r\nThis result may surprise you as SI had the second strongest relationship with volume, but don\u2019t forget about the correlation between SI and BA\/ac (r = 0.588). The predictor variable BA\/ac had the strongest linear relationship with volume, and using the sequential sums of squares, we can see that BA\/ac is already accounting for 70% of the variation in cubic foot volume (3611.17\/5176.56 = 0.6976). The information from SI may be too similar to the information in BA\/ac, and SI only explains about 13% of the variation on volume (686.37\/5176.56 = 0.1326) given that BA\/ac is already in the model.\r\n\r\nThe next step is to examine the residual and normal probability plots. A single outlier is evident in the otherwise acceptable plots.\r\n\r\n[caption id=\"\" align=\"aligncenter\" width=\"1030\"]\"13186.png\" Figure 2. Residual and normal probability plots.[\/caption]\r\n\r\nSo where do we go from here?<\/strong>\r\n\r\nWe will remove the non-significant variable and re-fit the model excluding the data for SI in our model. The Minitab output is given below.\r\n

      General Regression Analysis: CuFt versus BA\/ac, %BA Bspruce<\/h4>\r\n<\/colgroup>\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n
      \r\n

      Regression Equation<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      CuFt = -19.1142 + 0.615531 BA\/ac + 0.515122 %BA Bspruce<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      Coefficients<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      Term<\/p>\r\n<\/td>\r\n

      		\r\n

      Coef<\/p>\r\n<\/td>\r\n

      		\r\n

      SE Coef<\/p>\r\n<\/td>\r\n

      		\r\n

      T<\/p>\r\n<\/td>\r\n

      		\r\n

      P<\/p>\r\n<\/td>\r\n

      		\r\n

      95% CI<\/p>\r\n<\/td>\r\n

      		<\/td>\r\n<\/tr>\r\n
      \r\n

      Constant<\/p>\r\n<\/td>\r\n

      		\r\n

      -19.1142<\/p>\r\n<\/td>\r\n

      		\r\n

      4.10936<\/p>\r\n<\/td>\r\n

      		\r\n

      -4.6514<\/p>\r\n<\/td>\r\n

      		\r\n

      0.000<\/p>\r\n<\/td>\r\n

      		\r\n

      (-27.5776, -10.6508)<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      BA\/ac<\/p>\r\n<\/td>\r\n

      		\r\n

      0.6155<\/p>\r\n<\/td>\r\n

      		\r\n

      0.02980<\/p>\r\n<\/td>\r\n

      		\r\n

      20.6523<\/p>\r\n<\/td>\r\n

      		\r\n

      0.000<\/p>\r\n<\/td>\r\n

      		\r\n

      (0.5541, 0.6769)<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      %BA Bspruce<\/p>\r\n<\/td>\r\n

      		\r\n

      0.5151<\/p>\r\n<\/td>\r\n

      		\r\n

      0.04115<\/p>\r\n<\/td>\r\n

      		\r\n

      12.5173<\/p>\r\n<\/td>\r\n

      		\r\n

      0.000<\/p>\r\n<\/td>\r\n

      		\r\n

      (0.4304, 0.5999)<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      Summary of Model<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      S = 3.15431<\/p>\r\n<\/td>\r\n

      		\r\n

      R-Sq = 95.41%<\/p>\r\n<\/td>\r\n

      		\r\n

      R-Sq(adj) = 95.04%<\/p>\r\n<\/td>\r\n

      		<\/td>\r\n<\/tr>\r\n
      \r\n

      PRESS = 298.712<\/p>\r\n<\/td>\r\n

      		\r\n

      R-Sq(pred) = 94.49%<\/p>\r\n<\/td>\r\n

      		<\/td>\r\n<\/td>\r\n<\/td>\r\n<\/tr>\r\n
      \r\n

      Analysis of Variance<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      Source<\/p>\r\n<\/td>\r\n

      		\r\n

      DF<\/p>\r\n<\/td>\r\n

      		\r\n

      SeqSS<\/p>\r\n<\/td>\r\n

      		\r\n

      AdjSS<\/p>\r\n<\/td>\r\n

      		\r\n

      AdjMS<\/p>\r\n<\/td>\r\n

      		\r\n

      F<\/p>\r\n<\/td>\r\n

      		\r\n

      P<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      Regression<\/p>\r\n<\/td>\r\n

      		\r\n

      2<\/p>\r\n<\/td>\r\n

      		\r\n

      5170.12<\/p>\r\n<\/td>\r\n

      		\r\n

      5170.12<\/p>\r\n<\/td>\r\n

      		\r\n

      2585.06<\/p>\r\n<\/td>\r\n

      		\r\n

      259.814<\/p>\r\n<\/td>\r\n

      		\r\n

      0.0000000<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      BA\/ac<\/p>\r\n<\/td>\r\n

      		\r\n

      1<\/p>\r\n<\/td>\r\n

      		\r\n

      3611.17<\/p>\r\n<\/td>\r\n

      		\r\n

      4243.71<\/p>\r\n<\/td>\r\n

      		\r\n

      4243.71<\/p>\r\n<\/td>\r\n

      		\r\n

      426.519<\/p>\r\n<\/td>\r\n

      		\r\n

      0.0000000<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      %BA Bspruce<\/p>\r\n<\/td>\r\n

      		\r\n

      1<\/p>\r\n<\/td>\r\n

      		\r\n

      1558.95<\/p>\r\n<\/td>\r\n

      		\r\n

      1558.95<\/p>\r\n<\/td>\r\n

      		\r\n

      1558.95<\/p>\r\n<\/td>\r\n

      		\r\n

      156.684<\/p>\r\n<\/td>\r\n

      		\r\n

      0.0000000<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      Error<\/p>\r\n<\/td>\r\n

      		\r\n

      25<\/p>\r\n<\/td>\r\n

      		\r\n

      248.74<\/p>\r\n<\/td>\r\n

      		\r\n

      248.74<\/p>\r\n<\/td>\r\n

      		\r\n

      9.95<\/p>\r\n<\/td>\r\n

      		<\/td>\r\n<\/td>\r\n<\/tr>\r\n
      \r\n

      Total<\/p>\r\n<\/td>\r\n

      		\r\n

      27<\/p>\r\n<\/td>\r\n

      		\r\n

      5418.86<\/p>\r\n<\/td>\r\n

      		<\/td>\r\n<\/td>\r\n<\/td>\r\n<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\nWe will repeat the steps followed with our first model. We begin by again testing the following hypotheses:\r\n

      H0<\/span>: \u03b2<\/span>1<\/span> = \u03b2<\/span>2<\/span> = \u03b2<\/span>3<\/span> = 0<\/p>\r\n

      H1<\/span>: At least one of \u03b2<\/span>1<\/span>, \u03b2<\/span>2<\/span> , \u03b2<\/span>3<\/span> \u2260 0<\/p>\r\nThis reduced model has an F-statistic equal to 259.814 and a p-value of 0.0000. We will reject the null hypothesis. At least one of the predictor variables significantly contributes to the prediction of volume. The coefficients are still positive (as we expected) but the values have changed to account for the different model.\r\n\r\nThe individual t-tests for each coefficient (repeated below) show that both predictor variables are significantly different from zero and contribute to the prediction of volume.\r\n<\/colgroup>\r\n\r\n\r\n\r\n\r\n\r\n\r\n
      \r\n

      Coefficients<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      Term<\/p>\r\n<\/td>\r\n

      		\r\n

      Coef<\/p>\r\n<\/td>\r\n

      		\r\n

      SE Coef<\/p>\r\n<\/td>\r\n

      		\r\n

      T<\/p>\r\n<\/td>\r\n

      		\r\n

      P<\/p>\r\n<\/td>\r\n

      		\r\n

      95% CI<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      Constant<\/p>\r\n<\/td>\r\n

      		\r\n

      -19.1142<\/p>\r\n<\/td>\r\n

      		\r\n

      4.10936<\/p>\r\n<\/td>\r\n

      		\r\n

      -4.6514<\/p>\r\n<\/td>\r\n

      		\r\n

      0.000<\/p>\r\n<\/td>\r\n

      		\r\n

      (-27.5776, -10.6508)<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      BA\/ac<\/p>\r\n<\/td>\r\n

      		\r\n

      0.6155<\/p>\r\n<\/td>\r\n

      		\r\n

      0.02980<\/p>\r\n<\/td>\r\n

      		\r\n

      20.6523<\/p>\r\n<\/td>\r\n

      		\r\n

      0.000<\/p>\r\n<\/td>\r\n

      		\r\n

      ( 0.5541, 0.6769)<\/p>\r\n<\/td>\r\n<\/tr>\r\n

      \r\n

      %BA Bspruce<\/p>\r\n<\/td>\r\n

      		\r\n

      0.5151<\/p>\r\n<\/td>\r\n

      		\r\n

      0.04115<\/p>\r\n<\/td>\r\n

      		\r\n

      12.5173<\/p>\r\n<\/td>\r\n

      		\r\n

      0.000<\/p>\r\n<\/td>\r\n

      		\r\n

      ( 0.4304, 0.5999)<\/p>\r\n<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\nNotice that the adjusted R2<\/span> has increased from 94.97% to 95.04% indicating a slightly better fit to the data. The regression standard error has also changed for the better, decreasing from 3.17736 to 3.15431 indicating slightly less variation of the observed data to the model.\r\n\r\n[caption id=\"\" align=\"aligncenter\" width=\"1040\"]\"131751.png\" Figure 3. Residual and normal probability plots.[\/caption]\r\n\r\nThe residual and normal probability plots have changed little, still not indicating any issues with the regression assumption. By removing the non-significant variable, the model has improved.\r\n

      Model Development and Selection<\/h2>\r\nThere are many different reasons for creating a multiple linear regression model and its purpose directly influences how the model is created. Listed below are several of the more commons uses for a regression model:\r\n
        \r\n \t
      1. Describing the behavior of your response variable<\/li>\r\n \t
      2. Predicting a response or estimating the average response<\/li>\r\n \t
      3. Estimating the parameters (\u03b2<\/span>0<\/span>, \u03b2<\/span>1<\/span>, \u03b2<\/span>2<\/span>, \u2026)<\/li>\r\n \t
      4. Developing an accurate model of the process<\/li>\r\n<\/ol>\r\nDepending on your objective for creating a regression model, your methodology may vary when it comes to variable selection, retention, and elimination.\r\n\r\nWhen the object is simple description of your response variable, you are typically less concerned about eliminating non-significant variables. The best representation of the response variable, in terms of minimal residual sums of squares, is the full model, which includes all predictor variables available from the data set. It is less important that the variables are causally related or that the model is realistic.\r\n\r\nA common reason for creating a regression model is for prediction and estimating. A researcher wants to be able to define events within the x-space of data that were collected for this model, and it is assumed that the system will continue to function as it did when the data were collected. Any measurable predictor variables that contain information on the response variable should be included. For this reason, non-significant variables may be retained in the model. However, regression equations with fewer variables are easier to use and have an economic advantage in terms of data collection. Additionally, there is a greater confidence attached to models that contain only significant variables.\r\n\r\nIf the objective is to estimate the model parameters, you will be more cautious when considering variable elimination. You want to avoid introducing a bias by removing a variable that has predictive information about the response. However, there is a statistical advantage in terms of reduced variance of the parameter estimates if variables truly unrelated to the response variable are removed.\r\n\r\nBuilding a realistic model of the process you are studying is often a primary goal of much research. It is important to identify the variables that are linked to the response through some causal relationship. While you can identify which variables have a strong correlation with the response, this only serves as an indicator of which variables require further study. The principal objective is to develop a model whose functional form realistically reflects the behavior of a system.\r\n\r\nThe following figure is a strategy for building a regression model.\r\n\r\n[caption id=\"\" align=\"aligncenter\" width=\"370\"]\"153_1.tif\" Figure 4. Strategy for building a regression model.[\/caption]\r\n

        Software Solutions<\/h2>\r\n

        Minitab<\/h3>\r\n

        \"155_1.tif\"<\/span><\/p>\r\n

        \"155_2.tif\"<\/span><\/p>\r\n

        \"155_3.tif\"<\/p>\r\nThe output and plots are given in the previous example.\r\n

        Excel<\/h3>\r\n

        \"154_1.tif\"<\/span><\/p>\r\n

        \"154_2.tif\"<\/span><\/p>\r\n

        \"154_3.tif\"<\/span><\/p>\r\n

        \"154_4.tif\"<\/span><\/p>\r\n\r\n<\/div>","rendered":"

        \n

        It frequently happens that a dependent variable (y<\/em>) in which we are interested is related to more than one independent variable. If this relationship can be estimated, it may enable us to make more precise predictions of the dependent variable than would be possible by a simple linear regression. Regressions based on more than one independent variable are called multiple regressions<\/strong>.<\/p>\n

        Multiple linear regression is an extension of simple linear regression and many of the ideas we examined in simple linear regression carry over to the multiple regression setting. For example, scatterplots, correlation, and least squares method are still essential components for a multiple regression.<\/p>\n

        For example, a habitat suitability index (used to evaluate the impact on wildlife habitat from land use changes) for ruffed grouse might be related to three factors:<\/p>\n

        x<\/strong>1<\/span><\/sub> = stem density
        \nx<\/strong>2<\/span><\/sub> = percent of conifers
        \nx<\/strong>3<\/span><\/sub> = amount of understory herbaceous matter<\/p>\n

        A researcher would collect data on these variables and use the sample data to construct a regression equation relating these three variables to the response. The researcher will have questions about his model similar to a simple linear regression model.<\/p>\n

          \n
        • How strong is the relationship between y and the three predictor variables?<\/li>\n
        • How well does the model fit?<\/li>\n
        • Have any important assumptions been violated?<\/li>\n
        • How good are the estimates and predictions?<\/li>\n<\/ul>\n

          The general linear regression model takes the form of<\/p>\n

          \"13323.png\"<\/span>,<\/p>\n

          with the mean value of y given as<\/p>\n

          \"13331.png\"<\/span>,<\/p>\n

          where:<\/p>\n

            \n
          • y is the random response variable and \u03bc<\/span>y<\/span> is the mean value of y,<\/em><\/li>\n
          • \u03b2<\/span>0<\/span>, \u03b2<\/span>1<\/span>, \u03b2<\/span>2<\/span>, and \u03b2<\/span>k<\/span> are the parameters to be estimated based on the sample data,<\/li>\n
          • x<\/em>1<\/span>, x<\/em>2<\/span>,\u2026, x<\/em>k<\/span> are the predictor variables that are assumed to be non-random or fixed and measured without error, and k is the number of predictor variable,<\/li>\n
          • and \u03b5<\/span> is the random error, which allows each response to deviate from the average value of y<\/em>. The errors are assumed to be independent, have a mean of zero and a common variance (\u03c3<\/span>2<\/span>), and are normally distributed.<\/li>\n<\/ul>\n

            As you can see, the multiple regression model and assumptions are very similar to those for a simple linear regression model with one predictor variable. Examining residual plots and normal probability plots for the residuals is key to verifying the assumptions.<\/p>\n

            Correlation<\/h2>\n

            As with simple linear regression, we should always begin with a scatterplot of the response variable versus each predictor variable. Linear correlation coefficients for each pair should also be computed. Instead of computing the correlation of each pair individually, we can create a correlation matrix, which shows the linear correlation between each pair of variables under consideration in a multiple linear regression model.<\/p>\n

            \"13236.png\"<\/p>\n

            Table 1. A correlation matrix.<\/p>\n<\/div>\n

            In this matrix, the upper value is the linear correlation coefficient and the lower value is the p-value for testing the null hypothesis that a correlation coefficient is equal to zero. This matrix allows us to see the strength and direction of the linear relationship between each predictor variable and the response variable, but also the relationship between the predictor variables. For example, y<\/em> and x1<\/em> have a strong, positive linear relationship with r = 0.816, which is statistically significant because p = 0.000. We can also see that predictor variables x1<\/em> and x3<\/em> have a moderately strong positive linear relationship (r = 0.588) that is significant (p = 0.001).<\/p>\n

            There are many different reasons for selecting which explanatory variables to include in our model (see Model Development and Selection), however, we frequently choose the ones that have a high linear correlation with the response variable, but we must be careful. We do not want to include explanatory variables that are highly correlated among themselves. We need to be aware of any multicollinearity between predictor variables.<\/p>\n

            Multicollinearity<\/strong> exists between two explanatory variables if they have a strong linear relationship.<\/span><\/p>\n

            For example, if we are trying to predict a person\u2019s blood pressure, one predictor variable would be weight and another predictor variable would be diet. Both predictor variables are highly correlated with blood pressure (as weight increases blood pressure typically increases, and as diet increases blood pressure also increases). But, both predictor variables are also highly correlated with each other. Both of these predictor variables are conveying essentially the same information when it comes to explaining blood pressure. Including both in the model may lead to problems when estimating the coefficients, as multicollinearity increases the standard errors of the coefficients. This means that coefficients for some variables may be found not<\/strong> to be significantly different from zero, whereas without multicollinearity and with lower standard errors, the same coefficients might have been found significant. Ways to test for multicollinearity are not covered in this text, however a general rule of thumb is to be wary of a linear correlation of less than -0.7 and greater than 0.7 between two predictor variables. Always examine the correlation matrix for relationships between predictor variables to avoid multicollinearity issues.<\/p>\n

            Estimation<\/h2>\n

            Estimation and inference procedures are also very similar to simple linear regression. Just as we used our sample data to estimate \u03b2<\/span>0<\/span> and \u03b2<\/span>1<\/span> for our simple linear regression model, we are going to extend this process to estimate all the coefficients for our multiple regression models.<\/p>\n

            With the simpler population model<\/p>\n

            \"13364.png\"<\/span>x<\/em><\/p>\n

            \u03b2<\/span>1<\/span> is the slope and tells the user what the change in the response would be as the predictor variable changes. With multiple predictor variables, and therefore multiple parameters to estimate, the coefficients \u03b2<\/span>1,<\/span> \u03b2<\/span>2<\/span>,<\/em> \u03b2<\/span>3<\/span> and so on are called partial slopes or partial regression coefficients. The partial slope \u03b2<\/span>i<\/span> measures the change in y<\/em> for a one-unit change in x<\/em>i<\/span> when all other independent variables are held constant.<\/strong> These regression coefficients must be estimated from the sample data in order to obtain the general form of the estimated multiple regression equation<\/p>\n

            \"13373.png\"<\/p>\n

            and the population model<\/p>\n

            \"13386.png\"<\/p>\n

            where k<\/em> = the number of independent variables (also called predictor variables)<\/p>\n

            y\u0302<\/em><\/span> = the predicted value of the dependent variable (computed by using the multiple regression equation)<\/p>\n

            x<\/em>1<\/span>, x<\/em>2<\/span>, \u2026, x<\/em>k<\/span> = the independent variables<\/p>\n

            \u03b2<\/span>0<\/span> is the y-intercept (the value of y when all the predictor variables equal 0)<\/p>\n

            b<\/em>0<\/span> is the estimate of \u03b2<\/span>0<\/span> based on that sample data<\/p>\n

            \u03b2<\/span>1<\/span>,<\/em> \u03b2<\/span>2<\/span>,<\/em> \u03b2<\/span>3<\/span>,\u2026<\/em>\u03b2<\/span>k<\/span> are the coefficients of the independent variables x<\/em>1<\/span>, x<\/em>2<\/span>, \u2026, x<\/em>k<\/span><\/p>\n

            b<\/em>1<\/span>, b<\/em>2<\/span>, b<\/em>3<\/span>, \u2026, b<\/em>k<\/span> are the sample estimates of the coefficients \u03b2<\/span>1<\/span>,<\/em> \u03b2<\/span>2<\/span>,<\/em> \u03b2<\/span>3<\/span>,\u2026<\/em>\u03b2<\/span>k<\/span><\/p>\n

            The method of least-squares is still used to fit the model to the data. Remember that this method minimizes the sum of the squared deviations of the observed and predicted values (SSE).<\/p>\n

            The analysis of variance table for multiple regression has a similar appearance to that of a simple linear regression.<\/p>\n

            \"13226.png\"<\/p>\n

            Table 2. ANOVA table.<\/p>\n<\/div>\n

            Where k is the number of predictor variables and n is the number of observations.<\/p>\n

            The best estimate of the random variation \u03c3<\/span>2<\/span>\u2014the variation that is unexplained by the predictor variables\u2014is still s2<\/span>, the MSE. The regression standard error, s, is the square root of the MSE.<\/p>\n

            A new column in the ANOVA table for multiple linear regression shows a decomposition of SSR, in which the conditional contribution of each predictor variable given the variables already entered into the model<\/em> is shown for the order of entry that you specify in your regression. These conditional or sequential sums of squares<\/strong> each account for 1 regression degree of freedom, and allow the user to see the contribution of each predictor variable to the total variation explained by the regression model by using the ratio:<\/p>\n

            \"13413.png\"<\/span><\/p>\n

            Adjusted R2<\/span><\/h2>\n

            In simple linear regression, we used the relationship between the explained and total variation as a measure of model fit:<\/p>\n

            \"13422.png\"<\/p>\n

            Notice from this definition that the value of the coefficient of determination can never decrease with the addition of more variables into the regression model. Hence, R2<\/span> can be artificially inflated as more variables (significant or not) are included in the model. An alternative measure of strength of the regression model is adjusted for degrees of freedom by using mean squares rather than sums of squares:<\/p>\n

            \"13429.png\"<\/p>\n

            The adjusted R2<\/span> value represents the percentage of variation in the response variable explained by the independent variables, corrected for degrees of freedom. Unlike R2<\/span>, the adjusted R2<\/span> will not tend to increase as variables are added and it will tend to stabilize around some upper limit as variables are added.<\/p>\n

            Tests of Significance<\/h2>\n

            Recall in the previous chapter we tested to see if y<\/em> and x<\/em> were linearly related by testing<\/p>\n\n\n\n<\/colgroup>\n\n\n
            \n

            H0<\/span>: \u03b2<\/span>1<\/span> = 0<\/p>\n<\/td>\n

            		\n

            H1<\/span>: \u03b2<\/span>1<\/span> \u2260 0<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

            with the t-test (or the equivalent F-test). In multiple linear regression, there are several partial slopes and the t-test and F-test are no longer equivalent. Our question changes: Is the regression equation that uses information provided by the predictor variables x1<\/span>, x2<\/span>, x3<\/span>, \u2026, xk<\/span>, better than the simple predictor \"13615.png\"<\/span>(the mean response value), which does not rely on any of these independent variables?<\/p>\n

            H0<\/span>: \u03b2<\/span>1<\/span> = \u03b2<\/span>2<\/span> = \u03b2<\/span>3<\/span> = \u2026=\u03b2<\/span>k<\/span> = 0<\/p>\n

            H1<\/span>: At least one of \u03b2<\/span>1<\/span>, \u03b2<\/span>2<\/span> , \u03b2<\/span>3<\/span> , \u2026\u03b2<\/span>k<\/span> \u2260 0<\/p>\n

            The F-test statistic is used to answer this question and is found in the ANOVA table.<\/p>\n

            \"13437.png\"<\/p>\n

            This test statistic follows the F-distribution with df1<\/span> = k and df2<\/span> = (n-k-1). Since the exact p-value is given in the output, you can use the Decision Rule to answer the question.<\/p>\n

            If the p-value is less than the level of significance, reject the null hypothesis.<\/span><\/p>\n

            Rejecting the null hypothesis supports the claim that at least one of the predictor variables has a significant linear relationship with the response variable. The next step is to determine which predictor variables add important information for prediction in the presence of other predictors already in the model. To test the significance of the partial regression coefficients, you need to examine each relationship separately using individual t-tests.<\/p>\n\n\n\n<\/colgroup>\n\n\n
            \n

            H0<\/span>: \u03b2<\/span>i<\/span> = 0<\/p>\n<\/td>\n

            		\n

            H1<\/span>: \u03b2<\/span>i<\/span> \u2260 0<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n


            \n\"13447.png\"<\/span> with df = (n-k-1)<\/p>\n

            where SE(bi<\/sub>)<\/em> is the standard error of bi<\/span><\/sub><\/em>. Exact p-values are also given for these tests. Examining specific p-values for each predictor variable will allow you to decide which variables are significantly related to the response variable. Typically, any insignificant variables are removed from the model, but remember these tests are done with other variables in the model. A good procedure is to remove the least significant variable and then refit the model with the reduced data set. With each new model, always check the regression standard error (lower is better), the adjusted R2<\/span><\/sup> (higher is better), the p-values for all predictor variables, and the residual and normal probability plots.<\/p>\n

            Because of the complexity of the calculations, we will rely on software to fit the model and give us the regression coefficients. Don\u2019t forget\u2026 you always begin with scatterplots. Strong relationships between predictor and response variables make for a good model.<\/p>\n

            \n

            Example 1<\/h3>\n

            A researcher collected data in a project to predict the annual growth per acre of upland boreal forests in southern Canada. They hypothesized that cubic foot volume growth (y<\/em>) is a function of stand basal area per acre (x<\/em>1<\/span>), the percentage of that basal area in black spruce (x<\/em>2<\/span>), and the stand\u2019s site index for black spruce (x<\/em>3<\/span>). \u03b1<\/span> = 0.05.<\/p>\n<\/div>\n

            \"132151.png\"<\/p>\n

            Table 3. Observed data for cubic feet, stand basal area, percent basal area in black spruce, and site index.<\/p>\n<\/div>\n

            Scatterplots of the response variable versus each predictor variable were created along with a correlation matrix.<\/p>\n

            \"13205.png\"<\/p>\n

            Figure 1. Scatterplots of cubic feet versus basal area, percent basal area in black spruce, and site index.<\/p>\n<\/div>\n

            \"13195.png\"<\/p>\n

            Table 4. Correlation matrix.<\/p>\n<\/div>\n

            As you can see from the scatterplots and the correlation matrix, BA\/ac has the strongest linear relationship with CuFt volume (r = 0.816) and %BA in black spruce has the weakest linear relationship (r = 0.413). Also of note is the moderately strong correlation between the two predictor variables, BA\/ac and SI (r = 0.588). All three predictor variables have significant linear relationships with the response variable (volume) so we will begin by using all variables in our multiple linear regression model. The Minitab output is given below.<\/p>\n

            We begin by testing the following null and alternative hypotheses:<\/p>\n

            H0<\/span><\/sub>: \u03b2<\/span>1<\/span><\/sub> = \u03b2<\/span>2<\/span><\/sub> = \u03b2<\/span>3<\/span><\/sub> = 0<\/p>\n

            H1<\/span><\/sub>: At least one of \u03b2<\/span>1<\/span><\/sub>, \u03b2<\/span>2<\/span><\/sub> , \u03b2<\/span>3<\/span><\/sub> \u2260 0<\/p>\n

            General Regression Analysis: CuFt versus BA\/ac, SI, %BA Bspruce<\/h4>\n\n\n\n\n\n\n\n\n<\/colgroup>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
            \n

            Regression Equation<\/p>\n<\/td>\n<\/tr>\n

            \n

            CuFt = -19.3858 + 0.591004 BA\/ac + 0.0899883 SI + 0.489441 %BA Bspruce<\/p>\n<\/td>\n<\/tr>\n

            \n

            Coefficients<\/p>\n<\/td>\n<\/tr>\n

            \n

            Term<\/p>\n<\/td>\n

            		\n

            Coef<\/p>\n<\/td>\n

            		\n

            SE Coef<\/p>\n<\/td>\n

            		\n

            T<\/p>\n<\/td>\n

            		\n

            P<\/p>\n<\/td>\n

            		\n

            95% CI<\/p>\n<\/td>\n

            		<\/td>\n<\/tr>\n
            \n

            Constant<\/p>\n<\/td>\n

            		\n

            -19.3858<\/p>\n<\/td>\n

            		\n

            4.15332<\/p>\n<\/td>\n

            		\n

            -4.6675<\/p>\n<\/td>\n

            		\n

            0.000<\/p>\n<\/td>\n

            		\n

            (-27.9578, -10.8137)<\/p>\n<\/td>\n<\/tr>\n

            \n

            BA\/ac<\/p>\n<\/td>\n

            		\n

            0.5910<\/p>\n<\/td>\n

            		\n

            0.04294<\/p>\n<\/td>\n

            		\n

            13.7647<\/p>\n<\/td>\n

            		\n

            0.000<\/p>\n<\/td>\n

            		\n

            (0.5024, 0.6796)<\/p>\n<\/td>\n<\/tr>\n

            \n

            SI<\/p>\n<\/td>\n

            		\n

            0.0900<\/p>\n<\/td>\n

            		\n

            0.11262<\/p>\n<\/td>\n

            		\n

            0.7991<\/p>\n<\/td>\n

            		\n

            0.432<\/p>\n<\/td>\n

            		\n

            (-0.1424, 0.3224)<\/p>\n<\/td>\n<\/tr>\n

            \n

            %BA Bspruce<\/p>\n<\/td>\n

            		\n

            0.4894<\/p>\n<\/td>\n

            		\n

            0.05245<\/p>\n<\/td>\n

            		\n

            9.3311<\/p>\n<\/td>\n

            		\n

            0.000<\/p>\n<\/td>\n

            		\n

            (0.3812, 0.5977)<\/p>\n<\/td>\n<\/tr>\n

            \n

            Summary of Model<\/p>\n<\/td>\n<\/tr>\n

            \n

            S = 3.17736<\/p>\n<\/td>\n

            		\n

            R-Sq = 95.53%<\/p>\n<\/td>\n

            		\n

            R-Sq(adj) = 94.97%<\/p>\n<\/td>\n

            		<\/td>\n<\/tr>\n
            \n

            PRESS = 322.279<\/p>\n<\/td>\n

            		\n

            R-Sq(pred) = 94.05%<\/p>\n<\/td>\n

            		<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
            \n

            Analysis of Variance<\/p>\n<\/td>\n<\/tr>\n

            \n

            Source<\/p>\n<\/td>\n

            		\n

            DF<\/p>\n<\/td>\n

            		\n

            Seq SS<\/p>\n<\/td>\n

            		\n

            Adj SS<\/p>\n<\/td>\n

            		\n

            Adj MS<\/p>\n<\/td>\n

            		\n

            F<\/p>\n<\/td>\n

            		\n

            P<\/p>\n<\/td>\n<\/tr>\n

            \n

            Regression<\/p>\n<\/td>\n

            		\n

            3<\/p>\n<\/td>\n

            		\n

            5176.56<\/p>\n<\/td>\n

            		\n

            5176.56<\/p>\n<\/td>\n

            		\n

            1725.52<\/p>\n<\/td>\n

            		\n

            170.918<\/strong><\/p>\n<\/td>\n

            		\n

            0.000000<\/strong><\/p>\n<\/td>\n<\/tr>\n

            \n

            BA\/ac<\/p>\n<\/td>\n

            		\n

            1<\/p>\n<\/td>\n

            		\n

            3611.17<\/p>\n<\/td>\n

            		\n

            1912.79<\/p>\n<\/td>\n

            		\n

            1912.79<\/p>\n<\/td>\n

            		\n

            189.467<\/p>\n<\/td>\n

            		\n

            0.000000<\/p>\n<\/td>\n<\/tr>\n

            \n

            SI<\/p>\n<\/td>\n

            		\n

            1<\/p>\n<\/td>\n

            		\n

            686.37<\/p>\n<\/td>\n

            		\n

            6.45<\/p>\n<\/td>\n

            		\n

            6.45<\/p>\n<\/td>\n

            		\n

            0.638<\/p>\n<\/td>\n

            		\n

            0.432094<\/p>\n<\/td>\n<\/tr>\n

            \n

            %BA Bspruce<\/p>\n<\/td>\n

            		\n

            1<\/p>\n<\/td>\n

            		\n

            879.02<\/p>\n<\/td>\n

            		\n

            879.02<\/p>\n<\/td>\n

            		\n

            879.02<\/p>\n<\/td>\n

            		\n

            87.069<\/p>\n<\/td>\n

            		\n

            0.000000<\/p>\n<\/td>\n<\/tr>\n

            \n

            Error<\/p>\n<\/td>\n

            		\n

            24<\/p>\n<\/td>\n

            		\n

            242.30<\/p>\n<\/td>\n

            		\n

            242.30<\/p>\n<\/td>\n

            		\n

            10.10<\/p>\n<\/td>\n

            		<\/td>\n<\/td>\n<\/tr>\n
            \n

            Total<\/p>\n<\/td>\n

            		\n

            27<\/p>\n<\/td>\n

            		\n

            5418.86<\/p>\n<\/td>\n

            		<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

            The F-test statistic (and associated p-value) is used to answer this question and is found in the ANOVA table. For this example, F = 170.918 with a p-value of 0.00000. The p-value is smaller than our level of significance (0.0000<0.05) so we will reject the null hypothesis. At least one of the predictor variables significantly contributes to the prediction of volume.<\/p>\n

            The coefficients for the three predictor variables are all positive indicating that as they increase cubic foot volume will also increase. For example, if we hold values of SI and %BA Bspruce constant, this equation tells us that as basal area increases by 1 sq. ft., volume will increase an additional 0.591004 cu. ft. The signs of these coefficients are logical, and what we would expect. The adjusted R2<\/span><\/sup> is also very high at 94.97%.<\/p>\n

            The next step is to examine the individual t-tests for each predictor variable. The test statistics and associated p-values are found in the Minitab output and repeated below:<\/p>\n\n\n\n\n\n\n\n<\/colgroup>\n\n\n\n\n\n\n\n
            \n

            Coefficients<\/p>\n<\/td>\n<\/tr>\n

            \n

            Term<\/p>\n<\/td>\n

            		\n

            Coef<\/p>\n<\/td>\n

            		\n

            SE Coef<\/p>\n<\/td>\n

            		\n

            T<\/p>\n<\/td>\n

            		\n

            P<\/p>\n<\/td>\n

            		\n

            95% CI<\/p>\n<\/td>\n<\/tr>\n

            \n

            Constant<\/p>\n<\/td>\n

            		\n

            -19.3858<\/p>\n<\/td>\n

            		\n

            4.15332<\/p>\n<\/td>\n

            		\n

            -4.6675<\/p>\n<\/td>\n

            		\n

            0.000<\/p>\n<\/td>\n

            		\n

            (-27.9578, -10.8137)<\/p>\n<\/td>\n<\/tr>\n

            \n

            BA\/ac<\/p>\n<\/td>\n

            		\n

            0.5910<\/p>\n<\/td>\n

            		\n

            0.04294<\/p>\n<\/td>\n

            		\n

            13.7647<\/strong><\/p>\n<\/td>\n

            		\n

            0.000<\/strong><\/p>\n<\/td>\n

            		\n

            ( 0.5024, 0.6796)<\/p>\n<\/td>\n<\/tr>\n

            \n

            SI<\/p>\n<\/td>\n

            		\n

            0.0900<\/p>\n<\/td>\n

            		\n

            0.11262<\/p>\n<\/td>\n

            		\n

            0.7991<\/strong><\/p>\n<\/td>\n

            		\n

            0.432<\/strong><\/p>\n<\/td>\n

            		\n

            ( -0.1424, 0.3224)<\/p>\n<\/td>\n<\/tr>\n

            \n

            %BA Bspruce<\/p>\n<\/td>\n

            		\n

            0.4894<\/p>\n<\/td>\n

            		\n

            0.05245<\/p>\n<\/td>\n

            		\n

            9.3311<\/strong><\/p>\n<\/td>\n

            		\n

            0.000<\/strong><\/p>\n<\/td>\n

            		\n

            ( 0.3812, 0.5977)<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

            The predictor variables BA\/ac and %BA Bspruce have t-statistics of 13.7647 and 9.3311 and p-values of 0.0000, indicating that both are significantly contributing to the prediction of volume. However, SI has a t-statistic of 0.7991 with a p-value of 0.432. This variable does not significantly contribute to the prediction of cubic foot volume.<\/p>\n

            This result may surprise you as SI had the second strongest relationship with volume, but don\u2019t forget about the correlation between SI and BA\/ac (r = 0.588). The predictor variable BA\/ac had the strongest linear relationship with volume, and using the sequential sums of squares, we can see that BA\/ac is already accounting for 70% of the variation in cubic foot volume (3611.17\/5176.56 = 0.6976). The information from SI may be too similar to the information in BA\/ac, and SI only explains about 13% of the variation on volume (686.37\/5176.56 = 0.1326) given that BA\/ac is already in the model.<\/p>\n

            The next step is to examine the residual and normal probability plots. A single outlier is evident in the otherwise acceptable plots.<\/p>\n

            \"13186.png\"<\/p>\n

            Figure 2. Residual and normal probability plots.<\/p>\n<\/div>\n

            So where do we go from here?<\/strong><\/p>\n

            We will remove the non-significant variable and re-fit the model excluding the data for SI in our model. The Minitab output is given below.<\/p>\n

            General Regression Analysis: CuFt versus BA\/ac, %BA Bspruce<\/h4>\n\n\n\n\n\n\n\n\n<\/colgroup>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
            \n

            Regression Equation<\/p>\n<\/td>\n<\/tr>\n

            \n

            CuFt = -19.1142 + 0.615531 BA\/ac + 0.515122 %BA Bspruce<\/p>\n<\/td>\n<\/tr>\n

            \n

            Coefficients<\/p>\n<\/td>\n<\/tr>\n

            \n

            Term<\/p>\n<\/td>\n

            		\n

            Coef<\/p>\n<\/td>\n

            		\n

            SE Coef<\/p>\n<\/td>\n

            		\n

            T<\/p>\n<\/td>\n

            		\n

            P<\/p>\n<\/td>\n

            		\n

            95% CI<\/p>\n<\/td>\n

            		<\/td>\n<\/tr>\n
            \n

            Constant<\/p>\n<\/td>\n

            		\n

            -19.1142<\/p>\n<\/td>\n

            		\n

            4.10936<\/p>\n<\/td>\n

            		\n

            -4.6514<\/p>\n<\/td>\n

            		\n

            0.000<\/p>\n<\/td>\n

            		\n

            (-27.5776, -10.6508)<\/p>\n<\/td>\n<\/tr>\n

            \n

            BA\/ac<\/p>\n<\/td>\n

            		\n

            0.6155<\/p>\n<\/td>\n

            		\n

            0.02980<\/p>\n<\/td>\n

            		\n

            20.6523<\/p>\n<\/td>\n

            		\n

            0.000<\/p>\n<\/td>\n

            		\n

            (0.5541, 0.6769)<\/p>\n<\/td>\n<\/tr>\n

            \n

            %BA Bspruce<\/p>\n<\/td>\n

            		\n

            0.5151<\/p>\n<\/td>\n

            		\n

            0.04115<\/p>\n<\/td>\n

            		\n

            12.5173<\/p>\n<\/td>\n

            		\n

            0.000<\/p>\n<\/td>\n

            		\n

            (0.4304, 0.5999)<\/p>\n<\/td>\n<\/tr>\n

            \n

            Summary of Model<\/p>\n<\/td>\n<\/tr>\n

            \n

            S = 3.15431<\/p>\n<\/td>\n

            		\n

            R-Sq = 95.41%<\/p>\n<\/td>\n

            		\n

            R-Sq(adj) = 95.04%<\/p>\n<\/td>\n

            		<\/td>\n<\/tr>\n
            \n

            PRESS = 298.712<\/p>\n<\/td>\n

            		\n

            R-Sq(pred) = 94.49%<\/p>\n<\/td>\n

            		<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
            \n

            Analysis of Variance<\/p>\n<\/td>\n<\/tr>\n

            \n

            Source<\/p>\n<\/td>\n

            		\n

            DF<\/p>\n<\/td>\n

            		\n

            SeqSS<\/p>\n<\/td>\n

            		\n

            AdjSS<\/p>\n<\/td>\n

            		\n

            AdjMS<\/p>\n<\/td>\n

            		\n

            F<\/p>\n<\/td>\n

            		\n

            P<\/p>\n<\/td>\n<\/tr>\n

            \n

            Regression<\/p>\n<\/td>\n

            		\n

            2<\/p>\n<\/td>\n

            		\n

            5170.12<\/p>\n<\/td>\n

            		\n

            5170.12<\/p>\n<\/td>\n

            		\n

            2585.06<\/p>\n<\/td>\n

            		\n

            259.814<\/p>\n<\/td>\n

            		\n

            0.0000000<\/p>\n<\/td>\n<\/tr>\n

            \n

            BA\/ac<\/p>\n<\/td>\n

            		\n

            1<\/p>\n<\/td>\n

            		\n

            3611.17<\/p>\n<\/td>\n

            		\n

            4243.71<\/p>\n<\/td>\n

            		\n

            4243.71<\/p>\n<\/td>\n

            		\n

            426.519<\/p>\n<\/td>\n

            		\n

            0.0000000<\/p>\n<\/td>\n<\/tr>\n

            \n

            %BA Bspruce<\/p>\n<\/td>\n

            		\n

            1<\/p>\n<\/td>\n

            		\n

            1558.95<\/p>\n<\/td>\n

            		\n

            1558.95<\/p>\n<\/td>\n

            		\n

            1558.95<\/p>\n<\/td>\n

            		\n

            156.684<\/p>\n<\/td>\n

            		\n

            0.0000000<\/p>\n<\/td>\n<\/tr>\n

            \n

            Error<\/p>\n<\/td>\n

            		\n

            25<\/p>\n<\/td>\n

            		\n

            248.74<\/p>\n<\/td>\n

            		\n

            248.74<\/p>\n<\/td>\n

            		\n

            9.95<\/p>\n<\/td>\n

            		<\/td>\n<\/td>\n<\/tr>\n
            \n

            Total<\/p>\n<\/td>\n

            		\n

            27<\/p>\n<\/td>\n

            		\n

            5418.86<\/p>\n<\/td>\n

            		<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

            We will repeat the steps followed with our first model. We begin by again testing the following hypotheses:<\/p>\n

            H0<\/span>: \u03b2<\/span>1<\/span> = \u03b2<\/span>2<\/span> = \u03b2<\/span>3<\/span> = 0<\/p>\n

            H1<\/span>: At least one of \u03b2<\/span>1<\/span>, \u03b2<\/span>2<\/span> , \u03b2<\/span>3<\/span> \u2260 0<\/p>\n

            This reduced model has an F-statistic equal to 259.814 and a p-value of 0.0000. We will reject the null hypothesis. At least one of the predictor variables significantly contributes to the prediction of volume. The coefficients are still positive (as we expected) but the values have changed to account for the different model.<\/p>\n

            The individual t-tests for each coefficient (repeated below) show that both predictor variables are significantly different from zero and contribute to the prediction of volume.<\/p>\n\n\n\n\n\n\n\n<\/colgroup>\n\n\n\n\n\n\n
            \n

            Coefficients<\/p>\n<\/td>\n<\/tr>\n

            \n

            Term<\/p>\n<\/td>\n

            		\n

            Coef<\/p>\n<\/td>\n

            		\n

            SE Coef<\/p>\n<\/td>\n

            		\n

            T<\/p>\n<\/td>\n

            		\n

            P<\/p>\n<\/td>\n

            		\n

            95% CI<\/p>\n<\/td>\n<\/tr>\n

            \n

            Constant<\/p>\n<\/td>\n

            		\n

            -19.1142<\/p>\n<\/td>\n

            		\n

            4.10936<\/p>\n<\/td>\n

            		\n

            -4.6514<\/p>\n<\/td>\n

            		\n

            0.000<\/p>\n<\/td>\n

            		\n

            (-27.5776, -10.6508)<\/p>\n<\/td>\n<\/tr>\n

            \n

            BA\/ac<\/p>\n<\/td>\n

            		\n

            0.6155<\/p>\n<\/td>\n

            		\n

            0.02980<\/p>\n<\/td>\n

            		\n

            20.6523<\/p>\n<\/td>\n

            		\n

            0.000<\/p>\n<\/td>\n

            		\n

            ( 0.5541, 0.6769)<\/p>\n<\/td>\n<\/tr>\n

            \n

            %BA Bspruce<\/p>\n<\/td>\n

            		\n

            0.5151<\/p>\n<\/td>\n

            		\n

            0.04115<\/p>\n<\/td>\n

            		\n

            12.5173<\/p>\n<\/td>\n

            		\n

            0.000<\/p>\n<\/td>\n

            		\n

            ( 0.4304, 0.5999)<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

            Notice that the adjusted R2<\/span> has increased from 94.97% to 95.04% indicating a slightly better fit to the data. The regression standard error has also changed for the better, decreasing from 3.17736 to 3.15431 indicating slightly less variation of the observed data to the model.<\/p>\n

            \"131751.png\"<\/p>\n

            Figure 3. Residual and normal probability plots.<\/p>\n<\/div>\n

            The residual and normal probability plots have changed little, still not indicating any issues with the regression assumption. By removing the non-significant variable, the model has improved.<\/p>\n

            Model Development and Selection<\/h2>\n

            There are many different reasons for creating a multiple linear regression model and its purpose directly influences how the model is created. Listed below are several of the more commons uses for a regression model:<\/p>\n

              \n
            1. Describing the behavior of your response variable<\/li>\n
            2. Predicting a response or estimating the average response<\/li>\n
            3. Estimating the parameters (\u03b2<\/span>0<\/span>, \u03b2<\/span>1<\/span>, \u03b2<\/span>2<\/span>, \u2026)<\/li>\n
            4. Developing an accurate model of the process<\/li>\n<\/ol>\n

              Depending on your objective for creating a regression model, your methodology may vary when it comes to variable selection, retention, and elimination.<\/p>\n

              When the object is simple description of your response variable, you are typically less concerned about eliminating non-significant variables. The best representation of the response variable, in terms of minimal residual sums of squares, is the full model, which includes all predictor variables available from the data set. It is less important that the variables are causally related or that the model is realistic.<\/p>\n

              A common reason for creating a regression model is for prediction and estimating. A researcher wants to be able to define events within the x-space of data that were collected for this model, and it is assumed that the system will continue to function as it did when the data were collected. Any measurable predictor variables that contain information on the response variable should be included. For this reason, non-significant variables may be retained in the model. However, regression equations with fewer variables are easier to use and have an economic advantage in terms of data collection. Additionally, there is a greater confidence attached to models that contain only significant variables.<\/p>\n

              If the objective is to estimate the model parameters, you will be more cautious when considering variable elimination. You want to avoid introducing a bias by removing a variable that has predictive information about the response. However, there is a statistical advantage in terms of reduced variance of the parameter estimates if variables truly unrelated to the response variable are removed.<\/p>\n

              Building a realistic model of the process you are studying is often a primary goal of much research. It is important to identify the variables that are linked to the response through some causal relationship. While you can identify which variables have a strong correlation with the response, this only serves as an indicator of which variables require further study. The principal objective is to develop a model whose functional form realistically reflects the behavior of a system.<\/p>\n

              The following figure is a strategy for building a regression model.<\/p>\n

              \"153_1.tif\"<\/p>\n

              Figure 4. Strategy for building a regression model.<\/p>\n<\/div>\n

              Software Solutions<\/h2>\n

              Minitab<\/h3>\n

              \"155_1.tif\"<\/span><\/p>\n

              \"155_2.tif\"<\/span><\/p>\n

              \"155_3.tif\"<\/p>\n

              The output and plots are given in the previous example.<\/p>\n

              Excel<\/h3>\n

              \"154_1.tif\"<\/span><\/p>\n

              \"154_2.tif\"<\/span><\/p>\n

              \"154_3.tif\"<\/span><\/p>\n

              \"154_4.tif\"<\/span><\/p>\n<\/div>\n\n\t\t\t

              \n\t\t\t

              Candela Citations<\/h3>\n\t\t\t\t\t
              \n\t\t\t\t\t\t
              \n\t\t\t\t\t\t\t
              CC licensed content, Shared previously<\/div>