The metafor Package

A Meta-Analysis Package for R

User Tools

Site Tools


analyses:morris2008

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
Next revisionBoth sides next revision
analyses:morris2008 [2017/03/19 13:09] – external edit 127.0.0.1analyses:morris2008 [2021/11/08 13:17] Wolfgang Viechtbauer
Line 5: Line 5:
 Morris (2008) discusses various ways for computing a (standardized) effect size measure for pretest posttest control group designs, where the characteristic, response, or dependent variable assessed in the individual studies is a quantitative variable. Morris (2008) discusses various ways for computing a (standardized) effect size measure for pretest posttest control group designs, where the characteristic, response, or dependent variable assessed in the individual studies is a quantitative variable.
  
-As described by Becker (1988), we can compute the standardized mean change (with raw score standardization) for a treatment and control group with $$g_T = c(n_T-1) \frac{\bar{x}_{post,T} - \bar{x}_{pre,T}}{SD_{pre,T}}$$ and $$g_C = c(n_C-1) \frac{\bar{x}_{post,C} - \bar{x}_{pre,C}}{SD_{pre,C}},$$ where $\bar{x}_{pre,T}$ and $\bar{x}_{post,T}$ are the treatment group pretest and posttest means, $SD_{pre,T}$ is the standard deviation of the pretest scores, $c(m) = \sqrt{2/m} \Gamma[m/2] / \Gamma[(m-1)/2]$ is a bias-correction factor((The bias correction factor given on page 261 by Becker (1988) includes a slight error. See Morris (2000).)), $n_T$ is the size of the treatment group, and $\bar{x}_{pre,C}$, $\bar{x}_{post,C}$, $SD_{pre,C}$, and $n_C$ are the analogous values for the control group. Then the difference in the two standardized mean change values, namely $$g = g_T - g_C$$ indicates how much larger (or smaller) the change in the treatment group was (in standard deviation units) when compared to the change in the control group. Values of $g$ computed for a number of studies could then be meta-analyzed with standard methods.((Note that $g$ is used here to denote the bias-corrected value (as opposed to Becker (1988who uses $d$ to denote this).))+As described by Becker (1988), we can compute the standardized mean change (with raw score standardization) for a treatment and control group with $$g_T = c(n_T-1) \frac{\bar{x}_{post,T} - \bar{x}_{pre,T}}{SD_{pre,T}}$$ and $$g_C = c(n_C-1) \frac{\bar{x}_{post,C} - \bar{x}_{pre,C}}{SD_{pre,C}},$$ where $\bar{x}_{pre,T}$ and $\bar{x}_{post,T}$ are the treatment group pretest and posttest means, $SD_{pre,T}$ is the standard deviation of the pretest scores, $c(m) = \sqrt{2/m} \Gamma[m/2] / \Gamma[(m-1)/2]$ is a bias-correction factor((The bias correction factor given on page 261 by Becker (1988) includes a slight error. See Morris (2000).)), $n_T$ is the size of the treatment group, and $\bar{x}_{pre,C}$, $\bar{x}_{post,C}$, $SD_{pre,C}$, and $n_C$ are the analogous values for the control group. Then the difference in the two standardized mean change values, namely $$g = g_T - g_C$$ indicates how much larger (or smaller) the change in the treatment group was (in standard deviation units) when compared to the change in the control group. Values of $g$ computed for a number of studies could then be meta-analyzed with standard methods.((Note that $g$ is used here to denote the bias-corrected value (as opposed to Becker1988who uses $d$ to denote this). There was (and sometimes still is) some inconsistency in notation when referring to the biased and the bias-corrected version of standardized mean difference / change measures, but I would say the general trend has been to use $d$ for the biased version and $g$ for the bias-corrected version and this is the notation I am also using here.))
  
 Morris (2008) uses five studies from a meta-analysis on training effectiveness by Carlson and Schmidt (1999) to illustrate these computations. We can create the same dataset with: Morris (2008) uses five studies from a meta-analysis on training effectiveness by Carlson and Schmidt (1999) to illustrate these computations. We can create the same dataset with:
Line 99: Line 99:
 ==== The Actual Meta-Analysis ==== ==== The Actual Meta-Analysis ====
  
-For the actual meta-analysis part, we simply pass the ''yi'' and ''vi'' values to the ''rma()'' function. For example, a fixed-effects model can be fitted with:+For the actual meta-analysis part, we simply pass the ''yi'' and ''vi'' values to the ''rma()'' function. For example, an equal-effects model can be fitted with:
 <code rsplus> <code rsplus>
-rma(yi, vi, data=dat, method="FE", digits=2)+rma(yi, vi, data=dat, method="EE", digits=2)
 </code> </code>
 <code output> <code output>
-Fixed-Effects Model (k = 5)+Equal-Effects Model (k = 5)
  
-Test for Heterogeneity: +I^2 (total heterogeneity / total variability):   9.69% 
 +H^2 (total variability / sampling variability):  1.11 
 + 
 +Test for Heterogeneity:
 Q(df = 4) = 4.43, p-val = 0.35 Q(df = 4) = 4.43, p-val = 0.35
  
 Model Results: Model Results:
  
-estimate       se     zval     pval    ci.lb    ci.ub           +estimate    se  zval  pval  ci.lb  ci.ub  
-    0.95     0.14     6.62     <.01     0.67     1.23      *** +    0.95  0.14  6.62  <.01   0.67   1.23  *** 
  
 --- ---
-Signif. codes: ***’ 0.001 **’ 0.01 *’ 0.05 .’ 0.1 ‘ ’ 1+Signif. codes: '***0.001 '**0.01 '*0.05 '.0.1 ' ' 1
 </code> </code>
 Note that these results are slightly different than the ones in Table 5 due to the different ways of estimating the sampling variances. Note that these results are slightly different than the ones in Table 5 due to the different ways of estimating the sampling variances.
Line 121: Line 124:
 ==== Alternative Estimates ==== ==== Alternative Estimates ====
  
-In his article, Morris (2008) discusses two other ways of computing an effect size measure for pretest posttest control group designs. The second approach that pools the two pretest SDs actually can be more efficient under certain conditions. However, that approach assumes that the true pretest SDs are equal for the two groups. That may not be the case. The approach given above does not make that assumption and therefore is more broadly applicable (but may be less efficient).+In his article, Morris (2008) discusses two other ways of computing an effect size measure for pretest posttest control group designs. The second approach that pools the two pretest SDs actually can be more efficient under certain conditions. However, that approach assumes that the true pretest SDs are equal for the two groups. That may not be the case. The approach given above does not make that assumption and therefore is more broadly applicable (but may be slightly less efficient).
  
 If you really want to use the approach with pooled pretest SDs, then this can be done as follows: If you really want to use the approach with pooled pretest SDs, then this can be done as follows:
Line 139: Line 142:
 5 0.44 0.16 5 0.44 0.16
 </code> </code>
-The ''yi'' values above are the exact same value given in Table 5 (under the $d_{ppc2}$ column) by Morris (2008). Note that the equation used for computing the sampling variances above is slightly different from the one used in the paper, so the values for 'vi' above and the ones given in Table 5 (column $\hat{\sigma}^2(d_{ppc2})$ in Morris, 2008) differ slightly.+The ''yi'' values above are the exact same value given in Table 5 (under the $d_{ppc2}$ column) by Morris (2008). Note that the equation used for computing the sampling variances above is slightly different from the one used in the paper, so the values for ''vi'' above and the ones given in Table 5 (column $\hat{\sigma}^2(d_{ppc2})$ in Morris, 2008) differ slightly.
  
 ==== Pretest Posttest Correlations ==== ==== Pretest Posttest Correlations ====
analyses/morris2008.txt · Last modified: 2022/08/03 17:05 by Wolfgang Viechtbauer