Differences

This shows you the differences between two versions of the page.

--- tips:comp_two_independent_estimates [2023/02/18 19:24] – Wolfgang Viechtbauer
+++ tips:comp_two_independent_estimates [2024/03/28 09:00] – Wolfgang Viechtbauer
@@ Line 43: / Line 43: @@
 <code rsplus>
-dat.comp <- data.frame(estimate = c(coef(res1), coef(res2)), stderror = c(res1$se, res2$se),
+dat.comp <- data.frame(meta     = c("random", "other"),
-                       meta = c("random","other"), tau2 = round(c(res1$tau2, res2$tau2),3))
+                       estimate = c(coef(res1), coef(res2)),
-dat.comp
+                       stderror = c(res1$se, res2$se),
+                       tau2     = c(res1$tau2, res2$tau2))
+dfround(dat.comp, 3)
 </code>
 <code output>
-    estimate  stderror   meta  tau2
+    meta estimate stderror  tau2
--0.9709645 0.2759557 random 0.393
+ random   -0.971    0.276 0.393
--0.4812706 0.2169886  other 0.212
+  other   -0.481    0.217 0.212
 </code>
@@ Line 79: / Line 81: @@
 </code>
-While we find that studies using random assignment obtain on average larger (i.e., more negative) effects than studies not using random assignment ($b_1 = -0.490$, $SE = 0.351$), the difference between the two estimates is not significant ($z = -1.395$, $p = .163$).
+While we find that studies using random assignment obtain on average larger (i.e., more negative) effects than studies not using random assignment ($b_1 = -0.490$, $SE = 0.351$), the difference between the two estimates is not statistically significant ($z = -1.395$, $p = .163$).
 The test of the difference between the two estimates is really just a [[https://en.wikipedia.org/wiki/Wald_test|Wald-type test]], given by the equation $$z = \frac{\hat{\mu}_1 - \hat{\mu}_2}{\sqrt{SE[\hat{\mu}_1]^2 + SE[\hat{\mu}_2]^2}},$$ where $\hat{\mu}_1$ and $\hat{\mu}_2$ are the two estimates and $SE[\hat{\mu}_1]$ and $SE[\hat{\mu}_2]$ the corresponding standard errors. The test statistics can therefore also be computed with:
@@ Line 129: / Line 131: @@
 The result is very similar to what we saw earlier: The coefficient corresponding to the ''alloc'' dummy is equal to $b_1 = -0.490$ ($SE = 0.362$) and not significant ($p = .176$).
-However, the results are not exactly identical. The reason for this is as follows. When we fit separate random-effects models in the two subsets, we are allowing the amount of heterogeneity within each set to be different (as shown earlier, the estimates were $\hat{\tau}^2 = 0.393$ and $\hat{\tau}^2 = 0.212$ for studies using and not using random assignment, respectively). On the other hand, the mixed-effects meta-regression model fitted above has a single variance component for the amount of residual heterogeneity, which implies that the amount of heterogeneity //within each subset// is assumed to be the same ($\hat{\tau}^2 = 0.318$ in this example).
+However, the results are not exactly identical. The reason for this is as follows. When we fit separate random-effects models in the two subsets, we are allowing the amount of heterogeneity within each set to be different (as shown earlier, the estimates were $\hat{\tau}^2 = 0.393$ and $\hat{\tau}^2 = 0.212$ for studies using and not using random assignment, respectively). On the other hand, the mixed-effects meta-regression model fitted above has a single variance component for the amount of residual heterogeneity, which implies that the amount of heterogeneity //within each subset// is assumed to be the same ($\hat{\tau}^2 = 0.318$ in this example -- note that this falls somewhere between the two $\hat{\tau}^2$ values we obtained within the subsets).
 ==== Meta-Regression with All Studies but Different Amounts of (Residual) Heterogeneity ====