tips:comp_two_independent_estimates
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
tips:comp_two_independent_estimates [2022/08/03 11:31] – Wolfgang Viechtbauer | tips:comp_two_independent_estimates [2024/03/28 09:00] – Wolfgang Viechtbauer | ||
---|---|---|---|
Line 6: | Line 6: | ||
We will use the ' | We will use the ' | ||
+ | |||
<code rsplus> | <code rsplus> | ||
library(metafor) | library(metafor) | ||
Line 12: | Line 13: | ||
dat | dat | ||
</ | </ | ||
+ | |||
<code output> | <code output> | ||
| | ||
Line 32: | Line 34: | ||
First, we fit two separate random-effects models within each subset defined by the '' | First, we fit two separate random-effects models within each subset defined by the '' | ||
+ | |||
<code rsplus> | <code rsplus> | ||
res1 <- rma(yi, vi, data=dat, subset=alloc==" | res1 <- rma(yi, vi, data=dat, subset=alloc==" | ||
Line 38: | Line 41: | ||
We then combine the estimates and standard errors from each model into a data frame. We also add a variable to distinguish the two models and, for reasons to be explained in more detail below, we add the estimated amounts of heterogeneity within each subset to the data frame. | We then combine the estimates and standard errors from each model into a data frame. We also add a variable to distinguish the two models and, for reasons to be explained in more detail below, we add the estimated amounts of heterogeneity within each subset to the data frame. | ||
+ | |||
<code rsplus> | <code rsplus> | ||
- | dat.comp <- data.frame(estimate = c(coef(res1), | + | dat.comp <- data.frame(meta = c(" |
- | meta = c(" | + | estimate = c(coef(res1), |
- | dat.comp | + | stderror = c(res1$se, res2$se), |
+ | | ||
+ | dfround(dat.comp, 3) | ||
</ | </ | ||
+ | |||
<code output> | <code output> | ||
- | estimate | + | |
- | 1 -0.9709645 | + | 1 random |
- | 2 -0.4812706 | + | 2 |
</ | </ | ||
We can now compare the two estimates (i.e., the estimated average log risk ratios) by feeding them back to the '' | We can now compare the two estimates (i.e., the estimated average log risk ratios) by feeding them back to the '' | ||
+ | |||
<code rsplus> | <code rsplus> | ||
rma(estimate, | rma(estimate, | ||
</ | </ | ||
+ | |||
<code output> | <code output> | ||
Fixed-Effects with Moderators Model (k = 2) | Fixed-Effects with Moderators Model (k = 2) | ||
Line 71: | Line 80: | ||
Signif. codes: | Signif. codes: | ||
</ | </ | ||
- | While we find that studies using random assignment obtain on average larger (i.e., more negative) effects than studies not using random assignment ($b_1 = -0.490$, $SE = 0.351$), the difference between the two estimates is not significant ($z = -1.395$, $p = .163$). | + | |
+ | While we find that studies using random assignment obtain on average larger (i.e., more negative) effects than studies not using random assignment ($b_1 = -0.490$, $SE = 0.351$), the difference between the two estimates is not statistically | ||
The test of the difference between the two estimates is really just a [[https:// | The test of the difference between the two estimates is really just a [[https:// | ||
+ | |||
<code rsplus> | <code rsplus> | ||
with(dat.comp, | with(dat.comp, | ||
</ | </ | ||
+ | |||
<code output> | <code output> | ||
zval | zval | ||
-1.395 | -1.395 | ||
</ | </ | ||
+ | |||
This is the same value that we obtained above. | This is the same value that we obtained above. | ||
Line 86: | Line 99: | ||
Now let's take a different approach, fitting a meta-regression model with '' | Now let's take a different approach, fitting a meta-regression model with '' | ||
+ | |||
<code rsplus> | <code rsplus> | ||
rma(yi, vi, mods = ~ alloc, data=dat, digits=3) | rma(yi, vi, mods = ~ alloc, data=dat, digits=3) | ||
</ | </ | ||
+ | |||
<code output> | <code output> | ||
Mixed-Effects Model (k = 13; tau^2 estimator: REML) | Mixed-Effects Model (k = 13; tau^2 estimator: REML) | ||
Line 113: | Line 128: | ||
Signif. codes: | Signif. codes: | ||
</ | </ | ||
+ | |||
The result is very similar to what we saw earlier: The coefficient corresponding to the '' | The result is very similar to what we saw earlier: The coefficient corresponding to the '' | ||
- | However, the results are not exactly identical. The reason for this is as follows. When we fit separate random-effects models in the two subsets, we are allowing the amount of heterogeneity within each set to be different (as shown earlier, the estimates were $\hat{\tau}^2 = 0.393$ and $\hat{\tau}^2 = 0.212$ for studies using and not using random assignment, respectively). On the other hand, the mixed-effects meta-regression model fitted above has a single variance component for the amount of residual heterogeneity, | + | However, the results are not exactly identical. The reason for this is as follows. When we fit separate random-effects models in the two subsets, we are allowing the amount of heterogeneity within each set to be different (as shown earlier, the estimates were $\hat{\tau}^2 = 0.393$ and $\hat{\tau}^2 = 0.212$ for studies using and not using random assignment, respectively). On the other hand, the mixed-effects meta-regression model fitted above has a single variance component for the amount of residual heterogeneity, |
==== Meta-Regression with All Studies but Different Amounts of (Residual) Heterogeneity ==== | ==== Meta-Regression with All Studies but Different Amounts of (Residual) Heterogeneity ==== | ||
Using the '' | Using the '' | ||
+ | |||
<code rsplus> | <code rsplus> | ||
rma.mv(yi, vi, mods = ~ alloc, random = ~ alloc | trial, struct=" | rma.mv(yi, vi, mods = ~ alloc, random = ~ alloc | trial, struct=" | ||
</ | </ | ||
+ | |||
<code output> | <code output> | ||
Multivariate Meta-Analysis Model (k = 13; method: REML) | Multivariate Meta-Analysis Model (k = 13; method: REML) | ||
Line 150: | Line 168: | ||
Signif. codes: | Signif. codes: | ||
</ | </ | ||
- | Note that the two estimates of $\tau^2$ are now identical to the ones we obtained earlier from the separate random-effects models. Also, the coefficient, | ||
- | A discussion/ | + | Note that the two estimates of $\tau^2$ are now identical to the ones we obtained earlier from the separate random-effects models. Also, the coefficient, |
+ | |||
+ | A discussion/ | ||
+ | |||
+ | Rubio-Aparicio, | ||
Rubio-Aparicio, | Rubio-Aparicio, | ||
- | We can also do a likelihood ratio test (LRT) to examine whether there are significant differences in the $\tau^2$ values across subsets. This can be done with: | + | We can also conduct |
<code rsplus> | <code rsplus> | ||
Line 163: | Line 184: | ||
anova(res1, res0) | anova(res1, res0) | ||
</ | </ | ||
+ | |||
<code output> | <code output> | ||
df | df | ||
Line 170: | Line 192: | ||
So in this example, we would not reject the null hypothesis $H_0: \tau^2_1 = \tau^2_2$ ($p = .58$). | So in this example, we would not reject the null hypothesis $H_0: \tau^2_1 = \tau^2_2$ ($p = .58$). | ||
+ | |||
+ | ==== Other Types of Models ==== | ||
+ | |||
+ | The issue discussed above also arises for other types of models (e.g., multilevel meta-analytic models). When fitting a particular model within several subgroups, then the variance components of the model are automatically allowed to differ across the subgroups. On the other hand, when fitting the same type of model to all studies combined (but including a moderator to allow the mean effect size to differ across subgroups), then the variance components are assumed to be the same within each subgroups (unless one takes extra steps as illustrated above to allow the variance components to differ across subgroups). Consequently, | ||
+ |
tips/comp_two_independent_estimates.txt · Last modified: 2024/06/18 19:28 by Wolfgang Viechtbauer