tips:comp_two_independent_estimates
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
tips:comp_two_independent_estimates [2020/07/03 10:32] – Wolfgang Viechtbauer | tips:comp_two_independent_estimates [2023/02/18 17:35] – Wolfgang Viechtbauer | ||
---|---|---|---|
Line 6: | Line 6: | ||
We will use the ' | We will use the ' | ||
+ | |||
<code rsplus> | <code rsplus> | ||
library(metafor) | library(metafor) | ||
Line 12: | Line 13: | ||
dat | dat | ||
</ | </ | ||
+ | |||
<code output> | <code output> | ||
| | ||
Line 32: | Line 34: | ||
First, we fit two separate random-effects models within each subset defined by the '' | First, we fit two separate random-effects models within each subset defined by the '' | ||
+ | |||
<code rsplus> | <code rsplus> | ||
res1 <- rma(yi, vi, data=dat, subset=alloc==" | res1 <- rma(yi, vi, data=dat, subset=alloc==" | ||
Line 38: | Line 41: | ||
We then combine the estimates and standard errors from each model into a data frame. We also add a variable to distinguish the two models and, for reasons to be explained in more detail below, we add the estimated amounts of heterogeneity within each subset to the data frame. | We then combine the estimates and standard errors from each model into a data frame. We also add a variable to distinguish the two models and, for reasons to be explained in more detail below, we add the estimated amounts of heterogeneity within each subset to the data frame. | ||
+ | |||
<code rsplus> | <code rsplus> | ||
dat.comp <- data.frame(estimate = c(coef(res1), | dat.comp <- data.frame(estimate = c(coef(res1), | ||
Line 43: | Line 47: | ||
dat.comp | dat.comp | ||
</ | </ | ||
+ | |||
<code output> | <code output> | ||
estimate | estimate | ||
Line 49: | Line 54: | ||
</ | </ | ||
- | We can now compare the two estimates (i.e., the estimated average log risk ratios) by feeding them back to the '' | + | We can now compare the two estimates (i.e., the estimated average log risk ratios) by feeding them back to the '' |
<code rsplus> | <code rsplus> | ||
rma(estimate, | rma(estimate, | ||
</ | </ | ||
+ | |||
<code output> | <code output> | ||
Fixed-Effects with Moderators Model (k = 2) | Fixed-Effects with Moderators Model (k = 2) | ||
- | Test for Residual Heterogeneity: | + | Test for Residual Heterogeneity: |
QE(df = 0) = 0.000, p-val = 1.000 | QE(df = 0) = 0.000, p-val = 1.000 | ||
- | Test of Moderators (coefficient(s) 2): | + | Test of Moderators (coefficient(s) 2): |
QM(df = 1) = 1.946, p-val = 0.163 | QM(df = 1) = 1.946, p-val = 0.163 | ||
Model Results: | Model Results: | ||
- | estimate | + | estimate |
intrcpt | intrcpt | ||
- | metarandom | + | metarandom |
--- | --- | ||
Signif. codes: | Signif. codes: | ||
</ | </ | ||
+ | |||
While we find that studies using random assignment obtain on average larger (i.e., more negative) effects than studies not using random assignment ($b_1 = -0.490$, $SE = 0.351$), the difference between the two estimates is not significant ($z = -1.395$, $p = .163$). | While we find that studies using random assignment obtain on average larger (i.e., more negative) effects than studies not using random assignment ($b_1 = -0.490$, $SE = 0.351$), the difference between the two estimates is not significant ($z = -1.395$, $p = .163$). | ||
The test of the difference between the two estimates is really just a [[https:// | The test of the difference between the two estimates is really just a [[https:// | ||
+ | |||
<code rsplus> | <code rsplus> | ||
with(dat.comp, | with(dat.comp, | ||
</ | </ | ||
+ | |||
<code output> | <code output> | ||
- | zval | + | zval |
-1.395 | -1.395 | ||
</ | </ | ||
+ | |||
This is the same value that we obtained above. | This is the same value that we obtained above. | ||
Line 86: | Line 97: | ||
Now let's take a different approach, fitting a meta-regression model with '' | Now let's take a different approach, fitting a meta-regression model with '' | ||
+ | |||
<code rsplus> | <code rsplus> | ||
rma(yi, vi, mods = ~ alloc, data=dat, digits=3) | rma(yi, vi, mods = ~ alloc, data=dat, digits=3) | ||
</ | </ | ||
+ | |||
<code output> | <code output> | ||
Mixed-Effects Model (k = 13; tau^2 estimator: REML) | Mixed-Effects Model (k = 13; tau^2 estimator: REML) | ||
Line 98: | Line 111: | ||
R^2 (amount of heterogeneity accounted for): 0.00% | R^2 (amount of heterogeneity accounted for): 0.00% | ||
- | Test for Residual Heterogeneity: | + | Test for Residual Heterogeneity: |
QE(df = 11) = 138.511, p-val < .001 | QE(df = 11) = 138.511, p-val < .001 | ||
- | Test of Moderators (coefficient(s) 2): | + | Test of Moderators (coefficient(s) 2): |
QM(df = 1) = 1.833, p-val = 0.176 | QM(df = 1) = 1.833, p-val = 0.176 | ||
Model Results: | Model Results: | ||
- | | + | |
intrcpt | intrcpt | ||
- | allocrandom | + | allocrandom |
--- | --- | ||
Signif. codes: | Signif. codes: | ||
</ | </ | ||
+ | |||
The result is very similar to what we saw earlier: The coefficient corresponding to the '' | The result is very similar to what we saw earlier: The coefficient corresponding to the '' | ||
Line 120: | Line 134: | ||
Using the '' | Using the '' | ||
+ | |||
<code rsplus> | <code rsplus> | ||
rma.mv(yi, vi, mods = ~ alloc, random = ~ alloc | trial, struct=" | rma.mv(yi, vi, mods = ~ alloc, random = ~ alloc | trial, struct=" | ||
</ | </ | ||
+ | |||
<code output> | <code output> | ||
Multivariate Meta-Analysis Model (k = 13; method: REML) | Multivariate Meta-Analysis Model (k = 13; method: REML) | ||
- | Variance Components: | + | Variance Components: |
outer factor: trial (nlvls = 13) | outer factor: trial (nlvls = 13) | ||
Line 135: | Line 151: | ||
tau^2.2 | tau^2.2 | ||
- | Test for Residual Heterogeneity: | + | Test for Residual Heterogeneity: |
QE(df = 11) = 138.511, p-val < .001 | QE(df = 11) = 138.511, p-val < .001 | ||
- | Test of Moderators (coefficient(s) 2): | + | Test of Moderators (coefficient(s) 2): |
QM(df = 1) = 1.946, p-val = 0.163 | QM(df = 1) = 1.946, p-val = 0.163 | ||
Model Results: | Model Results: | ||
- | | + | |
intrcpt | intrcpt | ||
- | allocrandom | + | allocrandom |
--- | --- | ||
Signif. codes: | Signif. codes: | ||
</ | </ | ||
- | Note that the two estimates of $\tau^2$ are now identical to the ones we obtained earlier from the separate random-effects models. Also, the coefficient, | + | |
+ | Note that the two estimates of $\tau^2$ are now identical to the ones we obtained earlier from the separate random-effects models. Also, the coefficient, | ||
A discussion/ | A discussion/ | ||
- | Rubio-Aparicio, | + | Rubio-Aparicio, |
We can also do a likelihood ratio test (LRT) to examine whether there are significant differences in the $\tau^2$ values across subsets. This can be done with: | We can also do a likelihood ratio test (LRT) to examine whether there are significant differences in the $\tau^2$ values across subsets. This can be done with: | ||
Line 163: | Line 180: | ||
anova(res1, res0) | anova(res1, res0) | ||
</ | </ | ||
+ | |||
<code output> | <code output> | ||
- | df | + | df |
- | Full 4 29.2959 30.8875 35.9626 -10.6480 | + | Full 4 29.2959 30.8875 35.9626 -10.6480 |
Reduced | Reduced | ||
</ | </ | ||
So in this example, we would not reject the null hypothesis $H_0: \tau^2_1 = \tau^2_2$ ($p = .58$). | So in this example, we would not reject the null hypothesis $H_0: \tau^2_1 = \tau^2_2$ ($p = .58$). |
tips/comp_two_independent_estimates.txt · Last modified: 2024/04/18 11:36 by Wolfgang Viechtbauer