Differences

This shows you the differences between two versions of the page.

--- tips:hunter_schmidt_method [2019/01/17 12:26] – external edit 127.0.0.1
+++ tips:hunter_schmidt_method [2022/08/03 11:32] (current) – Wolfgang Viechtbauer
@@ Line 11: / Line 11: @@
   - switch to the Hunter and Schmidt estimator to estimate the amount of heterogeneity.
-**A few details about step 1**: The large-sample equation for the sampling variance of a (Pearson product-moment) correlation coefficient is $$\text{Var}[r_i] = \frac{(1-\rho_i^2)^2}{n_i - 1}$$ (although one could also just divide by $n_i$, and not $n_i - 1$). Since this equation involves the (unknown) true correlation coefficient $\rho_i$, it is not usable in practice. One approach is to substitute $r_i$ for $\rho_i$, but the resulting estimator for $\text{Var}[r_i]$ can be very inaccurate, especially when the sample size of a study is small. Hunter and Schmidt instead suggest to calculate the sample-size weighted average of the correlation coefficients, that is, $$\bar{r} = \frac{\sum n_i r_i}{\sum n_i},$$ and then substitute $\bar{r}$ into the equation for the variance. This makes perfect sense if the true correlations were actually homogeneous (since $\bar{r}$ is then a pretty good estimator for the true correlation across all studies). However, even if the true correlations are heterogeneous, this approach still tends to work rather well. The ''escalc()'' function actually provides this functionality, by using the ''vtype="AV"'' option (note: at the moment, you will have to install the development version of the metafor package for this to work; [[:installation#development_version|see here for instructions]]).
+**A few details about step 1**: The large-sample equation for the sampling variance of a (Pearson product-moment) correlation coefficient is $$\text{Var}[r_i] = \frac{(1-\rho_i^2)^2}{n_i - 1}$$ (although one could also just divide by $n_i$, and not $n_i - 1$). Since this equation involves the (unknown) true correlation coefficient $\rho_i$, it is not usable in practice. One approach is to substitute $r_i$ for $\rho_i$, but the resulting estimator for $\text{Var}[r_i]$ can be very inaccurate, especially when the sample size of a study is small. Hunter and Schmidt instead suggest to calculate the sample-size weighted average of the correlation coefficients, that is, $$\bar{r} = \frac{\sum n_i r_i}{\sum n_i},$$ and then substitute $\bar{r}$ into the equation for the variance. This makes perfect sense if the true correlations were actually homogeneous (since $\bar{r}$ is then a pretty good estimator for the true correlation across all studies). However, even if the true correlations are heterogeneous, this approach still tends to work rather well. The ''escalc()'' function actually provides this functionality, by using the ''vtype="AV"'' option.
 Let's use the data from the meta-analysis by McDaniel et al. (1990) (on the validity of employment interviews) for such an analysis. The three adjustments described above are applied as follows:
@@ Line 30: / Line 30: @@
 H^2 (total variability / sampling variability):  3.92
 Test for Heterogeneity:
 Q(df = 159) = 647.6782, p-val < .0001
 Model Results:
 estimate      se    zval    pval   ci.lb   ci.ub
 .2005  0.0295  6.7947  <.0001  0.1427  0.2583  ***
@@ Line 69: / Line 69: @@
 H^2 (total variability / sampling variability):  3.90
 Test for Heterogeneity:
 Q(df = 159) = 643.4014, p-val < .0001
 Model Results:
 estimate      se    zval    pval   ci.lb   ci.ub
 .2680  0.0391  6.8624  <.0001  0.1915  0.3446  ***