This vignette focuses on the bivariate random-effects model for Meta-Analysis of Diagnostic Studies as presented by Reitsma et al. (2005). The goal of the model is to jointly analyze sensitivity and specificity across studies while accounting for both within-study variability and between-study heterogeneity. Firstly, we introduce the model itself, explaining how sensitivity and specificity are defined within the bivariate framework and how the estimated parameters, including between-study variances and correlations, can be used to derive the SROC curve and AUC. Afterwards, we focus on how the model can be fitted and how the SROC curve can be computed, providing a brief overview of the internal steps involved in these calculations.
Model Specification
The bivariate model by Reitsma et al. (2005) jointly analyzes sensitivity and specificity across studies while preserving their two-dimensional nature. Consider \(k\) independent studies. For study \(i\), let \(p_{A,i}\) be the observed sensitivity measured in \(N_{A,i}\) diseased subjects and \(p_{B,i}\) the observed specificity measured in \(N_{B,i}\) non-diseased subjects. After a logit transformation, we define
\[ \theta_{A,i} = \text{logit}(p_{A,i}), \quad \theta_{B,i} = \text{logit}(p_{B,i}). \]
We assume that the true logit-sensitivities \(\theta_{A,i}\) and logit-specificities \(\theta_{B,i}\) are approximately normally distributed across studies:
\[ \theta_{A,i} \sim N(\theta_A, \sigma_A^2), \quad \theta_{B,i} \sim N(\theta_B, \sigma_B^2), \]
and we explicitly allow for a correlation \(\sigma_{AB}\) between logit-sensitivity and logit-specificity within studies. Combining these yields the bivariate normal model:
\[ \begin{pmatrix} \theta_{A,i} \\ \theta_{B,i} \end{pmatrix} \sim N \Bigg( \begin{pmatrix} \theta_A \\ \theta_B \end{pmatrix}, \begin{pmatrix} \sigma_A^2 & \sigma_{AB} \\ \sigma_{AB} & \sigma_B^2 \end{pmatrix} \Bigg) \]
Within each study, the variance of the estimated logits is approximated by the binomial variance:
\[ s_{A,i}^2 = \frac{1}{N_{A,i} p_{A,i} (1 - p_{A,i})}, \quad s_{B,i}^2 = \frac{1}{N_{B,i} p_{B,i} (1 - p_{B,i})}. \]
Treating these within-study variances as fixed, the final model for the observed logits becomes:
\[ \begin{pmatrix} \hat{\theta}_{A,i} \\ \hat{\theta}_{B,i} \end{pmatrix} \sim N \Bigg( \begin{pmatrix} \theta_A \\ \theta_B \end{pmatrix}, \mathbf{C}_i + \begin{pmatrix} \sigma_A^2 & \sigma_{AB} \\ \sigma_{AB} & \sigma_B^2 \end{pmatrix} \Bigg), \]
where \(\mathbf{C}_i = \text{diag}(s_{A,i}^2, s_{B,i}^2)\) contains the within-study variances.
The parameters of the model can be interpreted as follows: \(\theta_A\) and \(\theta_B\) are the overall mean logit-sensitivity and logit-specificity, \(\sigma_A^2\) and \(\sigma_B^2\) capture between-study heterogeneity, and \(\sigma_{AB}\) reflects the correlation between sensitivity and specificity.
SROC Computation
Conceptually, the median SROC curve is obtained by leveraging the conditional expectation of logit-sensitivity given logit-specificity:
\[ \hat{\theta}_A \mid \theta_B = \theta_A + \frac{\sigma_{AB}}{\sigma_B^2} (\theta_B - \theta_B), \]
for a grid of \(\theta_B\) values spanning the observed specificity range. The conditional values are then back-transformed to the original ROC scale via the inverse logit:
\[ \text{TPR} = \text{logit}^{-1}(\hat{\theta}_A \mid \theta_B), \quad \text{FPR} = 1 - \text{logit}^{-1}(\theta_B). \]
Plotting these points produces the SROC curve.
The area under the curve (AUC) can then be approximated by integrating the SROC across the full range of specificity:
\[ \text{AUC} = \int_0^1 \text{TPR}(\text{FPR}) \, d\text{FPR}. \]
Model Application in MetaROC
Inside the package, the model can be fitted by either the
fit_metaROC() function or the
metaROC.metaROC() method when setting
action = "estimate". The model can then be fitted by
setting model = "reitsma2005lmm":
library(metaROC)
set.seed(7)
data(hba1c)
stm_hba1c <- hba1c[hba1c$originally_published == 1,]
fit_reit <- fit_metaROC(stm_hba1c, model = "reitsma2005lmm")## Hello and welcome to metaROC!
## Requested model: reitsma2005lmm
## This is an LMM for a single threshold per study.
## See https://doi.org/10.1016/j.jclinepi.2005.02.022 for more details.
est_reit <- metaROC(action ="estimate", data = hba1c, model = "reitsma2005lmm")## Hello and welcome to metaROC!
## Chosen action: estimate## Warning in metaROC(action = "estimate", data = hba1c, model = "reitsma2005lmm"): You are trying to fit an STM to a real-world dataset that may report multiple thresholds for
## one or more studies. Please specify which threshold should be used by providing a column named
## threshold_stm. Otherwise, each reported threshold will be treated as a separate study.## Hello and welcome to metaROC!
## Requested model: reitsma2005lmm
## This is an LMM for a single threshold per study.
## See https://doi.org/10.1016/j.jclinepi.2005.02.022 for more details.As we have already filtered the HbA1c dataset using the
originally_published column, we do not need to worry about
the warning, since we have only one entry per study.
Internally, the bivariate model is fitted as illustrated in the model
specification section. The observed counts of true positives, false
negatives, false positives, and true negatives are used to compute the
logit-transformed sensitivities and specificities, and the model is
estimated using a linear mixed-effects approach. In practice, we rely on
the reitsma() function from the mada package
in R (Doebler and Sousa-Pinto (2022)),
which fits the model via restricted maximum likelihood (REML) and
returns an S3 object.
Once the model is fitted, we can then use the
summary.metaROC() method to gain an overview of the
estimation results:
summary(fit_reit, ci_type = "wald")##
## *** Results of Single threshold method (STM) ***
##
## Model: LMM for single threshold
##
## Total number of studies: 38
## Total number of thresholds: 38
## Number of different thresholds: 13
##
## Results with Wald confidence intervals:
##
## Youden index (sensitivity weight = 0.5): 0.5310
## Estimated Sensitivity and Specificity [95% CI]:
## Sens: 0.7211 [0.6690; 0.7678]
## Spec: 0.8093 [0.7630; 0.8483]
##
## AUC: 0.8312Firstly, the model name is returned, along with a reminder that the Reitsma LMM is a single-threshold method. Afterwards, a basic overview of the meta-analysis is provided. The output then provides the largest Youden index, although, since the threshold itself is not included in the model, this does not provide additional interpretative insights.
Following this, the summary reports the estimated sensitivity and specificity, along with their corresponding 95% confidence intervals. Finally, the AUC is displayed.
## specificity sensitivity youden_index
## 1 0.99900 0.02490102 0.02390102
## 2 0.98902 0.16336479 0.15238479
## 3 0.97904 0.25384368 0.23288368
## 4 0.96906 0.32293617 0.29199617
## 5 0.95908 0.37866940 0.33774940
## 6 0.94910 0.42510680 0.37420680
## 7 0.93912 0.46467455 0.40379455
## 8 0.92914 0.49895850 0.42809850
## 9 0.91916 0.52905828 0.44821828
## 10 0.90918 0.55576942 0.46494942The SROC for the Reitsma LMM is constructed as illustrated in the
model specification section. Conceptually, the process begins by using
the sroc() function from the mada package to
compute the expected relationship between sensitivity and specificity
across the range of interest based on the estimated bivariate normal
distribution. Following this, point estimates and confidence intervals
for sensitivity and specificity are obtained using
crosshair.reitsma(), which internally applies
inv.trafo() to back-transform the logit-scale estimates to
the original probability scale. Together, these steps produce the median
SROC curve and associated summary measures, representing the typical
diagnostic performance while accounting for both within-study
variability and between-study heterogeneity. The AUC is then obtained by
integrating the SROC curve across the full range of specificity
values.
Currently, the package does not support simulating data from this model, so this concludes the discussion of the Reitsma LMM.
To gain an overview of how to plot a fitted model and how to conduct simulation studies, particularly for evaluating models such as the Reitsma LMM, please refer to the other vignettes included in this package, which provide more detailed guidance on these topics.