This vignette focuses on the Moses–Shapiro–Littenberg approach for meta-analysis of diagnostic studies as presented by Moses et al. (1993). The goal of this method is to construct a summary receiver operating characteristic (SROC) curve by combining evidence from multiple studies using a regression-based framework. Firstly, we introduce the model itself, explaining how study-specific measures are transformed and how the SROC curve can be derived from the estimated regression parameters. Afterwards, we focus on how the model can be fitted and how the SROC curve can be computed, providing a brief overview of the internal steps involved in these calculations.
Model Specification
The MSL approach is based on an empirical transformation of sensitivity and specificity using the logit function. Let \(TPR_i\) and \(FPR_i\) denote the true positive rate (sensitivity) and false positive rate for study \(i\). These are transformed using the logit transformation
\[ U_i = logit(FPR_i) = \log\left(\frac{FPR_i}{1 - FPR_i}\right), \]
\[ V_i = logit(TPR_i) = \log\left(\frac{TPR_i}{1 - TPR_i}\right). \]
This transformation maps the unit square (ROC space) onto the entire real plane, which facilitates modeling. Under certain distributional assumptions, in particular when the underlying test results follow a logistic distribution, the ROC curve becomes approximately linear in the \((U, V)\) space.
To simplify interpretation, the transformed variables are further reparameterized as
\[ S_i = V_i + U_i, \quad D_i = V_i - U_i. \]
Here, \(S_i\) represents a measure related to the implicit diagnostic threshold used in study \(i\), while \(D_i\) corresponds to the log diagnostic odds ratio and reflects the discriminatory power of the test.
The relationship between these quantities is modeled using a linear regression
\[ D_i = A + B S_i. \]
The parameter \(A\) captures the overall diagnostic accuracy, while \(B\) reflects the effect of varying thresholds across studies. In particular, \(B = 0\) corresponds to a symmetric ROC curve.
In practice, zero cell counts in the \(2 \times 2\) tables may lead to undefined logits. To address this, a continuity correction is applied by adding \(1/2\) to each cell count before computing the transformed quantities. This adjustment is asymptotically negligible but ensures numerical stability in finite samples.
Construction of the SROC curve
The corresponding ROC curve can be computed by inserting a value for \(P \in (0,1)\) and evaluating
\[ Q = \left[1 + \exp\left(-\frac{A}{1 - B}\right) \left(\frac{1 - P}{P}\right)^{\frac{1 + B}{1 - B}} \right]^{-1}. \]
The SROC curve is then obtained as the locus of points \((P, Q)\) for varying values of \(P\).
In practice, this means that a grid of false positive rates is chosen, and the corresponding true positive rates are computed using the above expression. This provides a smooth estimate of the ROC curve implied by the fitted regression model.
It is important to note that the linear regression in \((S, D)\) space can be influenced by observations corresponding to extreme values of \(P\) and \(Q\). These points may have a disproportionate impact on the estimated slope, even though they are often of limited practical relevance. For this reason, it is common to focus on a clinically relevant region of the ROC space when interpreting or plotting the resulting SROC curve.
The calculation of the AUC is then straightforward, as it can be obtained by numerically integrating the estimated SROC curve over the range of false positive rates.
Model Application in MetaROC
bunch of code
Inside the package, the model can be fitted by either the
fit_metaROC() function or the
metaROC.metaROC() method when setting
action = "estimate". The model can then be fitted by
setting model = "moses1993lm":
library(metaROC)
set.seed(7)
data(hba1c)
stm_hba1c <- hba1c[hba1c$originally_published == 1,]
fit_mls <- fit_metaROC(stm_hba1c, model = "moses1993lm")## Hello and welcome to metaROC!
## Requested model: moses1993lm
## This is a LM for a single threshold per study.
## See https://doi.org/10.1002/sim.4780121403 for more details.
est_mls <- metaROC(action ="estimate", data = hba1c, model = "moses1993lm")## Hello and welcome to metaROC!
## Chosen action: estimate## Warning in metaROC(action = "estimate", data = hba1c, model = "moses1993lm"): You are trying to fit an STM to a real-world dataset that may report multiple thresholds for
## one or more studies. Please specify which threshold should be used by providing a column named
## threshold_stm. Otherwise, each reported threshold will be treated as a separate study.## Hello and welcome to metaROC!
## Requested model: moses1993lm
## This is a LM for a single threshold per study.
## See https://doi.org/10.1002/sim.4780121403 for more details.As we have already filtered the HbA1c dataset using the
originally_published column, we do not need to worry about
the warning, since we have only one entry per study.
Internally, the model is fitted by first transforming the observed study-level data into the logit space. Based on the reported counts of true positives, false positives, false negatives, and true negatives, the FPR and TPR are computed. To ensure numerical stability in the presence of zero cell counts, a continuity correction is applied before calculating the corresponding logit-transformed values.
From these transformed quantities, the variables \(U\) and \(V\) are obtained, which are then used to construct the summary measures \(S = V + U\) and \(D = V - U\). Optionally, the data can be restricted to a user-defined region of interest in the ROC space by specifying bounds on the FPR and TPR, thereby limiting the influence of less relevant observations.
Subsequently, a simple linear regression model is fitted (using the
stats::lm() function), relating \(D\) to \(S\) using ordinary least squares. The
resulting parameter estimates are then used to construct the SROC curve
as described in the previous section.
Once the model is fitted, we can then use the
summary.metaROC() method to gain an overview of the
estimation results:
summary(fit_mls, ci_type = "wald")##
## *** Results of Single threshold method (STM) ***
##
## Model: LM for single threshold
## Max FPR: 1 ; Min TPR: 0
##
## Total number of studies: 38
## Total number of thresholds: 38
## Number of different thresholds: 13
##
## Results with Wald confidence intervals:
##
## Youden index (sensitivity weight = 0.5): 0.5513
## Estimated Sensitivity and Specificity [95% CI]:
## Sens: 0.7713 [0.4643; 1.0783]
## Spec: 0.7800
##
## AUC: 0.8440Firstly, the model name is returned, along with a reminder that the Moses et al. (1993) model is a single-threshold method. Then we are also told which maximum false positive and minium true positive rate was used for constructing the SROC curve. Afterwards, a basic overview of the meta-analysis is provided. The output then provides the largest Youden index, although, since the threshold itself is not included in the model, this does not provide additional interpretative insights.
Following this, the summary reports the estimated sensitivity and specificity. As the sensitivity values for the ROC curve are calculated given chosen specificity values, there is no confidence interval for the specificity. Finally, the AUC is displayed.
By calling the SROC function we can also gain the fully
estimated SROC curve from the model
## specificity sensitivity sens_lo sens_up diff youden_index
## 1 1.00 0.0000000 0.000000000 0.0000000 1.0000000 0.0000000
## 2 0.99 0.1101125 -9.827085338 10.0473104 0.8798875 0.1001125
## 3 0.98 0.1991818 -3.722933085 4.1212967 0.7808182 0.1791818
## 4 0.97 0.2731453 -1.933078694 2.4793694 0.6968547 0.2431453
## 5 0.96 0.3356258 -1.106299073 1.7775506 0.6243742 0.2956258
## 6 0.95 0.3891383 -0.638591580 1.4168682 0.5608617 0.3391383
## 7 0.94 0.4355020 -0.342310893 1.2133149 0.5044980 0.3755020
## 8 0.93 0.4760702 -0.141235509 1.0933760 0.4539298 0.4060702
## 9 0.92 0.5118724 0.001263817 1.0224810 0.4081276 0.4318724
## 10 0.91 0.5437064 0.105087693 0.9823252 0.3662936 0.4537064The SROC for the MLS method is constructed as illustrated in the model specification section. Internally, the SROC curve is computed by first taking the fitted linear regression in \((S, D)\) space and reversing the transformations to obtain estimated TPR for a grid of FPR. Specifically, for each value of \(P\) (the FPR), the corresponding \(Q\) (the TPR) is calculated using the relation derived from the regression parameters \(A\) and \(B\):
\[ Q = plogis\left(\frac{A + (B+1) \cdot qlogis(P)}{1 - B}\right), \]
where plogis and qlogis denote the logistic
and inverse logistic transformations, respectively. Standard errors for
the estimated TPRs are computed using the delta method.
Once the TPR values and their confidence bounds are obtained, the curve can optionally be restricted to a user-defined region of interest in the ROC space by setting bounds on FPR and TPR. The AUC is calculated by numerically integrating the estimated curve.
Currently, the package does not support simulating data from this model, so this concludes the discussion of the MLS method.
To gain an overview of how to plot a fitted model and how to conduct simulation studies, particularly for evaluating models such as the Moses et al. (1993) model, please refer to the other vignettes included in this package, which provide more detailed guidance on these topics.