Note: Requires Diagacc but as it is not on CRAN thrwos error when compiling once commented out
This vignette focuses on the semiparametric approach for meta-analysis of diagnostic accuracy studies with multiple cut-offs as presented by Frömke et al. (2022). The goal of this method is to synthesize information from all reported thresholds within and across studies, enabling the estimation of summary measures such as sensitivity, specificity, and the AUC without imposing restrictive distributional assumptions on the underlying test values.
Firstly, we introduce the model itself, explaining how the approach uses rank-based estimators and pseudo-individual data to accommodate multiple cut-offs per study. Global and local ranks are employed to derive estimates of sensitivity, specificity, and AUC, while taking between-study heterogeneity into account. We also describe how pseudo-individual data are generated from aggregated study-level information when full individual-level data are unavailable.
Subsequently, we discuss how these components can be used to compute
confidence intervals for the summary measures and how the method
accounts for within-study correlation via rank-based variance
estimators. Afterwards, we outline how the estimated ranks and
pseudo-data can be combined to construct summary ROC curves and overall
measures of diagnostic performance across multiple thresholds. Finally
we discuss how the model can be fitted inside the metaROC
package.
Model specification
This section describes the semiparametric approach for meta-analysis of diagnostic accuracy studies with multiple cut-offs, following Frömke et al. (2022) Frömke et al. (2022).
Data and Ranks
Consider a set of studies \(s = 1, \dots, N\). Each study reports diagnostic test results for individuals \(k = 1, \dots, w_{si}\) with condition \(i = 0\) (without target condition) or \(i = 1\) (with target condition).
Let \(R_{sik}\) denote the global rank of observation \(X_{sik}\) across all studies, and \(R^{(i)}_{sik}\) the local rank within condition \(i\) across all studies. Define the rank differences:
\[D_{sik} = R_{sik} - R^{(i)}_{sik,}\] \[D_{si.} = \sum_{k=1}^{w_{si}} D_{sik}\] \[D_{.i} = \sum_{s=1}^{N} D_{si}\]
AUC Estimation
The AUC is expressed as the relative effect:
\[\hat{p} = \frac{1}{2} + \frac{1}{w_{::}} (\bar{R}_{:1} - \bar{R}_{:0}),\]
where \(w_{:i} = \sum_{s=1}^{N} w_{si}\) is the total number of individuals with condition \(i\), \(w_{::} = w_{:0} + w_{:1}\), and \(\bar{R}_{:i}\) is the mean global rank for condition \(i\).
Variance of the AUC Estimator
The variance of \(\hat{p}\) is computed as:
\[\hat{\sigma}^2 = \frac{N^2}{N-1} \left[\frac{1}{w_{.0} w_{.1}}\right]^2 \sum_{s=1}^{N} \Bigg( D_{s1.} - D_{s0.} - \Big( \frac{w_{s1}}{w_{.1}} D_{.1} - \frac{w_{s0}}{w_{.0}} D_{.0} \Big) \Bigg)^2.\]
This formula accounts for the correlation structure among individuals within the same study and properly weights studies by their sample sizes in each condition.
Confidence Interval for Small Samples
For small numbers of studies, the confidence interval for the AUC uses a t-approximation:
\[\hat{p} \pm t_{N-1, 1-\alpha/2} \sqrt{\frac{\hat{\sigma}^2}{N}}\]
where \(t_{N-1, 1-\alpha/2}\) is the \(1-\alpha/2\) quantile of the t-distribution with \(N-1\) degrees of freedom, and \(N\) being the number of studies.
Sensitivity and Specificity at Cut-offs
Sensitivity and specificity at a chosen threshold \(\gamma\) are estimated by replacing observations of one condition with a pseudo-sample at \(\gamma\):
- Sensitivity: replace all non-diseased (\(i=0\)) observations with a point mass at
\(\gamma\)
- Specificity: replace all diseased (\(i=1\)) observations with a point mass at \(\gamma\)
Ranks and AUC formulas are applied to these pseudo-samples, yielding rank-based estimates of sensitivity and specificity with corresponding confidence intervals.
This formulation allows meta-analysis of diagnostic accuracy across multiple thresholds while accounting for the ranking structure within and across studies, producing AUC estimates and confidence intervals suitable for small numbers of studies.
Application in MetaROC
Inside the package, the model can be fitted by either the
fit_metaROC() function or the
metaROC.metaROC() method when setting
action = "estimate". The model can then be fitted by
setting model = "froemke2022np":
library(metaROC)
set.seed(7)
data(hba1c)
#fit_fr <- fit_metaROC(hba1c, model = "froemke2022np")
# long computational time but fits the exact same model using `metaROC()` method
# est_fr <- metaROC(action ="estimate", data = hba1c, model = "froemke2022np")Internally, the semiparametric model for meta-analysis with multiple
thresholds (Frömke et al. (2022)) is
fitted using the diagacc::estimation() function from the
diagacc R package.
Conceptually, the fitting proceeds as follows:
Preprocessing and pseudo-data generation: If only aggregated data are available, pseudo-individual data are generated using a uniform distribution between study-specific cut-offs. This allows the rank-based methodology to be applied even without individual participant data.
Rank-based AUC and variance computation: The algorithm computes global and local ranks across studies to estimate the area under the curve (AUC) and its variability, accounting for differences between ranks within and across studies.
Confidence intervals for small samples: For a small number of studies, confidence intervals for the AUC are constructed using a t-distribution approximation, which improves statistical properties compared with standard normal-based intervals.
Threshold-specific sensitivity and specificity: For each study-specific threshold, pseudo-samples are created at the threshold value, and the rank-based methodology is applied to estimate sensitivity and specificity at that cut-off, together with their confidence intervals.
Iterative computation across thresholds: When multiple thresholds are reported, these steps are repeated for each threshold to produce a complete ROC curve and associated performance measures.
The returned object contains the estimated AUC, confidence intervals, threshold-specific sensitivity and specificity, a convergence status, and the original data. The fitting procedure is conceptually simple, relying on rank computations rather than iterative optimization, which makes it robust and efficient even with a small number of studies.
#summary(fit_fr, ci_type = "wald")First the model name is returned, along with a reminder that the Frömke et al. (2022) semiparametric model is a multiple threshold method. Afterwards, a basic overview of the meta-analysis is provided. Afterwards the output provides the largest Youden index together with the optimal threshold.
Following the estimation of the model parameters, the summary reports the estimated sensitivity and specificity for the diagnostic test, along the corresponding 95% confidence intervals sensitivity and specificity. Finally, the AUC is displayed, providing a summary measure of the overall diagnostic accuracy.
#SROC <- SROC(fit_fr)
#head(SROC$sroc_df, 10)The SROC for the Frömke et al. (2022) semiparametric model is constructed as described in the model specification section.
Conceptually, the SROC curve is generated by pairing the estimated sensitivity and specificity at each study-specific threshold. These threshold-specific points form the ROC curve across the observed range of cut-offs. The Youden index can be calculated for each threshold, and the AUC provides a single summary measure of overall diagnostic performance.
Currently, the package does not support simulating data from this model, so this concludes the discussion of the Frömke et al. (2022) semiparametric model.
To gain an overview of how to plot a fitted model and how to conduct simulation studies, particularly for evaluating models such as the Frömke et al. (2022) semiparametric model, please refer to the other vignettes included in this package, which provide more detailed guidance on these topics.