This vignette focuses on the fully non‑parametric approach for ROC curve estimation in random‑effects meta‑analysis as presented by Martinez-Camblor (2017). The goal of this methodology is to estimate an overall ROC curve from diagnostic studies without relying on a specific parametric form for the curve or the distribution of study effects. In contrast to parametric and semiparametric meta‑analytic methods, this approach uses the information from all available cut‑off points reported in the original studies and avoids imposing a particular statistical distribution on sensitivity and specificity.
Firstly, we introduce the non‑parametric estimator itself and explain how individual study ROC curves are interpolated and combined to produce a summary curve under both fixed‑ and random‑effects assumptions. Rather than modeling sensitivity and specificity jointly through latent variables or predetermined functional forms, the method weights and averages the observed ROC points from the contributing studies. This allows the summary curve to reflect the empirical relationship between true and false positive rates in the available data without structural assumptions.
Afterwards, we focus on how the non‑parametric ROC estimator can be
computed in practice, including how weights are chosen under
random‑effects meta‑analysis and how confidence bands are constructed.
The approach is implemented in the nsROC package in R,
which provides functions for estimating and plotting the summary ROC
curve based on this fully non‑parametric methodology.
Model Specification
General model specification
The fully non‑parametric ROC meta-analysis approach of Martinez-Camblor (2017) estimates a summary ROC curve directly from the observed sensitivity and specificity pairs reported by the individual studies, without assuming any specific parametric distribution for the curve.
Each original study contributes a set of sensitivity–specificity pairs (for example, for different thresholds), and a study‑specific ROC curve is obtained by interpolating these points. Let \(\widehat{R}_j(t)\) denote the ROC curve estimate for study \(j\) at false positive rate (FPR) \(t\). These individual ROC curve estimates are then combined across studies to define a summary ROC (sROC) function.
A key idea in the non‑parametric framework is that for each value of FPR \(t \in [0,1]\), the summary ROC curve is defined as the supremum (maximum) of the individual ROC values that meet the specificity constraint. In its general form, this can be expressed as:
\[ \widehat{R}_g(t) = \sup_{\gamma \in [0,1]} \big\{ F_{\xi}\big(F^{-1}_{\chi}(\gamma\,t)\big) + 1 - F_{\xi}\big(F^{-1}_{\chi}(1 - (1-\gamma)\,t)\big) \big\}, \]
Here, \(\widehat{R}_g(t)\) represents the summary ROC curve estimate at a given false positive rate \(t\). The supremum (sup) means we take the highest value of sensitivity that is compatible with the observed specificity constraints across all studies.
The fully non‑parametric estimator using empirical distributions can be written as:
\[ \widehat{\mathcal{R}}_g(t) = \sup_{\gamma \in [0,1]} \big\{ 1 - \hat{F}_{1 - \hat{F}_{\chi}(\xi)}\big(1 - \gamma\,t\big) + \hat{F}_{1 - \hat{F}_{\chi}(\xi)}\big((1 - \gamma)\,t\big) \big\}, \]
In this expression, the form involving \(\hat{F}\) uses empirical cumulative distribution functions, meaning it relies directly on the observed data rather than assuming any specific distribution shape. This effectively uses all reported cut-offs and ROC points, rather than just one pair per study (like the Youden index), making it fully non‑parametric.
Intuitively, the sROC curve is constructed by taking the best possible combination of observed ROC points across studies at each FPR, resulting in a summary curve that is entirely data-driven and flexible.
The fully non-parametric model uses all reported sensitivity–specificity pairs from each study, which correspond to the different thresholds evaluated in the original studies. Each study’s ROC curve is interpolated across these observed points, so the thresholds themselves do not need to be explicitly modeled. The summary ROC curve is then constructed by combining the interpolated curves across studies at each false positive rate, effectively aggregating information from all thresholds. This approach allows the model to incorporate multiple operating points per study and ensures that the sROC curve reflects the full empirical range of thresholds.
Fixed-effects model
In the fixed-effects version of the fully non-parametric ROC meta-analysis, it is assumed that all studies estimate the same underlying ROC relationship, and any differences between studies are due solely to within-study variability. Each study’s ROC curve is interpolated from its observed sensitivity–specificity points.
The summary ROC curve is constructed as a weighted average of the individual curves:
\[ \widehat{ROC}_{\text{FE}}(t) = \sum_{j=1}^{J} w_j(t)\,\widehat{ROC}_j(t), \]
where \(\widehat{ROC}_j(t)\) is the interpolated ROC value from study \(j\) at false positive rate \(t\), and the weights \(w_j(t)\) are proportional to study sample sizes:
\[ w_j(t) = \frac{n_j}{\sum_{k=1}^{J} n_k}. \]
This ensures that larger studies contribute more to the summary curve. Confidence bands around the summary curve can also be computed to reflect the uncertainty due to within-study variation.
Random-effects model
The random-effects version relaxes the assumption of a common underlying ROC and allows for between-study heterogeneity. The summary ROC curve is still a weighted combination of the interpolated study-specific curves:
\[ \widehat{ROC}_{\text{RE}}(t) = \sum_{j=1}^{J} w_j(t)\,\widehat{ROC}_j(t), \]
but the weights now account for both within-study variance \(\sigma_j^2(t)\) and between-study variance \(\tau^2(t)\):
\[ w_j(t) = \frac{1}{\sigma_j^2(t) + \tau^2(t)} \Big/ \sum_{k=1}^{J} \frac{1}{\sigma_k^2(t) + \tau^2(t)}. \]
This means that studies that deviate more from the overall pattern contribute less to the summary estimate, producing a curve that reflects the average diagnostic performance while accounting for heterogeneity across studies. Confidence bands are wider than in the fixed-effects case, reflecting the additional uncertainty due to variability between studies.
Model Application in MetaROC
Inside the package, the model can be fitted by either the
fit_metaROC() function or the
metaROC.metaROC() method when setting
action = "estimate". The model can then be fitted by
setting model = "martinez2017np":
library(metaROC)
set.seed(7)
data(hba1c)
fit_mart <- fit_metaROC(hba1c, model = "martinez2017np")## Hello and welcome to metaROC!
## Requested model: martinez2017np
## This is a non-parametric model for multiple thresholds.
## See https://doi.org/10.1177/0962280214537047 for more details.
## Number of papers included in meta-analysis: 38
## Model considered: random-effects## The area under the summary ROC curve (AUC) is 0.773.
## The optimal specificity and sensitivity (in the Youden index sense) for summary ROC curve are 0.797 and 0.645, respectively.
est_mart <- metaROC(action ="estimate", data = hba1c, model = "martinez2017np")## Hello and welcome to metaROC!
## Chosen action: estimate
## Hello and welcome to metaROC!
## Requested model: martinez2017np
## This is a non-parametric model for multiple thresholds.
## See https://doi.org/10.1177/0962280214537047 for more details.
## Number of papers included in meta-analysis: 38
## Model considered: random-effects## The area under the summary ROC curve (AUC) is 0.773.
## The optimal specificity and sensitivity (in the Youden index sense) for summary ROC curve are 0.797 and 0.645, respectively.Internally, the fully non-parametric meta-analysis model Martinez-Camblor (2017) is fitted using the
nsROC::metaROC() function. This function takes the study
data, including all available sensitivity and specificity points, and
constructs a summary ROC curve by interpolating the individual study ROC
curves and then combining them according to the chosen variant. Each
study curve is interpolated using a piecewise linear algorithm between
the observed ROC points, ensuring monotonicity of the curve. The output
is an S3 object containing the estimated summary ROC curve, optionally
confidence bands, and other metadata describing the fit.
summary(fit_mart, ci_type = "wald")##
## *** Results of Multiple threshold method (MTM) ***
##
## Model: Non-parametric model
## Variant: random-effects
##
## Total number of studies: 38
## Total number of thresholds: 124
## Number of different thresholds: 26
##
## Results with Wald confidence intervals:
##
## Youden index (sensitivity weight = 0.5): 0.4417
## Estimated Sensitivity and Specificity [95% CI]:
## Sens: 0.6449 [0.6287; 0.6610]
## Spec: 0.7968
##
## AUC: 0.7731Firstly, the model name is returned, along with a reminder that the Martinez-Camblor (2017) non parametric model is a multiple threshold method. We are also reminded which variant of the Martinez-Camblor (2017) was used. Afterwards, a basic overview of the meta-analysis is provided. The output then provides the largest Youden index, although, since the threshold itself is not direcetly modeled, this does not provide additional interpretative insights.
Following the estimation of the model, the summary reports the estimated sensitivity and specificity for the diagnostic test. Because the fully non-parametric approach of Martinez-Camblor (2017) is based on the observed sensitivity–specificity pairs and interpolates across them, confidence intervals are only provided for the sensitivity at each false positive rate, while specificity is treated as the reference axis and does not have associated confidence intervals. The method effectively uses all reported thresholds and ROC points from each study, rather than assuming a parametric form, so the uncertainty is captured primarily in the sensitivity dimension. Finally, the AUC is displayed, providing a summary measure of the overall diagnostic accuracy.
## sens_lo sensitivity sens_up spec_lo specificity spec_up
## 1 0.0009810911 0.003397911 0.005814731 NA 1.000000 NA
## 2 0.0096528764 0.012069697 0.014486517 NA 0.998999 NA
## 3 0.0190426951 0.022534404 0.026026113 NA 0.997998 NA
## 4 0.0284154076 0.032780894 0.037146379 NA 0.996997 NA
## 5 0.0377867632 0.042922271 0.048057778 NA 0.995996 NA
## 6 0.0471505888 0.052989076 0.058827563 NA 0.994995 NA
## 7 0.0565095068 0.063003010 0.069496514 NA 0.993994 NA
## 8 0.0658730349 0.072985231 0.080097427 NA 0.992993 NA
## 9 0.0752530876 0.082955417 0.090657746 NA 0.991992 NA
## 10 0.0829278836 0.091072204 0.099216525 NA 0.990991 NA
## youden_index
## 1 0.003397911
## 2 0.011068696
## 3 0.020532402
## 4 0.029777891
## 5 0.038918267
## 6 0.047984071
## 7 0.056997004
## 8 0.065978224
## 9 0.074947409
## 10 0.082063195The summary ROC curve is obtained directly from the fitted
nsROC::metaROC() object. For each false positive rate, the
function extracts the estimated sensitivity and constructs confidence
intervals for sensitivity only. The summary curve is then combined into
a single data frame, and a Youden-index–based metric can be calculated
to identify the optimal balance between sensitivity and specificity.
Finally, the AUC is computed numerically from the summary ROC points,
providing a single measure of the overall diagnostic accuracy across all
studies.
Currently, the package does not support simulating data from this model, so this concludes the discussion of the Martinez-Camblor (2017) non parametric model.
To gain an overview of how to plot a fitted model and how to conduct simulation studies, particularly for evaluating models such as the Martinez-Camblor (2017) non parametric model, please refer to the other vignettes included in this package, which provide more detailed guidance on these topics.