Plotting
The package also includes basic plotting functions for easier visualization of the data, as well as for visualizing the resulting model and evaluating its performance. Below, the basic functionality of these functions will be demonstrated. Keep in mind, there are various input parameters available to customize the resulting plots according to individual needs. For more details on these parameters, please refer to the documentation. Some of the plotting functions may prompt additional instructions in the console based on the provided dataset. Following these instructions will generate the next plot.
First, a simulated dataset will be created and then a model fitted.
Additionally, the real world dataset hba1c will be loaded
since real wold data can also be plotted.
library(metaROC)
set.seed(8)
control_sim = list(n_studies_min = 5, n_studies_max = 10,
n_subj_min = 20, n_subj_max = 500,
n_thresholds_min = 1, n_thresholds_max = 4,
thresholds_lower_bound = 5,
thresholds_upper_bound = 7,
true_preval_min = 0.1, true_preval_max = 0.3,
model = "stoye2024cloglog",
model_pars = list(beta11 = -38.2, beta12 = 10.26,
beta13 = -0.669,
beta21 = 3.516, beta22 = -0.306,
var_rand_H = 0.6561,
var_rand_D = 0.9216))
sim <- simulate_metaROC(type = "phaseII", n_sims = 5,
control = control_sim)
fitted_model <- fit_metaROC(data = sim$data[sim$data$replicate==5,],
model = "stoye2024cloglog", data_info = sim$data_info)
#> Hello and welcome to metaROC!
#> Requested model: stoye2024cloglog
#> This is a GLMM for the discrete Hazard for multiple thresholds.
#> See https://doi.org/10.1002/jrsm.1753 for more details.
#> Warning in fit_metaROC_stoye2024dGLMM(data = data, model = "cloglog", nAGQ =
#> nAGQ): NAs introduced by coercion
data("hba1c")Plotting Data
To plot either real-world or simulated data, simply provide the
dataset to the “plot_data()” function. If real-world data is used, it
has to include all the necessary columns also needed for model fitting.
For simulated data, one can directly use the object returned by
simulate_metaROC(). The function automatically detects
whether the dataset is real-world or simulated data. When real-world
data is provided, the function generates plots of study-specific ROC
curves. In contrast, simulated data plots include the true ROC curve
alongside another plot depicting True Sensitivity-Specificity against
Threshold curves. In cases where studies only report results for a
single threshold, only a point instead of a curve is available on the
ROC.
First the hba1c dataset will be plotted:
plot_data(hba1c)
#> Study specific ROC curves plottedAs this is a real world dataset, each study specific is plotted without an underyling true ROC curve.
Now the simulated data will be plotted. As explained earlier simply
providing the simulated dataset by simulate_metaROC() is
enough:
plot_data(sim, replicate = 5)
#> Study specific ROC curves plotted
#>
#> Press Enter to see the sensitivity/specificity plot against threshold...
Now the study specific ROC curve plot also includes the true ROC curve as the dataset was flagged as being simulated. The output also includes another plot, which appears upon pressing enter into the console. This plot highlights the true sensitivity and specificity values per threshold for a first investigation of which diagnostic threshold might be suitable.
Plotting a fitted model
Visulization of model estimates serves as a valuable tool, especially
for evaluation. While sifting through numerous numbers can be tedious,
appropriate plots can provide the same information and contextualize it
effectively. To facilitate this, the plot.metaROC() method
has been implemented. This method is designed for a fitted model of
class metaROC. The function generates up to five plots in
total. The first two plots are identical to the ones
plot_data() provides to visualize the data, with the 2nd
plot only shown if the data was simulated. The 3rd plot looks similar to
the first plot when setting studies = TRUE, visualizing the
individual study curves in grey, but includes the estimated ROC (and
true ROC when simulated) curve with its confidence interval. It is
followed by two Sensitivity-Specificity against threshold plots: one
using true values and another using estimated values, allowing for
coverage checks with confidence intervals in both. Lastly, a Youden
Index against threshold plot identifies the optimal threshold according
to the model under the specified Youden weight by
youd_weight.
plot(fitted_model, youd_weight = 0.6, studies = TRUE)
#> Study specific ROC curves plotted
#>
#> Press Enter to see the sensitivity/specificity plot against threshold...
#> Estimated ROC + grey study curves plotted. Press Enter to continue...
#> Model sens/spec vs threshold plotted. Press Enter to continue...
#> Youden index plotted.It is evident that, due to the small number of included studies, uncertainty in model estimates is high.
While plot.metaROC() is primarily meant to help with
model evaluation when info on the data generating process is provided,
it still produces helpful plots if the model was fitted to real-world
data. The plots then can not show the true ROC curve as well as the true
sensitivity and specificity values. However, it still plots the
estimated ROC curve as well as the estimated sensitivity and specificity
values. Finally, it can still serve as a tool to find the optimal
threshold estimated by the model depending on the weight assigned:
mod <- fit_metaROC(hba1c, model = "stoye2024cloglog")
#> Hello and welcome to metaROC!
#> Requested model: stoye2024cloglog
#> This is a GLMM for the discrete Hazard for multiple thresholds.
#> See https://doi.org/10.1002/jrsm.1753 for more details.
#> Warning in fit_metaROC_stoye2024dGLMM(data = data, model = "cloglog", nAGQ =
#> nAGQ): NAs introduced by coercion
plot(mod, studies = TRUE)
#> Study specific ROC curves plotted
#> Estimated ROC + grey study curves plotted. Press Enter to continue...
#> Model sens/spec vs threshold plotted. Press Enter to continue...
#> Youden index plotted.It should be noted: The plots might not look perfect in some edge cases and may not capture every detail of the model. They are mainly designed to help visualize model performance and make evaluation easier for users who aren’t as comfortable with coding.