Basic Workflow
First of all the package needs to be installed from CRAN (Comprehensive R Archive Network) or another repository. You can do this by running ‘install.packages(“MetaROC”)’ in your R console. This command will download and install the MetaROC package along with its dependencies.
Once the package is installed, you can load it into your R session using the ‘library(MetaROC)’ command. This command makes all the functions and datasets within the MetaROC package available for use in your R scripts or interactive sessions.
Setting a seed prior can also be useful for reproducibility, especially when using simulation functions. When using this package, it is very likely that data will be simulated. Hence, it is recommended to set a seed using a random integer.
In the following sections, the practical workflow for meta-analyzing ROC curves using GLMMs with simulated data will be demonstrated. If the analysis is to be performed on a real-world dataset, the first part will show how this can be done.
Since the package includes many more functionalities and some of the showcased functions have numerous input parameters, please refer to the documentation using ?name_of_function when using these functions.
Using real data
When using real event data, a dataframe with the following information for each study is required: study ID (a unique integer for each study), diagnostic threshold, number of healthy individuals (H), number of diseased individuals (D), true positives (TP), and true negatives (TN). By calling the ‘preprocess_event_data’ function with the dataframe as an argument, the data can be preprocessed for use with one of the fitting functions.
In the following, we define an imaginary dataset to show how a preprocessed dataframe should look. The dataset, ‘event_data’, includes information from three studies with IDs 1-3, each using two diagnostic thresholds (2 and 4). For each study and threshold, the dataframe provides the study ID, diagnostic threshold, number of healthy individuals (H), diseased individuals (D), true positives (TP), and true negatives (TN):
event_data <- data.frame(
study = rep(1:3, each = 2),
threshold = rep(c(2, 4), times = 3),
H = c(40, 55, 25, 35, 45, 60),
D = c(15, 25, 10, 20, 20, 30),
TP = c(10, 18, 8, 15, 20, 25),
TN = c(35, 40, 20, 30, 25, 50)
)
prepoccessed_data <- preprocess_event_data(event_data)
prepoccessed_data
#> # A tibble: 24 × 13
#> study threshold group event D H n TP TN FP FN freq
#> <int> <fct> <fct> <fct> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 1 2 H 1 15 40 55 10 35 5 5 35
#> 2 1 2 D 1 15 40 55 10 35 5 5 5
#> 3 1 2 H 0 15 40 55 10 35 5 5 5
#> 4 1 2 D 0 15 40 55 10 35 5 5 10
#> 5 1 4 H 1 25 55 80 18 40 15 7 40
#> 6 1 4 D 1 25 55 80 18 40 15 7 7
#> 7 1 4 H 0 25 55 80 18 40 15 7 15
#> 8 1 4 D 0 25 55 80 18 40 15 7 18
#> 9 2 2 H 1 10 25 35 8 20 5 2 20
#> 10 2 2 D 1 10 25 35 8 20 5 2 2
#> # ℹ 14 more rows
#> # ℹ 1 more variable: step <dbl>The preprocessed data now contains all the additional information needed for further analysis: the total number of participants (n), false positives (FP), false negatives (FN), event (where 1 represents being diagnosed as healthy and 0 as diseased), frequency (the number of individuals for the specific group and event at the current threshold), and step (indicating which unique threshold the current one corresponds to, converted into a numeric format). The preprocessed data is now presented as a tibble, which is a modern reimagining of the data frame in R. With these added columns, the tibble has all necessary information for the fitting functions and subsequent analyses.
Now calling one of the fitting functions with ‘preproccessed_data’ set as an argument will fit a model to the real world data.
Simulation
For simulation, a single function is used, allowing each parameter to be modified according to specific needs. Users can specify the range for the number of studies in the meta-analysis, the number of subjects per study, the number of thresholds, and the prevalence. Additionally, users need to define various parameters depending on the chosen distribution. For simulating copula data, further details must be provided.
By simply calling the function with its default values, data is simulated from a Generalized Linear Mixed Model (GLMM) using the complementary log-log (cloglog) link function.
sim <- simulate_metaROC()Using $ notation to call the different list entries from the ‘sim’ object allows one to retrieve and save the outputs from the simulation function for quicker access.
sim$data
#> # A tibble: 880 × 20
#> simnum study group event threshold n D H row_number rand_eff
#> <int> <fct> <fct> <fct> <fct> <int> <int> <int> <int> <dbl>
#> 1 1 1 D 1 5.1 327 122 205 1 2.07
#> 2 1 1 H 1 5.1 327 122 205 2 1.36
#> 3 1 1 D 0 5.1 327 122 205 3 2.07
#> 4 1 1 H 0 5.1 327 122 205 4 1.36
#> 5 1 1 D 1 5.6 327 122 205 5 2.07
#> 6 1 1 H 1 5.6 327 122 205 6 1.36
#> 7 1 1 D 0 5.6 327 122 205 7 2.07
#> 8 1 1 H 0 5.6 327 122 205 8 1.36
#> 9 1 1 D 1 5.7 327 122 205 9 2.07
#> 10 1 1 H 1 5.7 327 122 205 10 1.36
#> # ℹ 870 more rows
#> # ℹ 10 more variables: beta1 <dbl>, beta2 <dbl>, linear_term <dbl>, prop <dbl>,
#> # true_sens <dbl>, true_spec <dbl>, TP <int>, TN <int>, step <dbl>,
#> # freq <int>Just like when preprocessign real event data, the ‘data’ entry of the returned list contains the total number of participants (n), false positives (FP), false negatives (FN), event (where 1 represents being diagnosed as healthy and 0 as diseased), frequency (the number of individuals for the specific group and event at the current threshold), and step (indicating which unique threshold the current one corresponds to, converted into a numeric format).
Additionally, it contains within-study variability through the random effect, beta parameters and the linear predictor for modeling. True sensitivity and specificity values, vital for assessing model performance, are also included.
When fitting and evaluating a model later, only this entry, needs to be provided to the respective fitting function.
sim$model_id
#> [1] "cloglog"The ‘id’ entry stores the model type from which the data was simulated. Since nothing else was defined, the data was simulated from the default cloglog GLMM.
Lastly, the true sensitvity and specificty values for each threshold are also stored seperately for quicker access:
sim$true_sens
#> [1] 9.743437e-01 9.390857e-01 8.918839e-01 8.305920e-01 7.538201e-01
#> [6] 6.616916e-01 5.566446e-01 4.439531e-01 3.315107e-01 2.285146e-01
#> [11] 1.431558e-01 8.015143e-02 3.941465e-02 1.672865e-02 6.026129e-03
#> [16] 1.814647e-03 4.510439e-04 9.169205e-05 1.517260e-05 2.044557e-06
#> [21] 2.257752e-07
sim$true_spec
#> [1] 0.1493053 0.3188741 0.4961940 0.6635065 0.8025351 0.9014054 0.9596317
#> [8] 0.9869963 0.9968503 0.9994533 0.9999353 0.9999950 0.9999998 1.0000000
#> [15] 1.0000000 1.0000000 1.0000000 1.0000000 1.0000000 1.0000000 1.0000000Estimation
To fit a model to data in the MetaROC package, one requires either an output from a simulate function or a preprocessed dataset, similar to the real event data example provided earlier. By providing the ‘data’ entry from the simulation output to the fitting function, the model can be fitted. Different functions are available for fitting various models, although their outlines are generally similar. Some models may require additional inputs.
For instance, when fitting a Generalized Linear Mixed Model (GLMM), one must specify a Model identifier, which in this case is the link, to be used for fitting. Additionally, users have the option to specify the number of quadrature points used in optimization for fine-tuning the fitting process.
The other models included in the package can be fitted by replacing ‘GLMM’ with the desired model name. For example, ‘copula’ is to be used when a copula model is to be fitted, and ‘LMM’ respectively when a linear mixed model (LMM) is to be fitted.
As the previous data was simulated from a ‘cloglog’ GLMM, we are going to fit the same model to the data. Generally, fitting the type of model to the data it was generated from is almost always going to produce the best result. However, when comparing model performances on different data, other links or models can also be used. Especially when fitting to real-world data where the true data-generating process is unknown, it is essential to try multiple different model specifications to find the best fit.
fitted_model <- fit_metaROC_GLMM(data_sim = sim$data, distr_fit = "cloglog")Again using $ notation one can access the various information stored in the output from the fitting function.
fitted_model$mod
#> Generalized linear mixed model fit by maximum likelihood (Adaptive
#> Gauss-Hermite Quadrature, nAGQ = 0) [glmerMod]
#> Family: binomial ( cloglog )
#> Formula: event ~ -1 + threshold + threshold:group + (0 + group | study)
#> Data: subset(data_sim, subset = data_sim$step >= i)
#> Weights: freq
#> AIC BIC logLik deviance df.resid
#> 39503.14 39718.24 -19706.57 39413.14 835
#> Random effects:
#> Groups Name Std.Dev. Corr
#> study groupD 1.1892
#> groupH 0.8373 0.90
#> Number of obs: 880, groups: study, 41
#> Fixed Effects:
#> threshold5 threshold5.1 threshold5.2
#> -3.81305 -3.06302 -2.55779
#> threshold5.3 threshold5.4 threshold5.5
#> -2.22393 -2.17577 -1.93384
#> threshold5.6 threshold5.7 threshold5.8
#> -1.56670 -1.26995 -1.18046
#> threshold5.9 threshold6 threshold6.1
#> -0.78599 -0.56533 -0.31933
#> threshold6.2 threshold6.3 threshold6.4
#> -0.24744 0.02232 0.22469
#> threshold6.5 threshold6.6 threshold6.7
#> 0.24309 0.43200 0.75717
#> threshold6.8 threshold6.9 threshold7
#> 0.70553 0.82637 0.92116
#> threshold5:groupH threshold5.1:groupH threshold5.2:groupH
#> 2.10249 1.70129 1.51229
#> threshold5.3:groupH threshold5.4:groupH threshold5.5:groupH
#> 1.32969 1.59280 1.59323
#> threshold5.6:groupH threshold5.7:groupH threshold5.8:groupH
#> 1.55373 1.51883 1.68086
#> threshold5.9:groupH threshold6:groupH threshold6.1:groupH
#> 1.46891 1.40665 1.34735
#> threshold6.2:groupH threshold6.3:groupH threshold6.4:groupH
#> 1.49076 1.30860 1.20863
#> threshold6.5:groupH threshold6.6:groupH threshold6.7:groupH
#> 1.47722 1.36203 1.14134
#> threshold6.8:groupH threshold6.9:groupH threshold7:groupH
#> 1.23883 1.11596 1.17626Initially, the ‘mod’ entry presents fundamental details about the model, including its type, the outline of the fitted formula, and the dataset employed for fitting. Subsequently, model selection indicators such as AIC, BIC, and log likelihood are displayed. Moreover, estimated random effects and fixed effects for the thresholds and threshold,group factors are provided.
fitted_model$convergence_status
#> [1] 1The ‘convergence_status’ entry indicates whether the fitting process has converged successfully. In this example, the fitting function has converged, as its convergence status is 1.
fitted_model$par
#> $est
#> [1] -3.81305151 -3.06301625 -2.55778692 -2.22392743 -2.17576893 -1.93383965
#> [7] -1.56669760 -1.26995377 -1.18045828 -0.78599088 -0.56532501 -0.31933425
#> [13] -0.24743576 0.02232026 0.22469133 0.24308886 0.43200379 0.75716524
#> [19] 0.70553102 0.82636876 0.92116265 2.10249203 1.70128944 1.51228775
#> [25] 1.32968867 1.59280493 1.59322654 1.55372669 1.51882748 1.68086223
#> [31] 1.46891205 1.40665218 1.34734926 1.49076189 1.30860218 1.20862986
#> [37] 1.47722219 1.36202804 1.14133602 1.23882711 1.11595719 1.17626012
#>
#> $se
#> [1] 0.4723479 0.2679020 0.2852294 0.2708475 0.2641458 0.2782215 0.2485110
#> [8] 0.2520155 0.2522712 0.2472923 0.2468926 0.2490219 0.2456184 0.2500540
#> [15] 0.2505291 0.2488800 0.2562266 0.2508004 0.2635922 0.2493684 0.2494072
#> [22] 0.4487050 0.1780359 0.2071039 0.1877494 0.1729040 0.1973050 0.1424966
#> [29] 0.1501529 0.1556337 0.1425527 0.1432282 0.1485076 0.1392369 0.1544903
#> [36] 0.1586449 0.1837552 0.1661089 0.1605498 0.1846878 0.1629224 0.1698120The ‘par’ entry houses parameter specifics, comprising estimates and standard errors to calculate the estimated sensitivity and specificity for each unique threshold as outlined in the section about the GLMM specification.
fitted_model$step
#> [1] 1The ‘step’ entry tracks the algorithm’s progression in identifying optimal model parameters.
fitted_model$thresholds
#> [1] 5.0 5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9 6.0 6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8
#> [20] 6.9 7.0The ‘thresholds’ entry lists unique thresholds for easier access if the model needs to be evaluated or the results plotted.
fitted_model$fixed_cop_values
#> [1] NALastly, the ‘fixed_cop_values’ entry provides fixed copula values, which are relevant only for copula models fitted using ‘fit_metaROC_copula()’. Therefore, in this example, they are ‘NA’.
Evaluation
The ‘fit_MetaRoC_GLMM()’ output provides model selection criteria. However, assessing whether a model is good requires evaluating its bias, coverage, and convergence. To facilitate this evaluation, the ‘evaluate_metaROC()’ function is available. This function is designed to evaluate a fitted model using simulated data, where true parameters are known. Unless the true parameters are known, this function cannot be used on real-world data.
To use ‘evaluate_metaROC()’, input the fitted model, (simulated) data, and all thresholds returned by the fitting function. Additionally, specify the type of fitted model. For models utilizing a copula, additional parameters are required.
evaluated_model <- evaluate_metaROC(
fitted_model = fitted_model,
data_sim = sim,
all_thresholds = fitted_model$thresholds
)‘evaluated_model’ now contains the output provided by the evaluation function.
evaluated_model$convergence
#> [1] 1Checking the convergence status, one can see the evaluation was successful as the convergence status is 1. This means all necessary information can be extracted:
evaluated_model$estimated_pars
#> $est
#> [1] -3.81305151 -3.06301625 -2.55778692 -2.22392743 -2.17576893 -1.93383965
#> [7] -1.56669760 -1.26995377 -1.18045828 -0.78599088 -0.56532501 -0.31933425
#> [13] -0.24743576 0.02232026 0.22469133 0.24308886 0.43200379 0.75716524
#> [19] 0.70553102 0.82636876 0.92116265 2.10249203 1.70128944 1.51228775
#> [25] 1.32968867 1.59280493 1.59322654 1.55372669 1.51882748 1.68086223
#> [31] 1.46891205 1.40665218 1.34734926 1.49076189 1.30860218 1.20862986
#> [37] 1.47722219 1.36202804 1.14133602 1.23882711 1.11595719 1.17626012
#>
#> $se
#> [1] 0.4723479 0.2679020 0.2852294 0.2708475 0.2641458 0.2782215 0.2485110
#> [8] 0.2520155 0.2522712 0.2472923 0.2468926 0.2490219 0.2456184 0.2500540
#> [15] 0.2505291 0.2488800 0.2562266 0.2508004 0.2635922 0.2493684 0.2494072
#> [22] 0.4487050 0.1780359 0.2071039 0.1877494 0.1729040 0.1973050 0.1424966
#> [29] 0.1501529 0.1556337 0.1425527 0.1432282 0.1485076 0.1392369 0.1544903
#> [36] 0.1586449 0.1837552 0.1661089 0.1605498 0.1846878 0.1629224 0.1698120
#>
#> $fixed_cop_values
#> [1] NASimilar to the fitting function, the evaluation function yields the estimated parameters along with their standard errors, alongside fixed copula values if applicable. As we used the same data and model as before, these parameters are identical with the ones from the ‘fitted_model’ object before.
evaluated_model$bias_sens
#> [1] 3.817267e-03 -5.597747e-03 -2.798794e-02 -5.527799e-02 -6.170813e-02
#> [6] -6.275461e-02 -7.054058e-02 -7.687407e-02 -6.150568e-02 -5.732457e-02
#> [11] -4.616680e-02 -3.325443e-02 -1.793365e-02 -9.002649e-03 -3.817129e-03
#> [16] -1.197647e-03 -3.190439e-04 -7.569205e-05 -1.317260e-05 -2.044557e-06
#> [21] -2.257752e-07Moreover, the bias_sens vector encapsulates sensitivity biases across various cutoffs. For example, values like 4.658267e-03 and -9.869747e-03 quantify the deviation between estimated and true sensitivity at different thresholds. Similarly, the bias_spec vector denotes specificity biases.
evaluated_model$bias_spec
#> [1] 1.606265e-02 3.514289e-02 4.927901e-02 3.452048e-02 2.467188e-02
#> [6] 1.375464e-02 8.752301e-03 4.235731e-03 1.464688e-03 3.136786e-04
#> [11] 4.170621e-05 3.972789e-06 2.369034e-07 6.714748e-09 1.095519e-10
#> [16] 1.006084e-12 5.107026e-15 0.000000e+00 0.000000e+00 0.000000e+00
#> [21] 0.000000e+00Similarly, the bias_spec vector denotes specificity biases. These values, such as 3.234865e-02 and 6.725289e-02, articulate the disparity between estimated and true specificity across cutoffs.
evaluated_model$bias_auc
#> [1] -0.01621315Additionally, the bias of the Area Under the Curve (AUC), a metric gauging the model’s discriminatory prowess, is provided.
evaluated_model$coverage_spec
#> [1] 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1
evaluated_model$coverage_sens
#> [1] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1For both sensitivity and specificity, coverage for each threshold is indicated, determining whether the confidence intervals encompass the true values.
head(evaluated_model$roc_final)
#> sensitivity 2,5%-CI (sens) 97,5%-CI (sens) specificity 2,5%-CI (spec)
#> 5 0.978161 0.945796 0.991289 0.165368 0.172493
#> 5.1 0.933488 0.873928 0.964255 0.354017 0.390370
#> 5.2 0.863896 0.763179 0.922473 0.545473 0.595290
#> 5.3 0.775314 0.634946 0.865610 0.698027 0.749885
#> 5.4 0.692112 0.524807 0.808993 0.827207 0.871701
#> 5.5 0.598937 0.408949 0.743951 0.915160 0.944255
#> 97,5%-CI (spec) youden-index
#> 5 0.158508 0.143530
#> 5.1 0.321194 0.287505
#> 5.2 0.497937 0.409369
#> 5.3 0.645302 0.473341
#> 5.4 0.777608 0.519319
#> 5.5 0.878802 0.514097Most importantly, ‘roc_final’ contains the final ROC curve. For each threshold, it provides sensitivity and specificity pairs along with their 5% confidence intervals, as well as their corresponding Youden-Index. The Youden-Index offers a balanced measure of a test’s effectiveness in correctly identifying both positive and negative cases, with higher values indicating better discriminatory ability.
evaluated_model$cutoffs_in_model
#> [1] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE
#> [16] TRUE TRUE TRUE TRUE TRUE TRUEFinally, ‘cutoffs_in_model’ Indicates which thresholds are included in the model (TRUE for included, FALSE for not included). In the example all cutoffs are included.
Wrapper
The package also offers a wrapper function that consolidates all preceding steps into one. It starts by simulating data from a given model, then fitting a specified model to the simulated data, and finally evaluating it, all within a single function. While initially, the multitude of inputs might seem daunting and error tracing could be challenging, for those already familiar with the package, this function comes highly recommended. It offers an efficient means of swiftly accessing the final results.
Slightly adjusting the number of studies included yields the exact same model used in the step by step approach:
sim_wrap_model <- simulation_wrapper_metaROC(n_studies_min = 10,
n_studies_max = 50)The Wrapper output is identical as the one from the evaluation function. Notably, the specific values are not exactly identical because the functions are slightly different, but they are very close since the same model was used.
Pipeline
The ‘sim_pipeline_metaROC()’ function is designed to perform comprehensive simulations for meta-ROC analysis. It simulates data based on specified parameters, fits ROC models, and aggregates results to evaluate the performance and characteristics of different meta-analytic models for ROC curves. In contrast to the wrapper function it computes a pre defined number of simulation runs and outputs the aggregated results.
This function can be used to determine how well different meta-ROC models perform under various conditions. Additionally, the function helps in understanding the robustness of ROC models to changes in parameters such as the number of studies, study sizes, and true prevalence rates.
sim_pipeline <- sim_pipeline_metaROC(n_sims = 10,
n_studies_min = 5,
n_studies_max = 15,
model_kind_sim = "cloglog",
model_kind_fit = "cloglog",
true_preval_min = 0.4,
thresholds_lower_bound = 5,
thresholds_upper_bound = 7)
#> ##### Simulated model: cloglog, fitted model: cloglog #####
#> ##### True AUC: 0.85 #####
#> ##### Starting simulation run with 10 iterations #####
#> boundary (singular) fit: see help('isSingular')
#> Warning in fit_metaROC_GLMM(data_sim = data_sim, distr_fit = distr_fit, : NAs
#> introduced by coercion
#> Error: model did not compute properly.
#> Error: model did not compute properly.
#> Error: model did not compute properly.
#> Finished simulation run!As indicated by the error warnings, some models failed to converge properly. This is not unusual, especially in complex simulation studies. Checking the convergence rate helps determine whether the issue is systemic or within a normal range:
sim_pipeline$convergence_rate
#> [1] 0.7With a convergence rate of 0.9, indicating that 9 out of 10 simulation runs were successful, it is reasonable to assume that there is not a systemic issue with the model specification. The one failed simulation can likely be attributed to random chance or other factors.
Therefore, the other information can be extracted:
head(sim_pipeline$sim_info)
#> $n_sims
#> [1] 10
#>
#> $model_kind_sim
#> [1] "GLMM"
#>
#> $model_kind_fit
#> [1] "GLMM"
#>
#> $model_id_sim
#> [1] "cg"
#>
#> $model_id_fit
#> [1] "cg"
#>
#> $thresholds_lower_bound
#> [1] 5The ‘sim_info’ entry of the output provides a breakdown of the specific model defined when calling the function.
head(sim_pipeline$estimated_pars[, 1:7])
#> iteration true_auc model_sim model_fit beta1_5 beta1_5.1 beta1_5.2
#> 1 1 0.85 cloglog cloglog -4.100171 -3.518925 -3.556802
#> 2 2 0.85 cloglog cloglog -4.049583 -3.490500 -2.823630
#> 3 3 0.85 cloglog cloglog -3.042227 NA -2.299719
#> 4 4 0.85 cloglog cloglog NA NA -2.155683
#> 5 5 0.85 cloglog cloglog -4.341528 -3.618798 -3.121351
#> 6 6 0.85 cloglog cloglog NA NA NAJust like before, ‘estimated_pars’ contains the estimated parameters, but this time for each of the simulation runs.
sim_pipeline$bias_auc
#> mean_bias_auc median_bias_auc
#> 1 -0.01301056 -0.01156547Moreover, ‘bias_auc’ yields both the median and mean of the bias of the AUC curves over the simulation runs.
head(sim_pipeline$bias)
#> mean_bias_sens median_bias_sens mean_bias_spec median_bias_spec
#> 5 -0.002642733 -0.001767233 0.012107154 0.024178154
#> 5.1 0.001250453 0.014129253 0.013749491 0.006678891
#> 5.2 -0.008362938 -0.001251938 -0.008500420 -0.015303992
#> 5.3 -0.026768129 -0.037154986 0.018675626 0.020505483
#> 5.4 -0.039666701 -0.049142129 0.008477878 0.023941878
#> 5.5 -0.056271177 -0.052926606 0.003454212 0.017798640Accessing the ‘bias’ column, one gets more details about the mean and median bias of the sensitivity and specificity, respectively, for each threshold value contained in the studies.
head(sim_pipeline$coverage)
#> mean_coverage_sens median_coverage_sens mean_coverage_spec
#> 5 1 1 0
#> 5.1 1 1 0
#> 5.2 1 1 0
#> 5.3 1 1 0
#> 5.4 1 1 0
#> 5.5 1 1 0
#> median_coverage_spec
#> 5 0
#> 5.1 0
#> 5.2 0
#> 5.3 0
#> 5.4 0
#> 5.5 0Finally, the ‘coverage’ column has the same outline as the bias column, just providing the coverage value instead.