Measuring the accuracy of diagnostic tests is crucial in many application areas, including medicine, health care, and data mining. Good methods for determining diagnostic accuracy provide useful guidance on selection of patient treatment, and the ability to compare different diagnostic tests has a direct impact on quality of care. In this paper nonparametric predictive inference (NPI) for accuracy of diagnostic tests with binary test results is presented and discussed, together with methods for comparison of two such tests. NPI does not aim at inference for an entire population but instead explicitly considers future observations, which is particularly suitable for inference to support decisions on medical diagnosis for one future patient, or for a predetermined number of future patients, so the NPI approach provides an attractive alternative to standard methods.
Coolen-Maturi, T., Coolen-Schrijner, P., & Coolen, F. (2012). Nonparametric predictive inference for binary diagnostic tests. Journal of statistical theory and practice, 6(4), 665-680. https://doi.org/10.1080/15598608.2012.719800