Universal Sample Size Invariant Measures for Uncertainty Quantification in Density Estimation

Abstract

Previously, we developed a high throughput non-parametric maximum entropy method (PLOS ONE, 13(5): e0196937, 2018) that employs a log-likelihood scoring function to characterize uncertainty in trial probability density estimates through a scaled quantile residual (SQR). The SQR for the true probability density has universal sample size invariant properties equivalent to sampled uniform random data (SURD). Alternative scoring functions are considered that include the Anderson-Darling test. Scoring function effectiveness is evaluated using receiver operator characteristics to quantify efficacy in discriminating SURD from decoy-SURD, and by comparing overall performance characteristics during density estimation across a diverse test set of known probability distributions.

Keywords

• density estimation
• distribution free
• non-parametric statistical test
• decoy distributions
• size invariance
• scaled quantile residual
• maximum entropy method
• scoring function
• outlier detection
• overfitting detection