Improvement of the k-nn Entropy Estimator with Applications in Systems Biology

Abstract

In this paper, we investigate efficient estimation of differential entropy for multivariate random variables. We propose bias correction for the nearest neighbor estimator, which yields more accurate results in higher dimensions. In order to demonstrate the accuracy of the improvement, we calculated the corrected estimator for several families of random variables. For multivariate distributions, we considered the case of independent marginals and the dependence structure between the marginal distributions described by Gaussian copula. The presented solution may be particularly useful for high dimensional data, like those analyzed in the systems biology field. To illustrate such an application, we exploit differential entropy to define the robustness of biochemical kinetic models.

Keywords

• differential entropy
•  k-NN estimator
• bias correction
• Gaussian copula
• mutual information
• sensitivity indices
• p53-Mdm2 feedback loop model