The entropy of stochastic processes based on practical considerations

Abstract

The limitations of the Shannon entropy and the dynamic Shannon entropy are discussed. They only measure the aleatory uncertainty of stochastic processes. In this paper, based on the practical considerations, an entropy formula for measuring the uncertainty of stochastic processes is proposed, which measures not only the aleatory uncertainty but also the epistemic uncertainty. The uncertainty of the Gaussian process in four different situations is discussed. These works fill the gap between information theory and signal processing and thus can be used as a design guideline for constructing a secure source in the security field.