Probabilistic Confusion Matrix: A Novel Method for Machine Learning Algorithm Generalized Performance Analysis

Abstract

The paper addresses the issue of classification machine learning algorithm performance based on a novel probabilistic confusion matrix concept. The paper develops a theoretical framework which associates the proposed confusion matrix and the resulting performance metrics with the regular confusion matrix. The theoretical results are verified based on a wide variety of real-world classification problems and state-of-the-art machine learning algorithms. Based on the properties of the probabilistic confusion matrix, the paper then highlights the benefits of using the proposed concept both during the training phase and the application phase of a classification machine learning algorithm.

Keywords

• confusion matrix
• classification problem
• machine learning
• cross-validation
• class probabilities
• generalized algorithm performance