EDSUCh: A robust ensemble data summarization method for effective medical diagnosis

Abstract

Identifying rare patterns for medical diagnosis is a challenging task due to heterogeneity and the volume of data. Data summarization can create a concise version of the original data that can be used for effective diagnosis. In this paper, we propose an ensemble summarization method that combines clustering and sampling to create a summary of the original data to ensure the inclusion of rare patterns. To the best of our knowledge, there has been no such technique available to augment the performance of anomaly detection techniques and simultaneously increase the efficiency of medical diagnosis. The performance of popular anomaly detection algorithms increases significantly in terms of accuracy and computational complexity when the summaries are used. Therefore, the medical diagnosis becomes more effective, and our experimental results reflect that the combination of the proposed summarization scheme and all underlying algorithms used in this paper outperforms the most popular anomaly detection techniques.

Keywords

• Data summarization
• Ensemble
• Medical diagnosis
• Sampling