A hybrid memory trace collection and analysis toolkit for big data applications

Abstract

The rise of in-memory computing framework represented by Spark,the gradual deepening of new non-volatile memory research and the increasingly severe data security situation made the existing memory behavior analysis tools unable to meet the demand for big data applications.A software-hardware hybrid memory trace collection and analysis toolkit for big data applications was proposed.Based on the basic memory trace collected by hardware,the memory behavior information with rich semantic information can be obtained quickly,accurately and undistorted by combining software information synchronization and offline annotation.It also provides an implementation method for real-time security monitoring of large data access.Finally,a group of real big data applications were analyzed by this toolkit.

Keywords

• memory trace;memory behavior;big data