Classifying Musical Instrument Using Spatiotemporal Features ​with Deep Neural Networks

Abstract

Similar to trained human ears, a machine could be trained to recognize musical instruments presence in audio tracks for the purpose of musical information retrieval. The audio signal produced by an instrument has unique temporal and spectral features, which could be extracted for machine learning purpose. This research investigates the use of spatiotemporal features, which were converted from temporal and spectral features of monophonic music for this application. The performance of a recurrent neural network model called bidirectional Long Short Term Memory (Bi-LSTM) network and two convolutional neural networks (CNNs) in musical instruments classification with log Mel-spectrogram were analysed and evaluated. With a dataset of 14 musical instruments, the Bi-LSTM and 1-dimensional CNN models obtained a macro F1 score of 0.976 and 0.977 respectively, while 2-dimensional CNN model achieved 0.985.

Keywords

• long short-term memory network (lstm)
• convolutional neural network (cnn)
• musical instruments classification
• mel-spectrogram
• spectral features
• spatiotemporal features