Physics-inspired transfer learning for ML-prediction of CNT band gaps from limited data

Abstract

Abstract Recent years have seen a drastic increase in the scientific use of machine learning (ML) techniques, yet their applications remain limited for many fields. Here, we demonstrate techniques that allow overcoming two obstacles to the widespread adoption of ML, particularly relevant to nanomaterials and nanoscience fields. Using the prediction of the band gap values of carbon nanotubes as a typical example, we address the representation of the periodic data as well as training on extremely small datasets. We successfully showed that careful choice of the activation function allows capturing periodic tendencies in the datasets that are common in physical data and previously posed significant difficulty for neural networks. In particular, utilization of the recently proposed parametric periodic Snake activation function shows a dramatic improvement. Furthermore, tackling a typical lack of accurate data, we used the transfer learning technique utilizing more abundant low-quality computational data and achieving outstanding accuracy on a significantly expanded dataspace. This strategy was enabled by the use of a combination of the Snake and ReLU layers, capturing data periodicity and amplitude, respectively. Hence, retraining only ReLU layers allowed the transfer of the periodic tendencies captured from low-quality data to the final high-accuracy neural network. Those techniques are expected to expand the usability of ML approaches in application to physical data in general and the fields of nanomaterials in particular.