Toward Memory-Efficient and Interpretable Factorization Machines via Data and Model Binarization

Abstract

Factorization Machines (FM) is a general predictor that can efficiently model feature interactions in linear time, and thus has been broadly used for regression, classification and ranking tasks. Subspace Encoding Factorization Machine (SEFM) is one of the recent approaches which is proposed to enhance FM’s effectiveness by explicit nonlinear feature mapping for both individual features and feature interactions through equal-width binning per input feature. SEFM, despite its effectiveness, has a major drawback of increasing the memory cost of FM by $b$ times where $b$ is the number of bins adopted for the binning. To reduce the memory cost of SEFM, we propose Binarized FM (BiFM) in which each model parameter takes only a binary value (i.e., 1 or −1) and thus can be efficiently stored using one bit. We derive an algorithm which can learn the proposed FM with binary constraints using Straight Through Estimator (STE) with Adaptive Gradient Descent (Adagrad). For performance evaluation, we compare our proposed methods with a number of baselines based on eight different classification datasets. Our experimental results demonstrated that BiFM can achieve higher accuracy than SEFM at much less memory cost. BiFM also inherits the interpretability property from SEFM, and together with adaptive data binning methods can result in a more compact and interpretable set of classification rules.

Keywords

• Binarization
• factorization machines
• interpretability
• memory-efficient design