Toward Bootstrapping-Free Homomorphic Encryption-Based GRU Network for Text Classification

Abstract

Homomorphic encryption (HE) is a promising method in privacy-preserving cloud computing. Applying HE on feedforwad neural networks has been frequently reported recently but the research on recurrent neural networks is still insufficient. In previous studies, HE-based GRU is built with bootstrapping due to the changeable input length and large number of required multiplications, which is not time-efficient. In this study, we give a guideline of building bootstrapping-free HE-based GRU for text classification tasks. We discuss the methods of pre-processing of texts to decrease the input sequence length but keep the accuracy in a comparable level as the original GRU. The architecture of GRU is designed with flexibility to process the input sequence with different lengths while fixing the number of recurrent steps. At last, the HE parameter selection is discussed. We analyze the noise raised from HE operations and select the parameters that ensure the results from encrypted data are the same as that on plaintexts. The proposed model is evaluated on 6 popular text datasets, and the results show that the accuracy is only lower than the original GRU by at most 4.2%. Despite the complicated calculations in GRU, the proposed model is light-weighted and the fastest inference among our implementation costs only 10 minutes. We show the potential of applying HE schemes on complex models without bootstrapping to achieve fast encrypted computations.

Keywords

• CKKS
• gated recurrent unit
• homomorphic encryption
• privacy-preserving technique