IEEE Access (Jan 2023)

Optimizations of Privacy-Preserving DNN for Low-Latency Inference on Encrypted Data

  • Hyunhoon Lee,
  • Youngjoo Lee

DOI
https://doi.org/10.1109/ACCESS.2023.3318433
Journal volume & issue
Vol. 11
pp. 104775 – 104788

Abstract

Read online

Homomorphic encryption (HE) based on the CKKS scheme is a promising candidate for implementing privacy-preserving deep neural networks (PP-DNN) by performing operations directly on the encrypted data. However, due to the computational complexity of HE operation, even simple PP-DNNs require a huge amount of processing time. In order to reduce the processing time of PP-DNN, in this paper, we present an innovative, low-latency model optimization solution for PP-DNNs. Our proposed low-latency model optimization solution exploits second-order polynomials that approximate original activation functions, ensuring low-latency and accurate DNN performance. To further reduce the processing latency of PP-DNNs, we introduce the coefficient absorbing technique and a masking convolution for convolutional layers. The experimental results show that the proposed solution constructs bootstrapping-free PP-DNN and reduces the inference latency of CKKS-based ResNet-34 by 35% in the CIFAR-100 dataset and ResNet-32 by 77% in the CIFAR-10 dataset compared to previous approaches while maintaining the same level of inference accuracy. Moreover, through the layer-wise latency analysis, we show the efficacy of our approaches, and through validation in various scenarios, we demonstrate the generality of our methods.

Keywords