TensorRT Powered Model for Ultra-Fast Li-Ion Battery Capacity Prediction on Embedded Devices

Abstract

The LSTM neural network is often employed for time-series data prediction due to its strong nonlinear mapping capability and memory effect, allowing for better identification of complex data characteristics. However, the large computational workload required by neural networks can result in longer prediction times, making deployment on time-sensitive embedded devices challenging. To address this, TensorRT, a software development kit for NVIDIA hardware platforms, offers optimized network structures and reduced inference times for deep learning inference applications. Though TensorRT inference is GPU-based like other deep learning frameworks, TensorRT outperforms comparable frameworks in terms of inference speed. In this paper, we compare the inference time consumption and prediction deviation of various approaches on CPU, GPU, and TensorRT, while also exploring the effects of different quantization approaches. Our experiments demonstrate the accuracy and inference latency of the same model on the FPGA development board PYNQ-Z1 as well, though the best results were obtained using NVIDIA Jetson Xavier NX. The results show an approximately 50× improvement in inference speed compared to our previous technique, with only a 0.2% increase in Mean Absolute Percentage Error (MAPE). These works highlight the effectiveness and efficiency of TensorRT in reducing inference times, making it an excellent choice for time-sensitive embedded device deployments that require high precision and low latency.

Keywords

• optimization
• TensorRT
• deep learning
• long short-term memory
• state of health