Efficient Human Activity Recognition Using Lookup Table-Based Neural Architecture Search for Mobile Devices

Abstract

Mobile devices play a crucial role in human activity recognition as they enable real-time sensing of user interaction for learning algorithms like neural networks. To facilitate human activity recognition on mobile devices, it is important to deploy efficient neural network architectures due to the limited computational capacity of these devices. However, conventional neural architecture search methods often generate less effective architectures because they neglect the specific requirements of target devices on which the neural network would operate in real-time. Moreover, these methods are impractical in the mobile device environment due to their high computational cost for architecture search. To address these challenges, we propose an efficient neural architecture search method based on a latency lookup table. Our proposed method efficiently performs the network search process based on differentiable NAS while considering the actual latency of mobile devices, which is stored in a lookup table. The experimental results on public datasets provide evidence that the proposed method outperforms conventional methods in terms of speed. We achieved a search time of under 1.5 hours on each dataset, which is more than seven times faster on average compared to conventional methods. Furthermore, our in-depth analysis shows that the optimal architecture can vary depending on the target mobile devices, such as Galaxy A31 and S10. By tailoring the models to each device, optimized models achieved an additional 4-5% improvement in inference time for each respective device.

Keywords

• Human activity recognition
• deep learning
• neural architecture search