Fast and Scalable Design Space Exploration for Deep Learning on Embedded Systems

Abstract

Deep learning algorithms are used in various advanced applications, including computer vision, large language models and many others due to their increasing success over traditional approaches. Enabling these algorithms on various embedded systems is essential to extend the success of deep learning methods in the cutting-edge real-world systems and applications. The configurability of the embedded systems and software applications make them adaptable to different performance requirements such as latency, power, memory and accuracy. However, the vast number of combinations of hardware-software configurations makes the multi-objective optimization with respect to multiple performance metrics highly complex. Additionally, the lack of analytical form of the problem makes it harder to identify Pareto optimal configurations. To address these challenges, here we propose a fast, accurate and scalable search algorithm to efficiently solve these search spaces. We evaluate our algorithm, together with state-of-the-art methods, with different deep learning applications running on different hardware configurations, creating different search spaces, and show that our algorithm outperforms the other existing approaches by finding the Pareto frontier more accurately and with up to 15 times faster speed.

Keywords

• Deep learning
• embedded systems
• large language models
• Bayesian optimization
• search algorithms