TAS: A Temperature-Aware Scheduling for Heterogeneous Computing

Abstract

With the development of AI technology, the parameters and calculation overhead of advanced models have increased exponentially, resulting in the existing low-end GPU(Graphic Processing Unit) being unable to meet the computing power required for model operation. In order to speed up the inference speed in edge scenarios, various manufacturers have launched NPU(Neural Processor Unit), a special chip for neural networks, which can improve the overall inference efficiency and reduce energy consumption through a certain loss of precision. However, in the current common edge-side solutions, the problem of CPU+GPU+NPU co-processing is not well considered. At the same time, edge-side devices are more easily affected by the ambient temperature. In this paper, CPU+GPU+NPU is used to jointly process edge-side inference tasks, and we first established a heterogeneous device temperature perception model based on the ambient temperature of the edge device, then proposed a TAS(temperature-aware schedule) algorithm to control the running speed of the heterogeneous device, and then proposed a task scheduling algorithm for the heterogeneous device, namely TASTS(TAS-based task schedule). At the same time, we also use a hungarian matching algorithm to optimize the final result. This paper finally verified several models in real edge environment, found that it can improve the performance by 20-50% compared with conventional methods under temperature constraints.

Keywords

• Heterogeneous computing
• schedule
• temperature-aware