Instruction Set Extension of a RiscV Based SoC for Driver Drowsiness Detection

Abstract

This paper describes the design and implementation of a driver drowsiness detection (DDD) system using a modified RiscV processor on a field-programmable gate array (FPGA). To detect drowsiness, Convolutional Neural Network (CNN) is implemented on a RiscV processor. The CNN is trained to classify four primary driver’s expressions, including distraction, natural, sleep, and yawn. The trained CNN accuracy is 81.07% on validation data. Furthermore, due to FPGA memory limitations, written C code for the trained CNN is optimized in numerous ways. Optimizations include the usage of dynamic fixed-point data types and dynamic memory allocations. On the other hand, the processor is modified by adding three custom instructions, including custom store, conv2d( $2\times 2$ ), and multiply and accumulation (MAC) to enhance the computation rate. As a result, the processor with custom store, conv2d( $2\times 2$ ), and MAC as custom instructions achieved the best result in terms of latency, with an improvement factor of 1.7 over the base processor and 1.25 over the processor with only custom store and multiply and accumulation (MAC) in exchange of slight increase in area.

Keywords

• Convolutional neural network
• driver drowsiness detection
• FPGA
• hardware implementation
• modified RiscV processor