DISTRIBUTED CONVOLUTIONAL NEURAL NETWORK MODEL ON RESOURCE-CONSTRAINED CLUSTER

Abstract

Subject of Research. The paper presents the distributed deep learning particularly convolutional neural network problem for resource-constrained devices. General architecture of convolutional neural network and its specificity is considered, existing constraints that appear while the deployment process on such architectures as LeNet, AlexNet, VGG-16/VGG-19 are analyzed. Deployment of convolutional neural network for resource-constrained devices is still a challenging task, as there are no existing and widely-used solutions. Method. The method for distribution of feature maps into smaller pieces is proposed, where each part is a determined problem. General distribution model for overlapped tasks within the scheduler is presented. Main Results. Distributed convolutional neural network model for a resource-constrained cluster and a scheduler for overlapped tasks is developed while carrying out computations mostly on a convolutional layer since this layer is one of the most resource-intensive, containing a large number of hyperparameters. Practical Relevance. Development of distributed convolutional neural network based on proposed methods provides the deployment of the convolutional neural network on a cluster that consists of 24 RockPro64 single board computers performing tasks related to machine vision, natural language processing, and prediction and is applicable in edge computing.

Keywords

• convolutional neural network
• cluster
• scheduler
• distributed convolutional neural network
• deep learning