Drawing Interpretation Using Neural Networks and Accessibility Implementation in Mobile Application

Abstract

This paper continues the research of the previous work, regarding PandaSays mobile application, having its main purpose to detect the affective state of the child from his drawings, using MobileNet neural network. Children diagnosed with autism spectrum disorder, have difficulties in expressing their feelings and communicating with others. The purpose of PandaSays mobile application, is to help parents and tutors that have children diagnosed with autism, to communicate better with them and to understand their feelings. The main goal was to improve the model’s accuracy, trained with MobileNet neural network, which reached the value of 84.583%. For training the model, it was used Python programming language. The study focuses further on accessibility and its importance to children diagnosed with autism. Relevant screenshots of the mobile application are presented, in order to indicate that the application follows the accessibility guidelines and rules. Finally, there is presented the interaction with Marty robot and the efficiency of mobile application’s drawing prediction.

Keywords

• machine learning
• neural networks
• autism
• humanoid robots
• mobile application
• drawing interpretation