Enhanced Special Needs Assessment: A Multimodal Approach for Autism Prediction

Abstract

Autism spectrum disorder (ASD) poses significant challenges in early detection, necessitating innovative approaches for accurate identification. In this study, we propose a novel method utilizing machine learning models trained on a diverse dataset comprising facial images and behavioral ADOS scores. Employing cutting-edge convolutional neural network (CNN) architectures such as MobileNetV2, ResNet50, and InceptionV3, alongside a bespoke CNN model tailored for ASD detection, we explore the efficacy of our approach. Additionally, we introduce a multimodal concatenation model that integrates image features with behavioral scores to enhance predictive performance. Our results showcase promising outcomes, with the multimodal concatenation model achieving a remarkable accuracy of 97.05%. Furthermore, our models demonstrate competitive precision, recall, F1 score, and area under the ROC curve (AUC), underscoring their potential to facilitate early ASD diagnosis. These findings signify the significance of leveraging multimodal data fusion techniques to augment ASD detection accuracy, thereby contributing to advancements in early intervention strategies.

Keywords

• Autism detection
• MobileNetV2
• RestNet50
• InceptionV3
• multimodal concatenation
• convolution neural network