Doctor or AI? Efficient Neural Network for Response Classification in Health Consultations

Abstract

Patients seek quality healthcare because they trust their doctors and the healthcare system. However, the use of AI models in medical consultations has undermined this trust. AI systems typically depend on accurate and large volumes of data for training, but in cases of insufficient or incorrect data, this can lead to incomplete or flawed outputs. The inaccuracies in the response generated by AI systems may result in biased outcomes, compromising patient care and further eroding the trust patients place in the healthcare system. In this paper, we describe an innovative approach to distinguishing between responses generated by AI and those written by human doctors during health consultations, using an efficient neural network. As part of our feature extraction approach, we converted text into numerical representations via word-level tokenization, mapping to integer sequences. This allows the neural network to efficiently process text while preserving semantic structure and handling a large vocabulary with fixed sequence lengths. Through rigorous experimentation and evaluation, we showcase the effectiveness and reliability of our proposed neural network architecture, MedXNet, in accurately classifying diverse responses encountered in health consultations. For the classification approach, we combined BiLSTM, Transformer, and CNN layers to capture local and global dependencies in sequence inputs and a dense layer that was fully connected with dropout regularization and softmax activation. We compared MedXNet performance with different RNNs, including LSTM, Bi-LSTM, GRU, and 1D-CNN, across three datasets of increasing complexity. Dataset A represents simple data, dataset B introduces greater complexity, and dataset C poses the highest level of challenge. Our findings revealed that MedXNet outperforms the others with an accuracy of 98.74% on dataset A. Although the accuracy of MedXNet decreased on B, it remains the top performer. With 94.63% accuracy, MedXNet still achieves the highest accuracy in dataset C. Based on these findings, MedXNet demonstrated robustness across a wide range of data complexity levels, making it an ideal classification tool for doctor-written and AI-generated text in health consultations. This can enhance the trust patients have in the responses they receive during online medical consultations.

Keywords

• Artificial intelligence
• classification models
• deep learning
• machine learning
• MedXNet
• neural network