Large language model-driven sentiment analysis for facilitating fibromyalgia diagnosis

Abstract

Background Fibromyalgia (FM) is a complex disorder with widespread pain and emotional distress, posing diagnostic challenges. FM patients show altered cognitive and emotional processing, with a preferential allocation of attention to pain-related information. This attentional bias towards pain cues can impair cognitive functions such as inhibitory control, affecting patients’ ability to manage and express emotions. Sentiment analysis using large language models (LLMs) can provide insights by detecting nuances in pain expression. This study investigated whether open-source LLM-driven sentiment analysis could aid FM diagnosis.Methods 40 patients with FM, according to the 2016 American College of Rheumatology Criteria and 40 non-FM chronic pain controls referred to rheumatology clinics, were enrolled. Transcribed responses to questions on pain and sleep were machine translated to English and analysed by the LLM Mistral-7B-Instruct-v0.2 using prompt engineering targeting FM-associated language nuances for pain expression (‘prompt-engineered’) or an approach without this targeting (‘ablated’). Accuracy, precision, recall, specificity and area under the receiver operating characteristic curve (AUROC) were calculated using rheumatologist diagnosis as ground truth.Results The prompt-engineered approach demonstrated accuracy of 0.87, precision of 0.92, recall of 0.84, specificity of 0.82 and AUROC of 0.86 for distinguishing FM. In comparison, the ablated approach had an accuracy of 0.76, precision of 0.75, recall of 0.77, specificity of 0.75 and AUROC of 0.76. The accuracy was superior to the ablated approach (McNemar’s test p<0.001).Conclusion This proof-of-concept study suggests LLM-driven sentiment analysis, especially with prompt engineering, may facilitate FM diagnosis by detecting subtle differences in pain expression. Further validation is warranted, particularly the inclusion of secondary FM patients.