Abstract Background Recurrent chest pain episodes with no clear explanation may affect patients’ psychological wellbeing and health-related quality of life (HRQoL) negatively. Despite the fact that a significant amount of patients with non-cardiac chest pain (NCCP) might have a history of Cardiac Disease (CD), there is today a lack of knowledge on how CD influences the association between psychological wellbeing and HRQoL in patients with NCCP. Therefore, the aim of this study is to describe HRQoL in patients with NCCP, with or without history of CD, and to explore the association between HRQoL and cardiac anxiety, depressive symptoms, fear of body sensations and somatization. Methods Five hundred fifty-two patients discharged with NCCP from four hospitals in Southeast Sweden completed the EQ-5D, Cardiac Anxiety Questionnaire, Patient Health Questionnaire-9, Body Sensations Questionnaire, and Patient Health Questionnaire-15. Results Fifty precent reported at least moderate problems regarding pain/discomfort and 25% reported at least moderate problems in the HRQoL dimensions mobility, usual activities, and anxiety/depression. Patients with NCCP and history of CD reported significantly lower HRQoL (p ≤ 0.05) compared to patients with NCCP without CD. In the total study population, cardiac anxiety, depressive symptoms, and somatization had weak significant negative associations (beta = 0.187–0.284, p < 0.001) with HRQoL. In patients with history of CD, the association between depressive symptoms and HRQoL was moderate (beta = − 0.339, p < 0.001), compared to weak association in patients without CD (beta = − 0.193, p < 0.001). On the other hand, the association between cardiac anxiety and HRQoL was weak in both patients with history of CD (beta = − 0.156, p = 0.05), and in those without (beta = − 0.229, p < 0.001). Conclusions Patients with NCCP, in particular those with history of CD, reported low levels of HRQoL, which was associated with psychological distress. This should be considered when developing psychological interventions aiming to improve HRQoL in patients with NCCP.