Loss to follow-up (LTFU) from HIV pre-exposure prophylaxis (PrEP) care compromises the goal of HIV elimination. We investigated the proportion of LTFU and associated risk factors among men who have sex with men (MSM) enrolled in a PrEP demonstration project in Burkina Faso, Côte d’Ivoire, Mali, and Togo. CohMSM-PrEP, a prospective cohort study, was conducted between November 2017 and June 2021 in community-based clinics. MSM aged 18 years or older at substantial risk of HIV infection received a comprehensive prevention package, including PrEP and peer education. LTFU was defined as not returning to the clinic for six months. Associated risk factors were investigated using a time-varying Cox’s model. Of 647 participants followed up for a median time of 15 months, 372 were LTFU (57.5%). LTFU was associated with younger age (adjusted hazard ratio [95% Confidence Interval]; 1.50 [1.17–1.94]), unemployment (1.33 [1.03–1.71]), depression (1.63 [1.12–2.38]), and perceiving no HIV risk with stable male partners (1.61 [1.23–2.10]). Contacting peer educators outside of scheduled visits was protective (0.74 [0.56–0.97]). Our findings show that LTFU from PrEP care in West African MSM is a major challenge to achieving HIV elimination, but that the involvement of peer educators in PrEP delivery helps to limit LTFU by providing users with adequate support.