Since December 2019, the coronavirus disease 2019 (COVID-19), caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has spread throughout the world. To eradicate it, it is crucial to acquire a strong and long-lasting anti-SARS-CoV-2 immunity, by either natural infection or vaccination. We collected blood samples 12–305 days after positive polymerase chain reactions (PCRs) from 35 recovered individuals infected by SARS-CoV-2. Peripheral blood mononuclear cells were stimulated with SARS-CoV-2-derived peptide pools, such as the spike (S), nucleocapsid (N) and membrane (M) proteins, and we quantified anti-S immunoglobulins in plasma. After 10 months post-infection, we observed a sustained SARS-CoV-2-specific CD4+ T-cell response directed against M-protein, but responses against S- or N-proteins were lost over time. Besides, we demonstrated that O-group individuals presented significantly lower frequencies of specific CD4+ T-cell responses against Pep-M than non O-group individuals. The non O-group subjects also needed longer to clear the virus, and they lost cellular immune responses over time, compared to the O-group individuals, who showed a persistent specific immune response against SARS-CoV-2. Therefore, the S-specific immune response was lost over time, and individual factors might determine the sustainability of the body’s defenses, which must be considered in the future design of vaccines to achieve continuous anti-SARS-CoV-2 immunity.