We conducted a prospective cohort study at a community facility designated for the isolation of individuals with asymptomatic or mild COVID-19 between 10 January and 22 February 2021 to investigate the relationship of viral shedding with symptom changes of COVID-19. In total, 89 COVID-19 adult patients (12 asymptomatic, 16 presymptomatic, 61 symptomatic) were enrolled. Symptom scores, the genomic RNA and subgenomic RNA of SARS-CoV-2 from saliva samples with a cell culture were measured. Asymptomatic COVID-19 patients had a similar viral load to symptomatic patients during the early course of the disease, but exhibited a rapid decrease in viral load with the loss of infectivity. Subgenomic RNA and viable virus by cell culture in asymptomatic patients were detected only until 3 days after diagnosis, and the positivity of the subgenomic RNA and cell culture in symptomatic patients gradually decreased in both from 40% in the early disease course to 13% at 10 days and 4% at 8 days after the symptom onset, respectively. In conclusion, symptomatic patients have a high infectivity with high symptom scores during the early disease course and gradually lose infectivity depending on the symptom. Conversely, asymptomatic patients exhibit a rapid decrease in viral load with the loss of infectivity, despite a similar viral load during the early disease course.