Hyperlipidemia has been proposed as a risk factor of dementia and cognitive decline. However, the findings of the relationship between cholesterol level and cognitive/brain function have been inconsistent. Here, using a well-controlled sample from the Parkinson’s Progression Markers Initiative (PPMI), we investigated the probable non-linear relationship between plasma total cholesterol (TC) level, gray matter volume (GMv), and cognitive performance in 117 non-demented subjects (mean age, 61.5 ± 8.9 years), including 67 Parkinson’s disease (PD) patients and 50 demographically matched controls. A quadratic relationship between semantic fluency (SF) performance and TC levels was identified. Within the subjects with a desirable TC level (TC < 200 mg/dl), low TC (lTC) levels were associated with reduced SF performance, as well as reduced GMv in three medial temporal regions [including bilateral anterior hippocampus (HIP)]. In contrast, no significant relationship between TC and cognition performance/GMv was found in individuals with a high cholesterol level (i.e., TC ≥ 200 mg/dl). Further region of interest (ROI)-based analysis showed that individuals with TC levels ranging from 100 to 160 mg/dl had the lowest GMv in the medial temporal regions. These findings suggest that low-normal TC level may be associated with reduced cognitive function and brain atrophy in regions implicated in neurodegenerative diseases, adding to a growing body of literature supporting a probable non-linear relationship between cholesterol level and brain health. However, this finding needs to be verified with other large public cohort data that do not include PD patients.