Abstract Alzheimer's disease (AD) is one of the most common neurodegenerative causes of dementia, the pathology of which is still not much clear. It′s challenging to discover the disease modifying agents for the prevention and treatment of AD over the years. Emerging evidence has been accumulated to reveal the crucial role of up‐regulated glutaminyl cyclase (QC) in the initiation of AD. In the current study, the QC inhibitory potency of a library consisting of 1621 FDA‐approved compounds was assessed. A total of 54 hits, 3.33 % of the pool, exhibited QC inhibitory activities. The Ki of the top 5 compounds with the highest QC inhibitory activities were measured. Among these selected hits, compounds affecting neuronal signaling pathways and other mechanisms were recognized. Moreover, several polyphenol derivatives with QC inhibitory activities were also identified. Frameworks and subsets contained in these hits were analyzed. Taken together, our results may contribute to the discovery and development of novel QC inhibitors as potential anti‐AD agents.