Cancer immunotherapy based on antibodies targeting the immune checkpoint PD-1/PD-L1 pathway has seen unprecedented clinical responses and constitutes the new paradigm in cancer therapy. The antibody-based immunotherapies have several limitations such as high production cost of the antibodies or their long half-life. Small-molecule inhibitors of the PD-1/PD-L1 interaction have been highly anticipated as a promising alternative or complementary therapeutic to the monoclonal antibodies (mAbs). Currently, the field of developing anti-PD-1/PD-L1 small-molecule inhibitors is intensively explored. In this paper, we review anti-PD-1/PD-L1 small-molecule and peptide-based inhibitors and discuss recent structural and preclinical/clinical aspects of their development. Discovery of the therapeutics based on small-molecule inhibitors of the PD-1/PD-L1 interaction represents a promising but challenging perspective in cancer treatment.