Population growth and the high demand for food production has caused environmental degradation losses in biodiversity. The conservation of genetic resources is the most appropriate strategy for maintaining the variability of species of great importance, such as Capsicum. In this study, we describe the characterization of 69 pepper accessions of four Capsicum species from different regions of Brazil on the basis of qualitative and quantitative descriptors and ISSR markers. The 11 quantitative descriptors grouped the 69 populations into five clades. The fruit traits had the greater discrimination power. A strong correlation was detected between some pairs of quantitative descriptors: petiole and fruit length, fruit diameter and fresh fruit mass, leaf length and leaf width, canopy and plant height, stem diameter and plant height, stem diameter and canopy diameter. A negative correlation was also observed between plant height and pericarp thickness. Moreover, the molecular analysis grouped accessions into five clades. The morpho-agronomic and molecular characterization were efficient to estimate the genetic diversity between accessions, being important tools for the knowledge and use of accessions. This study is of great importance to preserve pepper germplasm and to provide data to facilitate the process of the selection in breeding programs.