Persimmon germplasm in the Western Caucasus represent one of the most northerly collections. In our study, 51 commercial cultivars of D. kaki, 3 accessions of D. virginiana and 57 D. lotus accessions from six geographically distant populations were investigated using 19 microsatellite and 10 inter simple sequence repeat (ISSR) markers. After STRUCTURE analysis, the single accessions of Diospyros were allocated to three genetic clusters. Genetic admixtures in the important genotypes of D. kaki were revealed, whereas D. lotus accessions showed no admixture with other genetic clusters. The correspondence of genetic data and phenotypical traits was estimated in the D. kaki collection. The most frost tolerant genotypes of the collection, such as “Mountain Rogers”, “Nikitskaya Bordovaya”, “Rossiyanka”, “MVG Omarova”, “Meader”, “Costata”, “BBG”, and “Jiro”, showed a high percentage of genetic admixtures and were grouped close to D. virginiana. Some of these genotypes are known to be interspecific hybrids with D. virginiana. A low level of genetic diversity between the distant D. lotus populations was revealed and it can be speculated that D. lotus was introduced to the Western Caucasus from a single germplasm source. These results are an important basis for the implementation of conservation measures, developing breeding strategies, and improving breeding efficiency.