Previous studies have screened key candidate genes for litter size in sheep, including fibrillin-1 (FBN1), family with sequence similarity 184 member B (FAM184B) and zinc finger and AT-hook domain containing (ZFAT). Therefore, it is necessary to verify these genes in the Xinggao mutton sheep population and determine the associated loci for litter size. In this study, three loci (FBN1 g.160338382 T > C, FAM184B g.398531673 C > T and ZFAT g.20150315 C > T) were firstly screened based on the population differentiation coefficient between the polytocous and monotocous sheep groups. Then, population genetic analysis and association analysis were performed on these loci. The results revealed that the g.160338382 T > C in FBN1 was significantly associated with the litter size of sheep. Moreover, there was no significant interaction effect between the g.160338382 T > C locus and FecB on litter size. Notably, g.160338382 T > C is adjacent to the anterior border of exon 58 and belongs to a splice polypyrimidine tract variant, which may lead to alternative splicing and ultimately cause changes in the structure and function of the protein. In summary, our results provided a potentially effective genetic marker for improving the litter size of sheep.