Soil physical and chemical properties play a central role in plant growth, influencing the availability of air, nutrients, and water. The aim of this two-year study was to evaluate the effect of soil texture and chemical properties (pH, electrical conductivity, organic carbon, organic matter, total, and active lime) on saffron (Crocus sativus L.) growth, yield, and quality. Corms were planted in pots filled with seven different soil textures obtained mixing an increasing quantity (33% and 66%) of sand to a clay soil (S1) and to a clay loam soil (S2) compared to a full (100%) sandy soil as a control (S7). A randomized complete block design comprising of seven pots with different types of soil (S1, S2, S3, S4, S5, S6, and S7) replicated three times was used. The results showed that the highest flower number (320.3 n m−2), stigma yield (2.0 g m−2), daughter corm production (7.9 kg m−2), and horizontal diameter (3.1 cm) were derived from S3 and S4 soils. These were characterized by a loam and sandy-loam texture, not very calcareous, with a sub-alkaline and neutral pH, low electrical conductivity, a content of organic matter between 5.46 and 8.67 g kg−1, and a content of active lime between 21.25 and 26.25 g kg−1. According to International Organization for Standardization (ISO) references, although all spice samples belonged to the first qualitative category, S1, S3, and S2 soils recorded the highest value for coloring power (290.5, 289.1, and 287.6 A1%1cm 440 nm, respectively). The highest values of bittering (109.2 A1%1cm 257 nm) and aromatic (26.6 A1%1cm 330 nm) power were reached by S3 soil. Positive correlations were found both between color with clay and organic matter, and aroma with total calcium carbonate. In conclusion, the assessment of soil conditions is particularly important to obtain the best saffron performance in terms of stigma and daughter corms yield as well as spice qualitative traits.