Seeds are involved in the vertical transmission of microorganisms from one plant generation to another and consequently act as reservoirs for the plant microbiota. However, little is known about the structure of seed-associated microbial assemblages and the regulators of assemblage structure. In this work, we have assessed the response of seed-associated microbial assemblages of Raphanus sativus to invading phytopathogenic agents, the bacterial strain Xanthomonas campestris pv. campestris (Xcc) 8004 and the fungal strain Alternaria brassicicola Abra43. According to the indicators of bacterial (16S rRNA gene and gyrB sequences) and fungal (ITS1) diversity employed in this study, seed transmission of the bacterial strain Xcc 8004 did not change the overall composition of resident microbial assemblages. In contrast seed transmission of Abra43 strongly modified the richness and structure of fungal assemblages without affecting bacterial assemblages. The sensitivity of seed-associated fungal assemblage to Abra43 is mostly related to changes in relative abundance of closely related fungal species that belong to the Alternaria genus. Variation in stability of the seed microbiota in response to Xcc and Abra43 invasions could be explained by differences in seed transmission pathways employed by these micro-organisms, which ultimately results in divergence in spatio-temporal colonization of the seed habitat.