Abstract Background Ischemia can induce rapid activation of microglia in the brain. As key immunocompetent cells, reactive microglia play an important role in pathological development of ischemic stroke. However, the role of activated microglia during the development of ischemia remains controversial. Thus, we aimed to investigate the function of reactive microglia in the early stage of ischemic stroke. Methods A Rose Bengal photothrombosis model was applied to induce targeted ischemic stroke in mice. CX3CR1CreER:R26iDTR mice were used to specifically deplete resident microglia through intragastric administration of tamoxifen (Ta) and intraperitoneal injection of diphtheria toxin (DT). At day 3 after ischemic stroke, behavioral tests were performed. After that, mouse brains were collected for further histological analysis and detection of mRNA expression of inflammatory factors. Results The results showed that specific depletion of microglia resulted in a significant decrease in ischemic infarct volume and improved performance in motor ability 3 days after stroke. Microglial depletion caused a remarkable reduction in the densities of degenerating neurons and inducible nitric oxide synthase positive (iNOS+) cells. Importantly, depleting microglia induced a significant increase in the mRNA expression level of anti-inflammatory factors TGF-β1, Arg1, IL-10, IL-4, and Ym1 as well as a significant decline of pro-inflammatory factors TNF-α, iNOS, and IL-1β 3 days after stroke. Conclusions These results suggest that activated microglia is an important modulator of the brain’s inflammatory response in stroke, contributing to neurological deficit and infarct expansion. Modulation of the inflammatory response through the elimination of microglia at a precise time point may be a promising therapeutic approach for the treatment of cerebral ischemia.