Mongolian cattle from China have strong adaptability and disease resistance. We aimed to compare the gut microbiota community structure and diversity in grazing Mongolian cattle from different regions in Inner Mongolia and to elucidate the influence of geographical factors on the intestinal microbial community structure. We used high throughput 16S rRNA sequencing to analyze the fecal microbial community and diversity in samples from 60 grazing Mongolian cattle from Hulunbuir Grassland, Xilingol Grassland, and Alxa Desert. A total of 2,720,545 high-quality reads and sequences that were 1,117,505,301 bp long were obtained. Alpha diversity among the three groups showed that the gut microbial diversity in Mongolian cattle in the grasslands was significantly higher than that in the desert. The dominant phyla were Firmicutes and Bacteroidetes, whereas Verrucomicrobia presented the highest abundance in the gut of cattle in the Alxa Desert. The gut bacterial communities in cattle from the grasslands versus the Alxa Desert were distinctive, and those from the grasslands were closely clustered. Community composition analysis revealed significant differences in species diversity and richness. Overall, the composition of the gut microbiota in Mongolian cattle is affected by geographical factors. Gut microbiota may play important roles in the geographical adaptations of Mongolian cattle.