Abstract Background Low-energy diets (LEDs) comprise commercially formulated food products that provide between 800 and 1200 kcal/day (3.3–5 MJ/day) to aid body weight loss. Recent small-scale studies suggest that LEDs are associated with marked changes in the gut microbiota that may modify the effect of the LED on host metabolism and weight loss. We investigated how the gut microbiota changed during 8 weeks of total meal replacement LED and determined their associations with host response in a sub-analysis of 211 overweight adults with pre-diabetes participating in the large multicentre PREVIEW (PREVention of diabetes through lifestyle intervention and population studies In Europe and around the World) clinical trial. Methods Microbial community composition was analysed by Illumina sequencing of the hypervariable V3-V4 regions of the 16S ribosomal RNA (rRNA) gene. Butyrate production capacity was estimated by qPCR targeting the butyryl-CoA:acetate CoA-transferase gene. Bioinformatics and statistical analyses, such as comparison of alpha and beta diversity measures, correlative and differential abundances analysis, were undertaken on the 16S rRNA gene sequences of 211 paired (pre- and post-LED) samples as well as their integration with the clinical, biomedical and dietary datasets for predictive modelling. Results The overall composition of the gut microbiota changed markedly and consistently from pre- to post-LED (P = 0.001), along with increased richness and diversity (both P < 0.001). Following the intervention, the relative abundance of several genera previously associated with metabolic improvements (e.g., Akkermansia and Christensenellaceae R-7 group) was significantly increased (P < 0.001), while flagellated Pseudobutyrivibrio, acetogenic Blautia and Bifidobacterium spp. were decreased (all P < 0.001). Butyrate production capacity was reduced (P < 0.001). The changes in microbiota composition and predicted functions were significantly associated with body weight loss (P < 0.05). Baseline gut microbiota features were able to explain ~25% of variation in total body fat change (post–pre-LED). Conclusions The gut microbiota and individual taxa were significantly influenced by the LED intervention and correlated with changes in total body fat and body weight in individuals with overweight and pre-diabetes. Despite inter-individual variation, the baseline gut microbiota was a strong predictor of total body fat change during the energy restriction period. Trial registration The PREVIEW trial was prospectively registered at ClinicalTrials.gov ( NCT01777893 ) on January 29, 2013.