Magnetic resonance imaging (MRI) is a promising non-invasive imaging technique that can be safely used to study placental development and function. However, studies of the human placenta performed by MRI are limited by uterine motion and motion in the uterus during MRI remains one of the major limiting factors. Here, we aimed to investigate the characterization of uterine activity during MRI in the second trimester of pregnancy using MRI-based motion tracking. In total, 46 pregnant women were scanned twice (first scan between 14 and 18 weeks and second scan between 19 and 24 weeks), and 20 pregnant subjects underwent a single MRI between 14 and 18 weeks GA, resulting in 112 MRI scans. An MRI-based algorithm was used to track uterine motion in the superior-inferior and left-right directions. Uterine contraction and maternal motion cases were separated by the experts, and unpaired Wilcoxon tests were performed within the groups of gestational age (GA), fetal sex, and placental location in terms of the overall intensity measures of the uterine activity. In total, 22.3% of cases had uterine contraction during MRI, which increased from 18.6% at 14–18 weeks to 26.4% at 19–24 weeks GA. The dominant direction of the uterine contraction and maternal motion was the superior to the inferior direction during early gestation.