Aflatoxin B1 (AFB1) is a common toxic mycotoxin and is detectable in pregnant women. Animal studies have revealed that AFB1 caused the lysis of erythrocytes and a decrease in hemoglobin. We conducted a prospective cohort study in Guangxi, China, in order to evaluate the association between AFB1 exposure and anemia in pregnant women during the entire pregnancy. A total of 616 pregnant women from the Guangxi Zhuang Birth Cohort were included in the study. Serum AFB1-albumin (AFB1-ALB) adduct levels were measured. The effect of AFB1-ALB adducts on hemoglobin (Hb), mean corpuscular volume (MCV), mean corpuscular hemoglobin (MCH), and mean corpuscular hemoglobin concentration (MCHC) were analyzed by using multivariable linear regression. The risks of anemia from AFB1-ALB adduct exposure were assessed by multivariable logistic regression. We found that the AFB1-ALB adduct was significantly associated with a decrease in Hb (β = −4.99, 95% CI: −8.42, −1.30), MCV (β = −4.58, 95% CI: −7.23, −1.94), MCH (β = −1.86, 95% CI: −2.87, −0.85), and MCHC (β = −5.23, 95% CI: −8.28, −2.17) in the first trimester with the third tertile of AFB1-ALB adducts when compared with the first tertile. Furthermore, the third tertile of the AFB1-ALB adduct significantly increased the risk of anemia by 2.90 times than compared to the first tertile in the first trimester (OR = 3.90, 95% CI: 1.67, 9.14). A significant positive does–response relationship existed between AFB1-ALB adduct levels and anemia risk (Ptrend = 0.001). When dividing anemia types, we only found that the third tertile of AFB1-ALB adduct increased the risk of microcytic hypochromic anemia (MHA) in the first trimester (OR = 14.37, 95% CI: 3.08, 67.02) and second trimester (OR = 4.75, 95% CI: 1.96, 11.51). These findings demonstrate the correlation between maternal AFB1 exposure during early pregnancy and risk of anemia, especially MHA, and during different trimesters in Southern China. More efforts should be made to diminish AFB1 exposure for pregnant women.