Health Technology Assessment (Aug 2024)
Development and validation of prediction models for fetal growth restriction and birthweight: an individual participant data meta-analysis
Abstract
Background Fetal growth restriction is associated with perinatal morbidity and mortality. Early identification of women having at-risk fetuses can reduce perinatal adverse outcomes. Objectives To assess the predictive performance of existing models predicting fetal growth restriction and birthweight, and if needed, to develop and validate new multivariable models using individual participant data. Design Individual participant data meta-analyses of cohorts in International Prediction of Pregnancy Complications network, decision curve analysis and health economics analysis. Participants Pregnant women at booking. External validation of existing models (9 cohorts, 441,415 pregnancies); International Prediction of Pregnancy Complications model development and validation (4 cohorts, 237,228 pregnancies). Predictors Maternal clinical characteristics, biochemical and ultrasound markers. Primary outcomes fetal growth restriction defined as birthweight 32 weeks). To assess if the performance of the prediction models is generalisable for various definitions of FGR, and assess the association between various birthweight centiles calculated using customised and population-based standards and perinatal morbidity and mortality. To estimate the net benefit (clinical utility) of the developed prediction models using decision curve analysis (DCA). To assess the costs and outcomes and the potential impact of resource use of the prediction models. Methods We followed existing recommendations for prediction model development and validation and reported in line with guidelines for prognostic research and IPD meta-analysis. Our meta-analysis utilised IPD within the IPPIC Network database. IPPIC is a living data repository of cleaned and harmonised data of pregnant women from large birth or population-based cohorts, study cohort data, registries or unpublished data from hospital records. The primary outcomes were (1) FGR defined as birthweight < 10th centile adjusted for gestational age, with serious complications such as stillbirth, neonatal death, or delivery before 32 weeks and (2) birthweight for deliveries at various potential gestational ages. We updated our previous searches (inception to July 2012) for relevant prediction models published until August 2019 for external validation. Models were validated if at least one IPPIC IPD cohort contained all the predictors included in the model, and the model outcome occurred in some of the participants in the IPD cohort. Partially missing predictors and outcome variables missing for < 90% of individuals in the cohorts were imputed using multiple imputation by chained equations, assuming that individual values were missing at random. Imputation was performed separately for each cohort to allow for the clustering of individuals within cohorts. The predictive performance of existing model was evaluated using measures of calibration (agreement between predicted and observed outcomes), and discrimination (how well model differentiates between those with and without the outcome, ideal value 1) for each cohort separately and then pooled using a random-effects model estimated using restricted maximum likelihood. Candidate predictors for development of FGR and birthweight models were identified following a prioritisation survey by clinical experts and from existing prediction models. Prediction models were developed using random intercept regression models with backward elimination for variable selection, and IECV was used for validation. Model predictive performance measures [calibration-in-the-large (CITL), the calibration slope, the c-statistic and Nagelkerke’s R2] were summarised using random-effects meta-analysis to give a pooled estimate of overall performance across cohorts. We assessed the clinical utility of IPPIC-FGR model using DCA. By weighing up potential benefit and harm, the net benefit of the model was plotted at various clinically relevant threshold probabilities. Decision curves were compared against ‘treat-all’ and ‘treat-none’ strategies across the range of predicted threshold probabilities at which the model may be clinically useful. We also evaluated the costs and outcomes of IPPIC-FGR model using a decision analytical model constructed using Microsoft Excel®. The costs and outcomes of IPPIC-FGR model was compared against existing strategies in the National Institute for Health and Care Excellence (NICE) 2008 Antenatal Care guideline [no monitoring for FGR and monitoring FGR of all fetuses using ultrasound and symphysis-fundal height (SFH) measurement]. Costs were from the perspective of the National Health Service, and no discounting was required due to the short timeframe from entry into the model to outcome. Results External validation of existing prediction models Overall, 119 published prediction models (55 articles) for FGR and birthweight were identified, with various definitions of FGR or birthweight outcome dichotomised. No study reported our predefined outcome of FGR. Of the eleven models that predicted birthweight on a continuous scale, only one (Poon 2011; 33,602 pregnancies) reported variables available in the IPPIC cohorts and was externally validated in nine IPPIC cohorts involving 441,415 pregnancies. The Poon model included gestational age at delivery, maternal weight, height, age, parity, smoking status, ethnicity, history of chronic hypertension, diabetes and assisted conception. Calibration slopes of the model ranged from 0.91 to 1.05, with a pooled calibration slope across all cohorts of 0.974 [95% confidence interval (CI) 0.938 to 1.011, τ2=0.0018]. On average, the model systematically underpredicted birthweight by 90.4 g (37.9 g to 142.9 g) across the validation cohorts and showed moderate heterogeneity in performance. Development and validation of IPPIC-FGR and IPPIC-birthweight models We developed the IPPIC-FGR model using data from four IPPIC cohorts (237,228 pregnancies). The model included gestational age at delivery, mother’s age, mother’s height, parity, smoking status, ethnicity, history of hypertension, and any history of pre-eclampsia, stillbirth or small for gestational age baby. The pooled apparent c-statistic was 0.96 (95% CI 0.51 to 1.0), and the pooled apparent calibration slope was 0.95 (95% CI 0.67 to 1.23). The IPPIC-birthweight model additionally included maternal weight, a history of diabetes and mode of conception, and was developed in same four IPPIC cohorts as for the IPPIC-FGR model. The pooled calibration slope across cohorts in the IECV was 1.0 (95% CI 0.78 to 1.23), thus showing no evidence of overfitting. Underestimation of birthweight was by 9.7 g on average across cohorts in the IECV (95% CI −154.3 g to 173.8 g) as assessed by CITL. Decision curve analysis The IPPIC-FGR model showed positive net benefit for predicted probability thresholds between 1% and 90% across all cohorts compared to a strategy of managing all pregnant women as if they will have growth-restricted fetuses, or managing them as if none will have growth-restricted fetuses (i.e. treat-all or treat-none strategies). Net benefit was greatest when the model was used in pregnancies <32 weeks’ gestation. While there was no overall benefit in using the IPPIC-FGR model in pregnancies at or above 32 weeks’ gestation compared to a strategy of treat-all, use of the model in pregnant women at this gestational age resulted in no additional harm in these group of women. Health economics analysis The health economics analysis based on NICE 2008 economic model for monitoring fetal growth showed the use of the IPPIC-FGR model was slightly more costly, and more perinatal deaths were saved for every 1000 FGR babies than the alternate strategy of no screening for FGR. When the IPPIC-FGR model was compared with screening using only SFH and ultrasound, the strategy was cheaper and again more perinatal deaths were prevented. Sensitivity analysis found that the results were robust and in line with the base-case analyses. The economic model did not take into account current pathways used to screen women at high risk of having FGR babies. Recommendations for clinical practice and research Incorporation of personalised predicted birthweight estimates (for various potential gestational ages) within existing growth charts, and risk stratification at booking for FGR can help plan intensity of fetal monitoring and timing of delivery. The impact of using IPPIC-FGR and IPPIC-birthweight models on changes in clinical practice and clinical outcomes needs further evaluation. Qualitative data are needed to determine the barriers and facilitators of their routine implementation in clinical practice. Our health economics analysis was based on the 2008 NICE model which is no longer reflective of current management strategies for risk assessing FGR. Therefore, in light of significant changes to current guidelines and care pregnant women at risk of FGRs receive, a detailed full economic evaluation is needed, which evaluates various strategies to risk assess FGR along current care pathways. Conclusion IPPIC-FGR and IPPIC-birthweight models accurately predict FGR and birthweight. The latter has better calibration than existing model. IPPIC-FGR model use is cost-effective. Both IPPIC models can help plan intensity of fetal monitoring in pregnancy and timing of delivery, to minimise adverse perinatal outcomes. Study registration This study is registered as PROSPERO CRD42019135045. Funding This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: 17/148/07) and is published in full in Health Technology Assessment; Vol. 28, No. 47. See the NIHR Funding and Awards website for further award information.
Keywords
