How Useful is Our IVF Prediction Model?
A measure of whether a prediction model is useful for physicians and patients, or clinical utility, is determined by its ability to identify patients with a certain prognosis.
Clinical utility can benefit patients directly by providing prognostic information that would otherwise not be available, or it can benefit patients indirectly by giving providers special insight to support more effective and less risky optimization of clinical protocols.
We can think of clinical utility in terms of whether a new IVF prediction model reclassifies the patients to have a different probability of IVF success, compared to estimates of their success without using the model.
Of course, predicting a different probability of success is not valuable unless the prediction itself is proven to be more accurate. In addition, usefulness can be measured by the model’s ability to distinguish patients based on their percentile ranking. Now, I will illustrate the concept of reclassification as validated by our research findings.
Nearly a decade of research culminated in published findings that validate the usefulness of two of our prediction models: Choi et al, Fertil Steril 2013 (PreIVF-D) and Choi et al. presented at the Society for Gynecologic Investigations 60th Annual Scientific Meeting, Orlando Fl., 2013 (PredictIVF-D). (The PreIVF-D and the PredictIVF-D models are the basis for the Univfy® PreIVF™ Report.)
The PreIVF-D model predicts the probability of live birth in the first IVF cycle and was tested retrospectively to determine its clinical utility.
Diverse Data
We wanted to create a model that could be used by the vast majority of clinics and so did not want the model to depend on very specific types of patient populations, payment systems, or clinical protocols.
Therefore, data from three university-affiliated outpatient IVF clinics in three different countries was analyzed in the study to ensure that diverse sampling of data was represented.
Data from over 13,000 first IVF cycles was used to build a primary model, which was then “trained” to create the PreIVF Diversity (PreIVF-D) model applying a statistical method called boosted tree to analyze data from over 1,000 first IVF cycles. The resulting PreIVF-D model was independently validated using another set of more than 1,000 first IVF cycles.
Live birth probability as predicted using the PreIVF-D model was compared to the probability based on maternal age alone, as age is the prevailing method or primary predictor used in infertility medical literature and websites.
In 86% of cases studied, PreIVF-D generated significantly different probabilities of success than those based on age, and more than half had higher live birth probabilities than predicted by age.
Over 40% of the patients had a personalized predicted success rate of greater than 45%, whereas the age-control model could not differentiate these patients from others. Most importantly, PreIVF-D had more than a 1,000-fold greater predictive power than the age-control model (likelihood scale).
In another research project focusing on chances of IVF success after one or more prior IVF treatments, PredictIVF Diversity (PredictIVF-D) showed similar improved live birth predictive power compared to age-based prediction.
Boosted tree methods were used to analyze live birth outcomes from over 20,000 IVF cycles to create the PredictIVF-D model. This preliminary model was then “trained” using seven years of data and validated using a separate data set derived from more than 1,000 IVF cycles.
In 73% of cases, PredictIVF-D live birth probabilities were different than those generated by the age-based control model. PredictIVF-D showed a 75% improved predictive power and increased the predictive dynamic range from four discrete age-based probabilities to a continuous probability spectrum from 7% to 60%.
Individualized Prognostics
This greater predictive power over a wider probability spectrum means that both patients who are more likely and those who are less likely to conceive from IVF can be identified.
With this knowledge, doctors can make better recommendations about IVF treatment, which is a benefit for all patients, both those who can get pregnant sooner using IVF and those who can choose other paths to parenthood sooner, knowing that IVF is highly unlikely to work for them.
Validation of these IVF prediction models is a major milestone in our efforts to provide new tools for better personalized counseling and thus better outcomes for patients.
About the Author:
Mylene Yao, M.D. | Co-Founder and CEO of Univfy®
Dr. Mylene Yao is a board-certified OB/GYN with more than 20 years of experience in clinical and reproductive medicine research. Prior to founding Univfy, she was on the faculty at Stanford University, where she led NIH-funded fertility and embryo genetics research and developed The Univfy AI Platform with the academic founding team. See her full bio here.