Ensuring Quality Care in the Assisted Reproductive Technology Laboratory With AI
Key findings
- This retrospective study evaluated whether a convolutional neural network (CNN) that predicts embryo implantation could serve as a quality assurance tool in the Massachusetts General Hospital Fertility Center
- The CNN makes its predictions by analyzing images of embryo development at 113 hours post-insemination captured by EmbryoScope time-lapse imaging
- CNN-predicted implantation rates were compared with actual pregnancy outcomes for physician- and embryologist-performed embryo transfers and embryologist-performed vitrification, warming and trophectoderm biopsy procedures
- For most providers, actual implantation rates were not significantly different from CNN predictions and were within 1 standard deviation of the expected rate
- The CNN identified two providers with performance below the AI-predicted implantation rates for embryo transfers and one embryologist with performance below the threshold for embryo warming and transfers. AI allowed for faster identification and feedback
In an assisted reproductive technology (ART) practice, successful fertilization, extended culture, and transfer depend on the technical dexterity of each individual physician and embryologist. Because of the delicate nature of ART procedures, quality assurance (QA)/quality control is essential.
Subscribe to the latest updates from OB/GYN Advances in Motion
A convolutional neural network (CNN), an image-based form of artificial intelligence (AI), is a cutting-edge tool to address this QA challenge. CNNs have pattern recognition capabilities far beyond human visual discrimination abilities, and they're exceptionally consistent for embryo grading and predicting an embryo's development.
Researchers at Massachusetts General Hospital have become the first to show that a CNN can be used to assess the performance of physicians and embryologists in an ART laboratory in a variety of procedures, including embryo transfer. Panagiotis Cherouveim, MD, a research fellow in the Department of Obstetrics and Gynecology, Victoria S. Jiang, MD, a clinical fellow in the Fertility Center, Charles L. Bormann, PhD, director of the Vincent Embryology Laboratory, and colleagues report in the Journal of Assisted Reproduction and Genetics.
Methods
At the Mass General Fertility Center, embryo development is recorded with EmbryoScope (Vitrolife, Sweden) time-lapse imaging. The CNN used in this study was developed through an ongoing collaboration between Brigham and Women's Hospital and Mass General. This AI algorithm (published in eLife) predicts implantation outcomes by analyzing embryo images collected at 113 hours post-insemination.
The CNN-predicted implantation outcomes were compared with actual outcomes to assess physician and embryologist performance. Throughout the study, all authors were blinded to the identity of those providers.
Results
The researchers analyzed the most recent 20 procedures of each type performed by each provider:
Embryo transfer by 8 physicians (n=160)—The CNN-predicted implantation rate differed significantly from the actual rate for one physician (20% vs. 61%, P=0.001). That physician also had the lowest implantation rate in the cohort (20% vs. 40%–60%). For five physicians, the actual implantation rates fell within 1 SD of the CNN predictions.
Embryo transfer by 8 embryologists (n=160)—The CNN-predicted implantation rate differed significantly from the actual rate for one embryologist (30% vs. 60%, P=0.011). That provider also had the lowest rate (30% vs. 50%–70%) and was the only one whose rate was more than 1 SD different from the CNN prediction.
Embryo vitrification by 8 embryologists (n=160)—The CNN-predicted implantation rates did not differ significantly from the actual rate for any provider. For seven embryologists, the actual rates fell within 1 SD of the CNN predictions.
Embryo warming by 8 embryologists (n=160)—The CNN-predicted implantation rate differed significantly from the actual rate for one embryologist (25% vs. 60%, P=0.004), the same provider who performed most poorly at embryo transfer. The actual implantation rate for embryos warmed by that embryologist was the lowest observed in the cohort (25% vs. 50%–70%). For the seven other embryologists, actual implantation rates were within 1 SD of the CNN predictions.
Trophectoderm biopsy by 6 embryologists (n=120)—The CNN-predicted implantation rate did not differ significantly from the actual rate for any provider. For all six embryologists, the actual rates fell within 1 SD of the CNN predictions.
Commentary
The clinical pregnancy rate isn't an optimal quality control indicator because it can take months to quantify, limiting its value as a prompt warning of poor culture conditions or procedural issues which may impact cycle outcomes. The Vienna consensus suggests implantation rate as an alternative, but embryo quality needs to be factored into account. By using embryo images prior to transfer, the CNN was able to evaluate morphologic markers of embryo quality with an objective lens.
For example, the actual implantation rate for one physician in the embryo transfer analysis was 50%, which might not be concerning during traditional QA evaluation. However, the comparison with the CNN flagged this physician's performance as being more than 1 SD from the predicted rate of 68% based on embryo quality.
Thus, CNN predictions can serve as a benchmark of ART staff performance, and significant deviations suggest the need for improvement in provider technique.
view original journal article Subscription may be required
Learn more about fertility care at Mass General
Refer a patient to the Mass General Fertility Center