Natural Language Processing of Admission Notes at Delivery Predicts Severe Maternal Morbidity

The rates of severe maternal morbidity (SMM) and mortality are higher in the U.S. than in other developed countries. More accurate risk stratification is a public health priority.

Subscribe to the latest updates from OB/GYN Advances in Motion

Thank you for subscribing!

Error: Please enter a valid email address.

Email Address Submit

Mark A. Clapp, MD, MPH, a specialist in the Maternal–Fetal Medicine Program in the Department of Obstetrics and Gynecology at Massachusetts General Hospital, Thomas H. McCoy, MD, the director of research at the Center for Quantitative Health in the Department of Psychiatry, and colleagues are developing a machine learning solution. As they explain in the American Journal of Obstetrics & Gynecology, their new natural language processing (NLP) algorithm identified a group of patients at high risk of SMM using only the history and physical notes available at the time of admission for delivery.

Methods

An NLP computer algorithm filters through the free text of electronic health records and identified words that predict an outcome. In this case, the research team limited the algorithm to single words that occurred in ≥5% and ≤80% of history and physical notes, for a vocabulary of 1,126 unique words.

The records came from patients admitted for delivery at Mass General and Brigham and Women's Hospital between July 1, 2016, and June 30, 2020. At one of the hospitals, there were records on 13,572 patients, which were split between a training set of records for deriving the algorithm (n=10,250) and a test set for retrospectively validating it internally (n=3,322).

At the other hospital, there were records on 23,397 patients, which were used for external validation.

Primary Outcome

The primary outcome was SMM, meaning any of the conditions listed in the composite definition of the Centers for Disease Control and Prevention:

• Internal test dataset—area under the curve (AUC) for the ability of the algorithm to predict SMM, 0.67 (95% CI, 0.63–0.72)
• External validation dataset—AUC, 0.72 (95% CI, 0.70–0.74)

In the external validation dataset, the algorithm distinguished a high-risk group, patients in the top two deciles of predicted risk (8.3% developed SMM vs. 1.9% in the lower two deciles; P<0.001).

Secondary Outcome

Many of the conditions included in the CDC composite definition of SMM relate to transfusion, so the researchers also examined non-transfusion SMM:

• Internal test dataset—AUC, 0.72 (95% CI, 0.65–0.80)
• External validation dataset—AUC, 0.76 (95% CI, 0.73–0.79)

In the external validation dataset, the high-risk group was patients in the top decile of predicted risk (7.1% developed non-transfusion SMM vs. 0.9% of those in lower deciles; P<0.001).

Laying the Groundwork

These results are early evidence that the NLP algorithm could scan an admission note and identify a patient presenting for labor and delivery as being at high risk of severe morbidity prospectively and without delaying clinical workflows. Notably, the algorithm doesn't require any demographic information or diagnosis codes.

Once developed into a tool, the algorithm would be an adjunct to clinician judgment. Local protocols would presumably influence the exact thresholds for what constitutes "high risk" and the subsequent actions to be taken, such as alerting the attending provider or initiating a care team huddle.

view original journal article Subscription may be required

Learn more about the Department of Obstetrics & Gynecology

Refer a patient to the Department of Obstetrics & Gynecology