Predictive Utility of Natural Language Processing Algorithms for Posterior Lumbar Fusion Patients
Key findings
- This exploratory analysis evaluated the utility of various sets of notes captured in electronic health records as input for natural language processing (NLP) algorithms designed to predict 90-day readmission after elective lumbar spinal fusion surgery
- The area under the receiver operating curve was highest for an algorithm using discharge summary notes as input (0.70), with moderate values for physical therapy (0.60), case management (0.60), operative (0.57) and nursing (0.57) notes
- Progress notes written by medical doctors, nurse practitioners or physician assistants were least useful for predicting 90-day readmission (0.49)
- Advantages of predicting readmission with NLP algorithms are the automated review of records and the lack of any requirement for programming or data entry
Since 2012, the Centers for Medicare and Medicaid Services has reduced payments to hospitals that readmit patients after care for six common conditions and surgical procedures. The program has been so successful, it's expected to be expanded to additional high-cost procedures, including spine surgery.
Subscribe to the latest updates from Orthopaedics Advances in Motion
Over the past few years, Massachusetts General Hospital researchers have been developing machine learning methods for detecting or predicting adverse outcomes. Natural language processing (NLP) algorithms they've developed, which automatically process narrative notes available in electronic health records (EHR), have proven capable of identifying patients with incidental durotomy, wound infection or intraoperative vascular injury.
As a first step toward predicting risk of readmission after posterior lumbar fusion, a Mass General research team explored which free-text EHR notes have the most utility in that patient population. Aditya V. Karhade, MD, MBA, a resident in the Department of Orthopaedic Surgery, Joseph H. Schwab, MD, MS, chief of the Orthopaedic Spine Center and director of spine oncology at Mass General, and co-director of the Stephan L. Harris Center for Chordoma Care at Mass General Cancer Center, and colleagues report the results in The Spine Journal.
Methods
The researchers applied machine−learning−based NLP algorithms to EHRs for 708 patients from two academic medical centers and three community hospitals in the Mass General Brigham system. The patients had undergone inpatient posterior lumbar fusion for spinal stenosis or spondylolisthesis between January 1, 2016, and December 31, 2020.
The algorithms were developed on a training set of 567 patients and were tested on the other 141 patients. Six types of notes generated during the index hospitalization were examined: discharge summary notes, operative notes, nursing notes, physical therapy notes, case management notes and progress notes written by a medical doctor, nurse practitioner or physician assistant.
Essentially, algorithms were trained to recognize the frequencies of words in the different sets of notes, then were compared to determine how well they predicted readmission within 90 days. No programming was required, but EHR data were preprocessed to standardize the input for the algorithms. For example, all words were converted to lowercase and some words were converted to stems, such as "wheez" for "wheezing."
Results
In the testing set of patients, the area under the receiver operating curve for prediction of 90-day readmission was:
- 0.70 for an algorithm that used all notes as input data
- 0.70 using discharge summary notes
- 0.60 using physical therapy notes
- 0.60 using case management notes
- 0.57 using operative notes
- 0.57 using nursing notes
- 0.49 using progress notes
The three most highly weighted key terms in discharge summary notes were "ipratropium," "type_2" (diabetes) and "inci_clean" (clean incision).
Advancing the Technology
Previous studies of readmission risk after posterior lumbar spinal fusion have involved manual chart review, inclusion only of EHR data that was entered into predefined categories or examination of databases that required data entry by trained clinicians. In contrast, NLP algorithms are automated and search free-text documentation that already exists for legal and billing purposes.
If an algorithm proves capable of detecting patients at risk of readmission after spinal surgery, it might minimize the financially motivated selection bias against surgical candidates whose cases are medically complicated.
This NLP research is a step towards creating autonomous systems for identifying patients at high risk of post-discharge complications, such as readmission, and appropriately triaging constrained resources to yield the greatest net benefit.
view original journal article Subscription may be required
Learn more about the Orthopaedic Spine Service
Learn more about orthopaedic fellowships at Mass General