Skip to content

Predictive Utility of Natural Language Processing Algorithms for Posterior Lumbar Fusion Patients

Key findings

  • This exploratory analysis evaluated the utility of various sets of notes captured in electronic health records as input for natural language processing (NLP) algorithms designed to predict 90-day readmission after elective lumbar spinal fusion surgery
  • The area under the receiver operating curve was highest for an algorithm using discharge summary notes as input (0.70), with moderate values for physical therapy (0.60), case management (0.60), operative (0.57) and nursing (0.57) notes
  • Progress notes written by medical doctors, nurse practitioners or physician assistants were least useful for predicting 90-day readmission (0.49)
  • Advantages of predicting readmission with NLP algorithms are the automated review of records and the lack of any requirement for programming or data entry

Since 2012, the Centers for Medicare and Medicaid Services has reduced payments to hospitals that readmit patients after care for six common conditions and surgical procedures. The program has been so successful, it's expected to be expanded to additional high-cost procedures, including spine surgery.

Over the past few years, Massachusetts General Hospital researchers have been developing machine learning methods for detecting or predicting adverse outcomes. Natural language processing (NLP) algorithms they've developed, which automatically process narrative notes available in electronic health records (EHR), have proven capable of identifying patients with incidental durotomy, wound infection or intraoperative vascular injury.

As a first step toward predicting risk of readmission after posterior lumbar fusion, a Mass General research team explored which free-text EHR notes have the most utility in that patient population. Aditya V. Karhade, MD, MBA, a resident in the Department of Orthopaedic Surgery, Joseph H. Schwab, MD, MS, chief of the Orthopaedic Spine Center and director of spine oncology at Mass General, and co-director of the Stephan L. Harris Center for Chordoma Care at Mass General Cancer Center, and colleagues report the results in The Spine Journal.


The researchers applied machine−learning−based NLP algorithms to EHRs for 708 patients from two academic medical centers and three community hospitals in the Mass General Brigham system. The patients had undergone inpatient posterior lumbar fusion for spinal stenosis or spondylolisthesis between January 1, 2016, and December 31, 2020.

The algorithms were developed on a training set of 567 patients and were tested on the other 141 patients. Six types of notes generated during the index hospitalization were examined: discharge summary notes, operative notes, nursing notes, physical therapy notes, case management notes and progress notes written by a medical doctor, nurse practitioner or physician assistant.

Essentially, algorithms were trained to recognize the frequencies of words in the different sets of notes, then were compared to determine how well they predicted readmission within 90 days. No programming was required, but EHR data were preprocessed to standardize the input for the algorithms. For example, all words were converted to lowercase and some words were converted to stems, such as "wheez" for "wheezing."


In the testing set of patients, the area under the receiver operating curve for prediction of 90-day readmission was:

  • 0.70 for an algorithm that used all notes as input data
  • 0.70 using discharge summary notes
  • 0.60 using physical therapy notes
  • 0.60 using case management notes
  • 0.57 using operative notes
  • 0.57 using nursing notes
  • 0.49 using progress notes

The three most highly weighted key terms in discharge summary notes were "ipratropium," "type_2" (diabetes) and "inci_clean" (clean incision).

Advancing the Technology

Previous studies of readmission risk after posterior lumbar spinal fusion have involved manual chart review, inclusion only of EHR data that was entered into predefined categories or examination of databases that required data entry by trained clinicians. In contrast, NLP algorithms are automated and search free-text documentation that already exists for legal and billing purposes.

If an algorithm proves capable of detecting patients at risk of readmission after spinal surgery, it might minimize the financially motivated selection bias against surgical candidates whose cases are medically complicated.

This NLP research is a step towards creating autonomous systems for identifying patients at high risk of post-discharge complications, such as readmission, and appropriately triaging constrained resources to yield the greatest net benefit.

Learn more about the Orthopaedic Spine Service

Learn more about orthopaedic fellowships at Mass General

Related topics


Aditya V. Karhade, MD, MBA, Christopher M. Bono, MD, Joseph H. Schwab, MD, MS, of the Department of Orthopaedic Surgery, and colleagues review recent progress in each of the six core areas of value-based health care and suggest how spine surgery centers can remain competitive in the years ahead.


Using an online survey, Quirina Thio, Joseph Schwab, MD, and colleagues documented poor agreement between surgeons who were asked to estimate survival for 12 patients with spinal metastasis. There was also a substantial discrepancy between the surgeons' estimates and the patients' actual survival.