Neural Network Beats Experts at Detecting Epileptiform Discharges on EEGs

Neurologists with fellowship training in clinical neurophysiology are the recognized experts at detecting interictal epileptiform discharges (IEDs)—the hallmark of epilepsy—on electroencephalograms (EEGs). However, these subspecialists are scarce, so most EEGs in the U.S. are read by nonspecialists. In much of the world, EEG expertise is unavailable altogether.

Subscribe to the latest updates from Neuroscience Advances in Motion

Thank you for subscribing!

Error: Please enter a valid email address.

Email Address Submit

Even when a clinical neurophysiologist reads EEGs, it's usually impractical for them to manually annotate IEDs thoroughly, particularly in recordings made over hours or days. There's a need for automated detection of IEDs. But efforts toward this goal have been limited by a lack of large, well-annotated data sets for training such an algorithm, the iterative process of refining its accuracy.

Jin Jing, PhD, Haoqi Sun, PhD, Jennifer A. Kim, MD, PhD, and M. Brandon Westover, MD, PhD, of the Department of Neurology at Massachusetts General Hospital, and colleagues recently used 9,571 EEG recordings to train a deep neural network to detect IEDs. In JAMA Neurology, they report that the algorithm, which they've named SpikeNet, performed at or above the accuracy of a panel of fellowship-trained clinical neurophysiologists.

Methods for Training

The researchers selected 1,051 EEGs, with and without IED waveforms, that had been reviewed by eight experts as part of clinical care at Mass General. Using those EEGs and 8,520 control EEGs that did not have IEDs, they trained SpikeNet to detect IEDs, then to classify an entire EEG as containing IEDs or no IEDs.

Methods for Testing

For detecting IEDs, and for classifying whole EEGs, SpikeNet's performance was measured in two ways:

• Binary classification — SpikeNet was tested for its ability to discriminate definite IEDs (those with more than six votes from the eight experts) from definite non-IEDs (those with one or zero votes). Its performance was measured via area under the receiver operating characteristic curve (AUC). SpikeNet was compared with the standard commercial IED detection software, Persyst 13
• Calibration between SpikeNet and experts on all labeled IEDs — A calibration curve for each expert or SpikeNet showed the probability of voting for IEDs (y-axis) versus the fraction of experts who agreed (x-axis). The calibration error was the average deviation from the diagonal line, with zero representing perfect calibration

Results

Detection of IEDs:

• Binary classification — SpikeNet achieved an AUC of 0.980, outperforming Persyst 13, which had an AUC of 0.882 (P < .05)
• Calibration error — The average calibration error for SpikeNet was 0.041, which outperformed the experts (0.183) and Persyst 13 (0.066) (P < .05 for both comparisons)

Detection of Whole EEGs:

• Binary classification — SpikeNet performed satisfactorily with an AUC of 0.847
• Calibration error — SpikeNet achieved an average calibration error of 0.126, compared with 0.197 for the expert panel

A Useful Tool for Accelerated Review of EEGs

The hope for SpikeNet is that it will facilitate accurate diagnosis of a broad range of patients with epilepsy and other disorders in which IED detection is necessary, such as delayed cerebral ischemia in subarachnoid hemorrhage. It should also be able to warn of clinical decline in critically ill patients.

At first, SpikeNet will presumably be integrated into neurophysiologists' practices to accelerate their work. Subsequent research, though, should focus on removing the need for subspecialists to be involved in the tedious task of annotating EEGs.

view original journal article Subscription may be required

Learn more about the Department of Neurology at Mass General

Learn more about neurology research at Mass General