AI Approach Can Predict Placebo Responders in Clinical Trials in Major Depressive Disorder
Key findings
- An individual's propensity to respond to placebo is unknown at baseline and so cannot be controlled by the standard randomization process for clinical trials; therefore, the groups compared may be unbalanced
- This paper describes the use of an artificial neural network to predict an individual's propensity to respond to placebo and a propensity weighting methodology that reduces baseline imbalances between treatment arms
- When applied to data from a randomized, controlled trial that compared paroxetine with placebo, the model performed well in predicting placebo response in the placebo group (AUC, 0.81; 95% CI, 0.64–0.97)
- When the model was used to predict all subjects' propensity to respond to placebo, a large majority of subjects had a high (>0.8) probability, and the treatment effect of paroxetine was larger when the propensity weighting approach was applied
- Regulatory agencies should consider encouraging propensity weighting methodology for predicting placebo response in randomized, controlled trials of interventions for depression and other central nervous system disorders
In randomized, controlled trials, the placebo effect is the clinical improvement associated with a patient's interactions with their clinician, the information they receive about their condition and treatment, the therapeutic care conditions, and the expectation of a clinical benefit. As the placebo effect increases, the difference between the placebo and active arms decreases, reducing the likelihood the trial will meet its primary endpoint.
Subscribe to the latest updates from Psychiatry Advances in Motion
Most attempts to deal with this problem involve excluding study subjects who respond to placebo during run-in periods or removing centers with high placebo response rates.
Now, Maurizio Fava, MD, psychiatrist-in-chief and executive director of the Clinical Trials Network and Institute at Massachusetts General Hospital, in collaboration with Roberto Gomeni and Françoise Bressolle-Gomeni of Pharmacometrica, have developed a propensity weighting method that doesn't require excluding any subject from randomization and analysis, they report a sample application of the model in Translational Psychiatry.
Methods
Using a GlaxoSmithKline database, the researchers obtained unpublished results from a randomized, double-blind trial that compared placebo with two fixed doses of controlled-release paroxetine. An artificial intelligence (AI) model was developed to predict each individual's propensity to respond to placebo at week 8, the end of the trial.
Placebo response was defined as a clinically relevant change from baseline in the Montgomery–Åsberg Depression Rating Scale or the 17-item Hamilton Depression Rating Scale (reduction ≥38% or ≥41% in total score, respectively). The propensity-weighted analysis was conducted in five steps:
- The pre-randomization data (i.e., screening and baseline) and week 8 data were gathered for study participants who had been assigned to placebo
- An artificial neural network was created to estimate the probability of responding to placebo by week 8
- The model developed in step 2 was validated by comparing the model-predicted probability of placebo response to the actual placebo response
- Based on pre-randomization data, the model was used to predict the individual probability of all study participants to respond to placebo
- The inverse of the individual probability was used as a weighting factor in a mixed-effect model for repeated measures (MMRM) analysis that assessed the treatment effect (the baseline-corrected change from placebo at the study end)
Results
The model's ability to predict placebo response in the placebo group was very good, as the area under the receiver operating characteristic curve was 0.81 (95% CI, 0.64–0.97).
When the model was used to predict each subject's propensity to respond to placebo, there was a large imbalance in the distribution of probabilities, and a large majority of subjects had a high (>0.8) propensity to show a placebo response.
The treatment effect of paroxetine was larger when the propensity weighting factor was included in an MMRM analysis.
Commentary
The FDA and other regulatory agencies already support using propensity weighting to ensure the comparability of treatment arms in observational studies. The results of this analysis suggest the agencies should consider extending the methodology to randomized, controlled trials in depression and other central nervous system disorders.
The method described here could be applied to any prospective phase 2 or phase 3 trial when the following conditions are satisfied:
- The study includes a collection of screening and pre-treatment baseline data
- The criteria for assessing response to placebo are pre-specified in the statistical analysis plan
- The acceptable criteria for the predictive performance of the artificial neural network are also pre-specified in the statistical analysis plan, and the specified AUC cut-offs are greater than the noninformative threshold of 0.5
view original journal article Subscription may be required
Explore research in the Department of Psychiatry
Learn more about the Mass General Research Institute