Of Ventilators and the Undiscovered Country

Many people are saying..mortality among intubated patients with COVID-19 is exceedingly high.

Subscribe to the latest updates from FLARE Advances in Motion

Thank you for subscribing!

Error: Please enter a valid email address.

Email Address Submit

Introduction

Multiple media outlets have published articles discussing an April 22nd report in JAMA (Richardson et al. 2020). One CNN headline, which is representative of other descriptions, reads: “Nearly all Covid-19 patients put on ventilators in New York's largest health system died, study finds.” In this FLARE, we explore the data presented in the JAMA study and what it actually says about mortality among patients requiring invasive mechanical ventilation.

We Start by Asking, “How Many Patients Received Invasive Mechanical Ventilation?”

This is the denominator in the mortality figure -- the total patients receiving the treatment. The authors state they will be describing outcomes for 320 patients who were discharged alive or who died by the end of data collection for their reported results.

However, a total of 1151 patients were actually mechanically ventilated, not just 320 (see Table 1 below). This includes patients currently admitted, patients discharged and still alive, and patients who died. Of the total cohort of 1151 patients receiving invasive mechanical ventilation, 831 (72%) remain alive and in the hospital at the time that this study chose to examine endpoints; these patients’ outcomes were censored (namely, their outcomes were unknown at the time of data collection). Presumably, they were not included in the mortality calculation because they had not reached the outcome of interest (death or discharge). We cannot overemphasize this point -- the mortality calculation erroneously excludes from the denominator these many patients who received the exposure of interest (ventilation). We don’t know how many are still ventilated, how many were extubated, or other information about their clinical course.

This disconnect between the denominator reported by the authors and the still unknown true denominator has resulted in grave misinterpretation of these data in the media. The facts remain that most of the intubated patients in this cohort are still alive and their outcomes are censored. This is an intrinsic challenge in publishing early cohort data when many patients have not reached any “outcome,” such as death or discharge alive.

Earlier FLAREs (April 19, April 8) described a NYC case series (Goyal et al. 2020) and ICU experience in Italy (Grasselli et al. 2020). These studies reported 14.6% mortality rates among mechanically ventilated patients, and 26% ICU mortality rates at the time of censoring. The wide differences in reported mortality represent, in part, different choices on the way to deal with unknown outcomes. Most case series in COVID-19 have reported mortality as the number of fatalities in the numerator and total number of patients in the denominator - this can understate the mortality if large numbers of patients who have not completed their course of treatment subsequently die. But to report on only patients who have reached some definitive outcome (death or discharge), as the authors of the JAMA report have done, can overstate (more likely) or understate mortality because patients with early achievement of definitive outcomes are likely to be qualitatively different from the remaining patients - either much sicker or much healthier.

The following table refers to patients who received mechanical ventilation in the study:

Why Are Most Intubated Patients Still Alive With Unknown Outcomes at the Time of Censoring?

The length of stay for the patients who had complete outcomes data (and were therefore included in the analysis) was relatively short -- about 4 days. Observations over such a short period of time are not informative in the study of critical illness, where treatment and recovery can occur over weeks. From published reports (Bhatraju et al. 2020; Grasselli et al. 2020), patients who receive mechanical ventilation for severe COVID-19 routinely require several days of mechanical ventilation. Those patients may then require 1-2 weeks more in the hospital. It would be highly unusual for a patient to be critically ill, require mechanical ventilation, and yet be alive and discharged 4 days after hospital admission. An analysis which focuses only on the small group of patients with complete data is likely to overstate the mortality since most intubated patients will either remain intubated, and therefore be excluded from the analysis, or be among the sickest possible patients who quickly die.

Choices about handling missing data, therefore, may either overstate or understate the true mortality. But the risks associated with over- and understatement are not the same. Overstatement of mortality may be perceived by some to justify the use of unproven therapies. This could be dangerous and would take away resources from the necessary high quality randomized controlled trials in critically ill COVID-19 patients.

Mortality, Data Censoring, and Schröedinger’s Ventilator

In summary, Richardson and colleagues describe a large cohort of patients, providing additional data on baseline patient characteristics. However, these incomplete data cannot be used to determine mortality rates among ventilated patients. The majority of patients (72%) requiring mechanical ventilation in this cohort remain in the hospital yet were excluded from mortality analysis. They are left in limbo, supported by what can only be described as Schröedinger’s ventilator - both alive and dead in the data until a future time when investigators can ‘open the box’ and analyze the complete outcomes.

References:

• Bhatraju, Pavan K., Bijan J. Ghassemieh, Michelle Nichols, Richard Kim, Keith R. Jerome, Arun K. Nalla, Alexander L. Greninger, et al. 2020. “Covid-19 in Critically Ill Patients in the Seattle Region - Case Series.” The New England Journal of Medicine, March. https://doi.org/10.1056/NEJMoa2004500.
• Goyal, Parag, Justin J. Choi, Laura C. Pinheiro, Edward J. Schenck, Ruijun Chen, Assem Jabri, Michael J. Satlin, et al. 2020. “Clinical Characteristics of Covid-19 in New York City.” The New England Journal of Medicine, April. https://doi.org/10.1056/NEJMc2010419.
• Grasselli, Giacomo, Alberto Zangrillo, Alberto Zanella, Massimo Antonelli, Luca Cabrini, Antonio Castelli, Danilo Cereda, et al. 2020. “Baseline Characteristics and Outcomes of 1591 Patients Infected With SARS-CoV-2 Admitted to ICUs of the Lombardy Region, Italy.” JAMA: The Journal of the American Medical Association, April. https://doi.org/10.1001/jama.2020.5394.
• Richardson, Safiya, Jamie S. Hirsch, Mangala Narasimhan, James M. Crawford, Thomas McGinn, Karina W. Davidson, Douglas P. Barnaby, et al. 2020. “Presenting Characteristics, Comorbidities, and Outcomes Among 5700 Patients Hospitalized With COVID-19 in the New York City Area.” JAMA: The Journal of the American Medical Association, April. https://doi.org/10.1001/jama.2020.6775.

View all FLARE content

Learn more about research in the Division of Pulmonary and Critical Care Medicine