Interrater Reliability of the 2015 Pediatric Acute Lung Injury Consensus Conference Criteria for Pediatric ARDS

Background: Diagnostic guidelines for pediatric ARDS (PARDS) were developed at the 2015 Pediatric Acute Lung Injury Consensus Conference (PALICC). Although this was an improvement in creating pediatric-specific diagnostic criteria, there remains potential for variability in identification of PARDS. Research Question: What is the interrater reliability of the 2015 PALICC criteria for diagnosing moderate to severe PARDS? What clinical criteria and patient factors are associated with diagnostic disagreements? Study Design and Methods: Patients with acute hypoxic respiratory failure admitted from 2016 to 2021 who received invasive mechanical ventilation were retrospectively reviewed by two pediatric ICU physicians. Reviewers evaluated whether the patient met the 2015 PALICC definition of moderate to severe PARDS and rated their diagnostic confidence. Interrater reliability was measured using Gwet's agreement coefficient. Results: Thirty-seven of 191 encounters had a diagnostic disagreement. Interrater reliability was substantial (Gwet's agreement coefficient, 0.74; 95% CI, 0.65-0.83). Disagreements were caused by different interpretations of chest radiographs (56.8%), ambiguity in origin of pulmonary edema (37.8%), or lack of clarity if patient's current condition was significantly different from baseline (27.0%). Disagreement was more likely in patients who were chronically ventilated (OR, 4.66; 95% CI, 2.16-10.08; P <.001), had a primary cardiac admission diagnosis (OR, 3.36; 95% CI, 1.18-9.53; P =.02), or underwent cardiothoracic surgery during the admission (OR, 4.90; 95% CI, 1.60-15.00; P =.005). Reviewers were at least moderately confident in their decision 73% of the time; however, they were less likely to be confident if the patient had cardiac disease or chronic respiratory failure. Interpretation: The interrater reliability of the 2015 PALICC criteria for diagnosing moderate to severe PARDS in this cohort was substantial, with diagnostic disagreements commonly caused by differences in chest radiograph interpretations. Patients with cardiac disease or chronic respiratory failure were more vulnerable to diagnostic disagreements. More guidance is needed on interpreting chest radiographs and diagnosing PARDS in these subgroups.