Introduction

The mortality rate in the pediatric intensive care unit (PICU) in economically developed countries has decreased in the last decades to approximately 3% [26]. Moreover, a substantial part of the PICU population (55% in a recent study) has a mortality risk of < 1% [35]. Although these are low-risk patients, some of these patients die on the PICU. Patient factors like complex chronic conditions (CCCs) do not explain all deceased patients in this patient group [6, 35]. For quality purposes, it is interesting to analyze whether adverse events (AEs) or even medical errors play a role in the death of low-risk PICU patients [3, 16]. An AE is an unintended injury that results in temporary or permanent disability, death, or prolonged hospital stay and that is caused by health care management rather than by the patient’s underlying disease process [38]. A national project on preventable deaths in Dutch hospitals showed that preventable AEs contributed to 4.1% of hospital deaths [38, 39]. In most international AE studies, (young) children were excluded or the number of included PICU admissions was not specified or very low, so data about PICU patients are scarce [2,3,4, 18, 38].

Because of their vulnerability, intensive care patients are more prone to iatrogenic events [10, 12, 13]. The incidence of AEs in the PICU population depends on the method used to detect AEs [1, 17, 19, 22, 28, 31, 33, 36]. Studies using a trigger tool method show that 59–76% of all PICU patients encounter at least one AE during their stay [1, 17, 36].

Although one could speculate that AE incidence is higher in the more complex and sicker patients needing extensive support (high-risk patients), AEs also occur in the less severely ill PICU patients [1, 17, 22]. To our knowledge, no studies have focused on the occurrence of AEs in low-risk PICU patients. The incidence of AEs among low-risk patients might be underestimated when only the general PICU population is examined. Analyzing medical records from non-survivors with a low risk of dying is an efficient tool to discover problems in the quality of care [14]. If low-risk PICU patients deteriorate or die because of preventable AEs, there is a potential for improving their outcome.

The aim of this exploratory study was to study the occurrence of AEs in the low-risk non-survivors (LNs), compared to low-risk survivors (LSs), high-risk non-survivors (HNs), and high-risk survivors (HSs) in two PICUs. Of all AEs, we studied the severity, preventability, and nature. The study was designed as a retrospective exploratory study that used chart review to examine the feasibility of detecting AEs in this patient group.

Methods

Study design and setting

This is a retrospective patient record study to measure the occurrence of AEs in low-risk non-survivors and to compare the results with patients with a different risk profile and different outcomes, using a random stratified sample of 48 records. The study was performed in two PICUs. Data collection was performed in 2015.

Admission selection

Admissions in each PICU between 1 January 2006 and 1 January 2012 were stratified into four groups with different risk profiles and different outcomes. The study group consisted of LNs. Three control groups were chosen: LSs, HNs, and HSs. Low-risk admissions were defined as admissions with a mortality risk in the simply recalibrated Pediatric Index of Mortality (PIM) 2 score and/or recalibrated Pediatric Risk of Mortality II score (further referred as “PRISM”) of < 1% [24, 25, 27, 29, 35]. High-risk admissions were defined as admissions with a mortality risk in the simply recalibrated PIM2 and/or PRISM of ≥ 30% [35].

Other inclusion criteria were the following: age < 18 years and PICU length of stay of at least 2 h. Exclusion criteria were the following: patients already deceased before admission (for example, brain dead patients, admitted for organ donation), corrected age < 36 weeks (gestational age), invalid or impossible PIM2/PRISM score, and no clinical data available.

The mortality risk scores and PICU outcome data were provided by the national PICU registry (Pediatric Intensive Care Evaluation (PICE) registry) [23]. The PICE registry is a national database containing anonymized information of admission characteristics, severity of illness, and patient outcome. Data quality is assessed using standard procedures including audit site visits. Of all patients, both PIM2 and PRISM scores are collected. The models were recalibrated for the study period to predict the overall mortality in the total population in this period without altering the relative weights of risk factors in the models and thus retaining the discriminative power of the models [35, 37]. A local copy from the PICE registry was sent to the local PICUs including all admissions between 2006 and 2012. The database of these two PICUs (total of 11,216 admissions: PICU-1, 8438 admissions; PICU-2, 2778 admissions) contained 39 LNs.

Since the study was designed as an exploratory study, a selection of roughly one third of the LN was used for the study. Twelve LNs were selected for the study. Because the number of patients between the two participating centers was unequal, nine admissions from PICU-1 and three admissions from PICU-2 were selected for each study group, using a computer-based research randomizer [34]. To avoid different population characteristics, the patients in the control groups (LS (n = 12), HN (n = 12), HS (n = 12)) were stratified based on PICU center, gender, and age category. After stratification, the patients were randomly chosen using the computer-based research randomizer.

To verify if the risk profile of patients was correct, the PIM2 and PRISM scores were checked using available physiologic and laboratory data. If a discrepancy was discovered, e.g., the corrected mortality risk turned out to be >2% in LN and LS or < 30% in HN and HS, the patient was excluded from the study. The next from the list of available patients (with the same risk group/outcome/PICU center/gender/age category) was selected until, in each group, 12 patients were included.

Data collection

An established set of triggers was modified to local characteristics of the PICU population and was used in a retrospective chart review to discover AEs (Table 4, online only) [1]. In the first stage, patient charts were manually reviewed for the presence of 19 triggers. In the second stage, each positive trigger was followed by an in-depth investigation for the presence of associated AEs. Both stages were performed by a pediatric intensivist (CV) with more than 15 years of PICU experience who was trained in the use of the trigger tool method.

Primary outcomes were the occurrence of AEs and AE rate (AE/PICU day). For the AE rate, only AEs occurring during the PICU admission were included. AEs that occurred shortly before PICU admission and were beyond doubt related to the PICU admission were scored as “AE pre PICU.” The severity of AEs was rated using the National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP) Index for Categorizing Errors (Table 5, online only) [20]. Preventability of AEs was scored on a 6-point scale (Table 6, online only) [3]. AEs with a preventability score of 4–6 were defined as preventable. A preventable AE results from an error in management due to failure to follow accepted practice at an individual or system level. Accepted practice was taken to be “the current level of expected performance for the average practitioner or system that manages the condition in question” [38]. AEs were grouped into eight categories, based on the classification made by Hogan et al. (Table 7, online only) [15]. If problems were encountered in AE determination and categorizing AEs, a decision was taken after discussion within the research group.

The ANZPIC registry diagnostic code list was used for diagnosis classification [30]. An admission was classified as having a CCC or a non-complex chronic condition (NCCC) if either the primary diagnosis, the primary underlying diagnosis, or the first additional diagnosis was a diagnosis defined as a CCC or NCCC according to a modified Feudtner’s list [5, 7, 8]. PICE diagnoses not appearing on these lists were classified before analyzing the data according to expert opinion (CV, JL) [35]. The list of the PICE database diagnoses grouped as a CCC and NCCC is described in Table 8 and Table 9 (online only).

Socio-economic status of the family was obtained by coupling the four digits of the postal code to the socio-economic status of the neighborhood in 2006 (The Netherlands Institute for Social Research) and grouped into three categories [32].

Data analysis

Normal distribution of continuous variables was tested using sampling distributions and skewness and kurtosis tests. Not normally distributed data were reported by median and inter-quartile range (IQR). Non-parametric tests (Mann-Whitney U) were used for the analyses of not normally distributed data. For categorical variables, Fisher’s exact test or the chi-square test was used (software: IBM SPSS Statistics 22).

Reliability study

To assess the reliability of the record review process, a random sample of nine records (20%) was reviewed by a second investigator.

Results

Respondent characteristics

A total of 48 patients were randomly selected. Nine admissions were excluded, and therefore, nine new admissions were chosen as described (Fig. 1, flowchart).

Fig. 1
figure 1

Flowchart of the study population. LN low-risk non-survivor, LS low-risk survivor, HN high-risk non-survivor, HS high-risk survivor, PIM2 Pediatric Index of Mortality score, PRISM Pediatric Risk of Mortality

Full size image

Patient characteristics are listed in Table 1. The four groups were different on admission characteristics, mortality risk scores, presence of CCCs, and outcome characteristics like length of stay. The LN group had more medical admissions and higher PRISM mortality risk compared to the LS group. The PIM2 mortality risks between LN and LS were comparable. LN patients were more often mechanically ventilated; had more ventilator days, more central venous catheters, and more central venous catheter days; and had a longer length of stay compared to LS patients.

Table 1 Patient characteristics
Full size table

In the LN group, most patients had a CCC (not resulting in a higher PIM2 or PRISM score) in contrast to the HN, where CCCs occurred in a minority of patients. In the HN, cardiopulmonary resuscitation was a frequent reason for admission, often resulting in brain death as the cause of death. In the majority of the LN, patients died after limiting therapeutic options. The length of stay in the LN was much longer compared to the HN and also longer compared to the HS.

Adverse events

The occurrence of AEs in the LN group was significantly higher compared to that in the LS and HN groups (Table 2). Eighty-three percent of the LN patients suffered from at least one AE. Twenty-five AEs occurred in the LN group. The AE rate (AE per PICU day) in the LN group was significantly higher compared to that in the LS and HN groups (median 0.12 AE/PICU day).

Table 2 Adverse events
Full size table

In Table 3, preventability, severity, and classification of all identified AEs are shown. In the LN group, eight preventable AEs occurred. In five of these preventable AEs, the severity was high (grade G-I). Two patients, in both the LN groups, died after a preventable AE. Looking at all 15 preventable AEs found among all subgroups in this study, most preventable AEs were related to problems in clinical monitoring (n = 5), infection control (n = 5), and diagnosis (n = 2). Detailed information about all patients with AEs including description, timing, severity, and preventability of the AEs is shown in Table 10 (online only). The day on which the AE occurred varied from day 0 (preceding the PICU admission) to the last days of the PICU stay.

Table 3 Preventability, severity, and classification of adverse events
Full size table

Inter-observer agreement

Nine patient records were reviewed by the second investigator. Inter-observer agreement was 8/9 (89%).

Discussion

Major findings

In this exploratory study, AEs occurred in 83% of the LN. The occurrence of AEs and AE rate in these LN patients were significantly higher compared to those in LS patients and also higher compared to those in HN patients. A substantial part of the AEs in the LN group was preventable and had severe consequences, including two LN patients who died after a preventable AE. Screening patients with a low mortality risk is a valuable tool to discover problems in the quality of care and might reduce preventable death by implementing targeted quality improvement measures.

A possible explanation for the higher occurrence of AEs in the LN group might be that “low-risk” as defined by a calculated low mortality risk does not always reflect a true low risk of dying. Mortality risk scores such as PIM2 or PRISM scores perform reasonably well for the PICU population in general with an AUC between 0.83 and 0.90 but not for each individual [37]. Many patients in the LN group are sicker than they appear based on the PIM2 or PRISM score. Misclassifications do occur. For example, seven LN patients were admitted to the PICU with major comorbidity such as hemato-oncology patients and patients with complex congenital heart disorder. These low-risk patients with a CCC are often at high risk for AEs [35]. Patients with congenital heart disorders are sometimes admitted preoperatively to the PICU. Mortality risk scores can be obtained before surgery and do not measure true postoperative risk. New PRISM methods like PRISM IV might reflect mortality risk better in these patients because the risk score is measured after surgery [26]. However, severe and preventable AEs did occur in patients with and without a CCC, so to our opinion, this is not the only explanation.

Comparing the AE rate from this study with other studies is difficult because in this exploratory study, we did not include the general PICU population but focused on the low- and high-risk groups. A single PICU study on patient safety factors in 47 PICU non-survivors found that 36% of non-survivors suffered at least one AE of category I and 60% suffered a “critical incident” [19]. These results cannot be compared with our study not only because of different population characteristics but also due to different outcome measures. The “critical incidents” used in the study of Monroe could either be AEs or medical errors not causing harm (categories B–D), a category which is too wide in our opinion [21].

From the viewpoint of quality improvement, preventable AEs are the most interesting. Looking at the nature of the 15 preventable AEs found in this study, problems in clinical monitoring (n = 5), infection control (n = 5), and diagnosis (n = 2) were most prevalent. For example, a pediatric early warning system might lead to timely recognition of deterioration and thus lead to lower mortality [11]. During the study period, pediatric early warning systems and sepsis bundles were implemented in the participating hospitals, but the effectiveness could not be systematically examined yet.

The length of stay in the LN group was significantly longer compared to that in all other groups. A longer duration of stay may be the consequence of the AEs or might have contributed to an increased chance for AEs, and this cannot be estimated from this retrospective study.

Limitations

Our study has several limitations. First, children in the age group of 1–4 years were not present in the randomly chosen LN group and therefore not in the other groups, possibly giving rise to bias. Second, a relatively high number of admissions were excluded from the study. The decision to exclude patients was made on predefined criteria. Remarkably, in seven patients, the PIM2/PRISM score turned out to be false after verifying with the data from the medical record. This should encourage better surveillance of the database. Third, poor quality of the information in patient records might lead to underestimation of the number of AEs. The assessment of AEs with a trigger tool method depends on the presence of data in the medical record. However, in a patient record review study in Dutch hospitals, poor quality of the information present in the medical record was associated with higher rates of AEs [40]. Another weakness of all retrospective studies is hindsight bias [9, 39]. Knowledge of the final outcome may have influenced judgment on severity and preventability. This could lead to an overestimation of preventable severe AEs as judged by the investigators. Finally, the mortality prediction models do not perform perfectly. However, we found that both in real LN and in LN with a CCC, severe AEs and AEs contributing to death occur.

Conclusion

This exploratory study shows that AEs do occur in PICU low-risk non-survivors. The occurrence of AEs in low-risk non-survivors was higher compared to that in low-risk survivors and high-risk non-survivors. Some AEs were severe and preventable and contributed to morbidity and mortality. The exact scale and nature of this safety problem should be analyzed in a larger multi-center study.