Introduction

Attention to patient safety has intensified considerably in recent years with increasing recognition that medical errors are common events. In ICUs, the frequencies of selected medical errors have been estimated at 498–568/1,000 patient-days [1, 2]. This demands that we look for means of preventing errors or mitigating their consequences [3]. Flaws in system design and human factors are the main sources of medical errors [4]. Consequently, both the culture that underlies the system and work-related effects on staff well-being must be investigated to identify means of diminishing medical errors.

Burnout is a state of psychological distress related to chronic stress. Among healthcare professionals, burnout related to work-related stress is a combination of emotional exhaustion, depersonalisation and a diminished perception of personal accomplishment. Depression is a combination of depressed mood, feelings of guilt and worthlessness, feelings of helplessness and hopelessness, loss of appetite and sleep deprivation. Burnout has been identified in all categories of healthcare professionals, including ICU staff members. In France, burnout was present in 50 and 33 % of physicians and nurses, respectively [5, 6] and symptoms of depression in 23 % of physicians [7]. Studies of potential links between burnout, symptoms of depression and the risk of medical errors [8–10] produced conflicting results outside the ICU setting. No proof that burnout or depression increases the risk of medical errors in the ICU has been published to date.

The safety of ICU patients is more strongly dependent on the effectiveness of the healthcare team than on the expertise of a specific individual [11]. The terms “safety culture” and “safety climate” are often used interchangeably. However, the “safety climate” is a component of the safety culture consisting in staff attitudes about patient safety within the organisation. The “safety culture” is a broader concept that encompasses the thinking about patient safety within an organisation and the actions, structures and processes introduced to maximise patient safety. Thus, the safety culture is beliefs, perceptions and values shared by staff members and associated with habitual behaviours that decrease the risk of medical errors. The Manchester Patient Safety Framework is a set of specific actions designed to raise awareness among staff members of their responsibility in ensuring patient safety, so that patient safety becomes an integral part of all acts performed by all staff members [12]. The impact of developing a safety culture on the risk of medical errors in the ICU is unclear.

The primary objective of this study was to assess potential associations linking staff symptoms of depression, burnout and the strength of the safety culture to the risk of medical errors in the ICU. The secondary objective of the study was to evaluate patients and centre risk factors. To reach this objective, we conducted a prospective, observational, multicentre study in French ICUs.

Materials and methods

Study design

We conducted a prospective, observational, multicentre study. For feasibility reasons, the participating ICUs were located in two areas in France: Ile de France (the Paris area) and the South-East, where the two main investigators (MGO and JFT, respectively) work and could therefore supervise the centres closely. The 38 ICUs in these two areas that had participated in our previous study [2] were invited to participate. All were closed ICUs, each with a formal head. The physicians and nurses held formal handover meetings twice a day to ensure continuity of care. Daily rounds were performed in all centres.

As the level of staff burnout can interfere with the self-reporting of medical errors [13], we hired two clinical research assistants who did not belong to the ICU teams, each of whom collected the study data in one of the two study locations. Before study initiation, each assistant was trained in data collection by two of us (JFT and MGO). Each assistant worked in the ICU for 8 h a day 5 days a week, for 2 weeks per ICU, collecting data in real time and reviewing patient charts for data recorded at night or over the weekend. A staff physician and a staff nurse in each ICU provided help as needed. Each ICU chose a 2-week period without holidays to ensure that as many staff members as possible would be present. Consecutive ICU patients participated in the study from ICU day 1 at 8.00 am to ICU day 15 at midnight. For patients with multiple stays during the 15-day period, all stays were included. At study completion, medical errors and adverse events were validated by the physician and nurse in charge of the study in each centre. The data were not validated externally.

Definition of medical errors and adverse events

Medical errors were failure of a planned action to be completed as intended (i.e. error of execution) or use of a wrong plan to achieve an aim (i.e. error of planning). Adverse events were patient harms caused by medical interventions [14]. The procedure used to select medical errors has been described previously and used in a multicentre study [2]. Briefly, 60 experts selected medical errors using a Delphi technique. To facilitate the process, a definition of each medical error was provided. Harm was defined using specific scales and severity using pre-established lists. This selection procedure is described in detail in the electronic supplementary material and the definitions of selected medical errors are reported in Table E1.

Data collection

The characteristics of the study ICUs and patients are reported in the electronic supplementary material. Each ICU staff member (physicians, nurses, nursing assistants and physiotherapists) present during the 2-week data collection period was asked to complete an anonymous three-section questionnaire handed to them by the clinical research assistant for completion on day 1. The first page explained the study objective. The first section collected the data related to ICU, patient and staff characteristics listed in Tables 1, 2 and 3.

Table 1 Characteristics of the 31 study ICUs
Full size table
Table 2 Characteristics of the 766 patients
Full size table
Table 3 Characteristics of the 1,534 ICU staff members
Full size table

Burnout and symptoms of depression were evaluated in the second section. The Maslach Burnout Inventory is a 22-item questionnaire considered to be a standard validated tool for measuring burnout [15]. We used the Fontaine French version [16]. Burnout is a complex and evolutionary state with symptoms related to emotional exhaustion (nine questions), depersonalisation (five questions) and personal accomplishment (eight questions). Items are scored on a 7-point frequency scale from never (0) to always (6). The complexity of burnout leads to difficulties in defining the syndrome. Studies have used either a global score or the sub-scores on two or three dimensions [5, 6, 10, 17–20]. Importantly, in this study, we used two definitions to define burnout. The first definition (definition 1 in Table 4) was the combination of high exhaustion and depersonalisation scores with a low personal accomplishment score [16, 21]. The second definition (definition 2 in Table 4) was a global MBI score greater than −9, as previously used in the ICU [5, 6]. To score each dimension of burnout, we used the following cut-offs: low, moderate and high exhaustion, at most 17, 18–29 and at least 30, respectively; low, moderate and high depersonalisation, at most 5, 6–11 and at least 12, respectively; and low, moderate and high personal accomplishment, at least 40, 34–39 and at most 33, respectively [16].

Table 4 Multivariate analysis to identify factors associated with medical errors using definition 1 of burnout (combination of high exhaustion, high depersonalization and low personal accomplishment)
Full size table

Symptoms of depression were assessed using the Center of Epidemiologic Studies Depression scale (CES-D), a questionnaire of 20 items scored on a 4-point frequency scale from never (0) to always (3), so that the total score can range from 0 to 60, with higher scores indicating greater severity of depression symptoms. The study validating the French version of the questionnaire showed that scores of at least 17 for men and of at least 23 for women indicated symptoms of depression [22, 23].

The safety culture was evaluated in the third section with the Safety Attitudes Questionnaire-ICU version (SAQ-ICU), specifically designed to measure the safety culture at both the individual and the group levels in ICUs. This scale uses 63 items in six domains: teamwork climate (perceived quality of collaboration between personnel), job satisfaction (positivity about work experience), perception of management (approval of managerial decisions), safety climate (perception of a strong and proactive organisational commitment to safety), working conditions (perceived quality of the work environment and logistical support) and stress recognition (acknowledgement of how work conditions are influenced by stressors) [24]. Among the numerous available tools for measuring the safety culture [25], we chose the SAQ because it is relatively short, easy to complete and, compared to other tools, both better validated and better suited to use in ICUs [25]. Each question is scored on a 5-point scale (disagree strongly, 0; disagree slightly, 25; neutral, 50; agree slightly, 75; and agree strongly, 100). Negatively worded items are scored in reverse. These individual responses are aggregated to provide a snapshot of the ICU culture. We defined the existence of an effective safety culture as an SAQ-ICU score of at least 75 [23, 26].

Statistical analysis

The primary objective was to assess the potential impact of burnout and depression on the risk of medical errors and adverse events in ICU patients, taking into account potential patient-based and centre-based confounding factors. The data are described as median [interquartile range] or number (%), separately for physicians and other staff. Questionnaires with more than 25 % of missing items were excluded [27]. When less than 25 % of items were missing, we imputed each missing value to the median for the MBI and CES-D scores and to the mode for the SAQ-ICU score, as previously recommended [27].

To identify factors associated with the number of medical errors per patient, we performed a univariate analysis using a hierarchical negative binomial model with a centre random effect. The staff or centre variables were at the ICU level; for each, the median or percentage was calculated. Medical errors were at the patient level. Clinically relevant factors with P < 0.25 were entered into a multivariate hierarchical model. Backward selection was performed until all variables yielded P values less than 0.05. Quantitative variables that were not linear in the logit were transformed into dummy variables using either the median value (symptoms of depression and burnout) or the inflection point of the cubic spline functions [Nine Equivalents of Manpower Score (NEMS) and staff off work on the previous day].

Finally, SAQ-ICU, exhaustion, depersonalisation and personal accomplishment scores were entered into the model to assess their effect after adjustment on the other parameters. The same analysis strategy was used for the number of adverse events per patient and the number of medical errors excluding insulin administration errors. All univariate and multivariate analyses were routinely adjusted for patient duration of exposure. All statistical analyses were performed using SAS 9.3 software (SAS Institute Inc., Cary, NC).

Results

Characteristics of ICUs, patients and staff

Of the 38 ICUs, 31 (81.6 %) participated in the study, from August 2009 to December 2011. Tables 1, 2 and 3 report the characteristics of the ICUs, patients (n = 766, 4,312 patient-days) and staff (n = 1,534). Table E2 reports the characteristics of the eight ICUs that did not participate in the study.

Safety culture results

For the SAQ-ICU, 1,192/1,534 (78 %) questionnaires had all SAQ-ICU items completed; of the remaining 342 questionnaires, 313 had 588/19,719 (2.3 %) missing items and 29 (2 %) had more than 25 % of missing items. The median SAQ-ICU score was 60.7 [56.8–64.7] for physicians and 57.5 [52.4–61.9] for other staff. Figure E1 reports the sub-scores in the six safety culture domains, separately for physicians and other staff. Scores were lowest for approval with management decisions. Online Figure E2 shows SAQ-ICU score variations across centres. Online Table E3 compares respondents with less or more than 25 % of missing items in the SAQ-ICU questionnaire. Being divorced or widowed, religiosity, low depersonalisation and low exhaustion were associated with characteristics of participants having more than 25 % of missing SAQ-ICU items.

Medical errors

Of 3,469 collected medical errors, 722 (20.8 %) were classified as adverse events. Frequencies of medical errors and adverse events were 804.5/1,000 patient-days and 167.4/1,000 patient-days, respectively. The most common medical errors were insulin administration errors, with a frequency of 817.4/1,000 days of insulin treatment and a frequency of related adverse events of 146.2/1,000 days. Online Table E4 reports the frequency of medical errors according to number of days in the study and to associated procedures or treatments.

Prevalence of burnout and symptoms of depression

The participation rate was 330/401 (82 %) for physicians and 1,204/1,587 (76 %) for other staff. For the MBI, 1,282/1,534 (84 %) questionnaires had all items completed and of the remaining 252 questionnaires, 213 had 334/4,684 (7.1 %) missing items and 39 (2.5 %) had more than 25 % of missing items. A high level of burnout according to definition 1 was found in 2.5 % of physicians (n = 8) and 3.7 % (n = 43) of other staff members. With definition 2, the corresponding prevalence was 40.3 % (n = 133/330 physicians) and 39.8 % (479/1,204 other staff).

Among physicians, 35/330 (10.6 %) had high emotional exhaustion (score ≥30), 81 (24.5 %) high depersonalisation (≥12) and 104 (31.5 %) low personal accomplishment (≥40); corresponding results in other staff were 161/1,204 (13.4 %), 190 (15.8 %) and 381 (31.6), respectively. Figure E3 reports the variations across centres in percentages of staff members with high emotional exhaustion, high depersonalisation and low personal accomplishment. The proportion of respondents who wanted to leave their job was 41.3 % overall, 34.8 % (115/330) among physicians and 43.1 % (519/1,204) among other staff.

For the CES-D, 1,333/1,534 questionnaires (87 %) had all items completed; among the remaining 201 questionnaires, 186 had 236/3,720 (6.34 %) missing items and 15 (1 %) had more than 25 % of missing items. CES-D scores indicated symptoms of depression in 62/330 (18.8 %) physicians and 188/1,204 (15.6 %) other staff. The MBI burnout score correlated significantly with the CES-D score (correlation coefficient, 0.60; P < 0.0001). The MBI burnout and CES-D scores did not correlate significantly with the SAQ-ICU score (correlation coefficients, −0.28 and −0.23, respectively).

Online Tables E5 and E6 report the characteristics that differed significantly in participants with more than 25 % of missing items for the MBI questionnaire (older age and being divorced or widowed) and CES-D questionnaire (older age and low depersonalisation), compared to the other participants.

Relations linking ICU, patient and staff characteristics to medical errors

Online Tables E7, E8 and E9 show the results of the univariate analysis of ICU-, patient- and staff-related risk factors for medical errors and adverse events. The independent risk factors identified by multivariable analysis are listed in Tables 4 and 5 for medical errors and in online Tables E10 and E11 for adverse events, for each of the two definitions of burnout. In the analyses of medical errors, symptoms of depression were significant independent risk factors, whereas symptoms of burnout were not. The medical SAQ-ICU score had a protective effect in only one of the four analyses (Table 4). The other ICU-related factors independently associated with medical errors or adverse events were more than 40 % of the staff off work on the previous day and training in safety programs. One-on-one training delivered to junior physicians during duties and presence of a risk-management unit decreased the risk of medical errors or adverse events. The results were not modified when each burnout sub-domain was forced into a supplementary multivariate analysis (data not shown) or when no imputation for missing items was performed (Online Tables E12, E13, E14 and E15).

Table 5 Multivariate analysis to identify factors associated with medical errors using definition 2 of burnout (Maslach Burnout Inventory score greater than −9)
Full size table

Discussion

Our prospective observational study in 31 ICUs in France found a high frequency of selected medical errors, most of which caused no harm to the patients. None of the ICUs had an effective SAQ-ICU score. Burnout was found in 3 or 40 % of healthcare workers, depending on the definition used (presence of all three MBI subdomains or MBI score greater than −9). Symptoms of depression were an independent factor for medical errors but burnout was not associated with medical errors. The safety culture score had only a limited influence on medical errors. Other independent risk factors for medical errors or adverse events were related to the ICU organisation (40 % of the ICU staff off work on the previous day), staff (having received specific training in safety programs) and patients (comorbidities, workload, insulin therapy or urinary catheter within 48 h after ICU admission, and inadequate staff knowledge of and/or compliance with the ventilator weaning procedure). Only one-on-one training of junior physicians during duties and presence of a hospital risk-management unit were associated with a lower risk of medical errors and adverse events, respectively.

In a previous study, high emotional exhaustion, severe job dissatisfaction and nurse understaffing were associated with mortality and other adverse events in surgical patients [28]. To our knowledge, our study provides the first data on potential links connecting staff well-being, safety culture as assessed using the SAQ-ICU, and patient safety in the ICU. Extensive studies of burnout syndrome done in the past few years found prevalences of 29–50 % [5, 6, 20, 29]. The large difference in the prevalence of burnout syndrome found in this study according to the definition used indicates a need for efforts to develop a definition of clinically meaningful burnout, as well as different approaches for evaluating burnout. It illustrates the difficulties raised by establishing definitions and scoring tools for complex states [30].

A reasonable hypothesis is that healthcare staff who are burned out, depressed and/or anxious are unable to engage fully in patient care and safety. Among 7,905 surgeons, 8 % committed major medical errors due to lapses in judgment, and both burnout and depression were independent risk factors for major medical errors after adjustment for personal and professional factors [10]. However, this study did not determine whether distress caused the errors or vice versa. In our study, medical errors were not significantly associated with the components of burnout syndrome, with either of the two very different prevalences yielded by the two definitions of burnout. In contrast, ICUs where more than 15 % of the staff members reported symptoms of depression had higher frequencies of medical errors. Similarly, among paediatric residents, depression and burnout were both common (20 and 74 %, respectively), and depression increased the risk of medical errors 6.2-fold, whereas burnout had no effect [17]. The relationship between burnout and depression is controversial. The timing differs between the two syndromes. Depression involves the rapid loss of adaptive mechanisms associated with personal suffering, whereas burnout develops only after several years of chronic professional stress [16]. Burnout and depression may be present concomitantly, although the dehumanisation component of burnout may protect against depression [31]. Further work is clearly needed to clarify the links between burnout, depression and medical errors. In our study, 41 % of staff members overall wanted to leave their job, in keeping with other studies [5, 6]. However, we did not obtain follow-up data to determine how many actually changed jobs. The young age of our participants is consistent with earlier data on ICU staff in France. We cannot determine whether this factor influenced our results.

The importance of a strong safety culture in healthcare institutions has been emphasised for over a decade, but potential effects of a safety culture on medical errors have not been examined in depth [31, 32]. In an open randomised controlled study in 60 primary-care group practices, implementing an adapted version of the Manchester Patient Safety Framework had no impact on error management or on safety culture and climate indicators but improved both the frequency and the quality of medical error reports [32]. In 23 ICUs, the development of action plans based on baseline SAQ scores did not significantly improve SAQ scores or the frequency of healthcare-associated infections [33]. A study in 57 ICUs showed that a better safety climate was associated with only a modest decrease in medication errors (odds ratio 0.98, 95 % confidence interval 0.96–0.99) [1]. Safety climate in this study [1] was assessed using 53 items from the Vienna Safety Climate Questionnaire (VSCQ). Our study found a weak link between safety culture in physicians and the risk of medical errors. SAQ-ICU scores were lowest for approval of management decisions, which has been considered the most crucial component of the safety culture since the formulation of this concept [34], and they were associated with impact of hospital mortality [35]. In keeping with our results, a review of 21 studies showed that strategies designed to improve the safety culture had little impact on the safety climate [36]. Several factors may have contributed to the absence of a strong link between safety culture scores and patient safety in our study. The safety culture scores were low overall in our study and varied little across centres. Furthermore, the higher reporting rates of medical errors in ICUs whose staff members were trained in safety programs complicated the analysis of the results and may have led to underestimation of the impact of the safety culture. Although ‘culture’ has been defined in many ways over time and across fields of activity, the relevant point for patient safety is culture as a set of values that determine how people act. In contrast to other visions of culture, the safety culture in healthcare departments should be explicit (i.e. known consciously by the staff) and should be voluntarily induced by the department leaders. Leaders must create the conviction in the minds of staff members that patient safety is a priority. The difficulty in attaining these goals has led some authors to suggest a need for studies involving experts in the field of culture, such as anthropologists; or for the use of ethnographic qualitative methodologies, when conducting research on the safety culture [37].

Our results add important information on the effect of ICU organisational factors. Higher workload increased the risk of medical errors [38–40]. We found that medical errors were more common on days when at least 40 % of staff members were back at work after a day off. This result may be related to poorer knowledge about the patients admitted during an absence from the unit. For junior physicians, one-on-one education by a senior physician during night and weekend duties was associated with fewer medical errors during workdays, and supervision was crucial to improving human performance [41]. The associations linking training in safety programs to a higher risk of medical errors and presence of a risk-management unit to a lower risk are not conflicting. The association of training should not be interpreted as negative: instead, it indicates both a greater individual tendency to disclose events without fear of punishment and the existence, in hospitals that provide training, of an atmosphere of trust and understanding that promotes disclosure of unwanted events.

Strengths of our study include the high response rates and data collection by clinical research assistants to improve reporting rates. This methodology explains the higher rate of medical errors in our study. The use of two definitions of burnout supports the robustness of our finding that medical errors were not significantly associated with burnout. Several methodological limitations of the study should be acknowledged. First, the 31 ICUs were not selected at random and had previous experience with studying the reporting of medical errors. A Hawthorne effect related to the presence of an external observer very probably affected our results, as demonstrated in a previous study by our group [2]. Second, aggregation of all data at the ICU level decreased the power of the analysis. Third, we used only the quantitative SAQ-ICU to assess the safety culture. Quantitative data may need to be supplemented with qualitative data obtained during interviews or focus groups to gain a better understanding of the underpinnings of the safety culture [37].

In conclusion, our study highlights the complexity of assessing the risk factors for medical errors and therefore the challenges raised by developing improvement strategies. Our results support routine screening of ICU staff members for symptoms of depression, careful planning of nurse staffing according to level of knowledge of admitted patients, individual education and training of junior physicians during night and weekend duties, development of ICU safety programs and close attention to patients who generate a high workload. Our finding that the safety culture as assessed using the SAQ-ICU had a small but significant influence on the risks of medical errors may indicate that a better tool is needed to assess the safety culture.