Producing effective and achievable safety strategies from adverse event investigations in healthcare

It is estimated that one in ten patients will experience an ‘adverse event’; that is
something going wrong in the way in which care is delivered. These events are often
investigated and recommendations made, with the intention of preventing recurrence or
improving safety. There are, however, increasing concerns that these recommendations
and preceding investigations are not contributing to improved safety, and potentially
contributing to safety clutter. The aim of this PhD was to explore how the generation of
recommendations might be improved. Three studies were undertaken: 1) a scoping
review; 2) an experimental scenario study; and 3) a modified Delphi study. Study 1
(scoping review) highlighted that recommendations tended to focus on individuals’
behaviour rather than latent system deficiencies, with a lack of agreement about how
recommendations should be judged for effectiveness. These two findings led onto the
subsequent studies.

Firstly, given the scoping review findings that investigation recommendations seem to
‘blame’ the actions of individuals, and focus improvement efforts on changing their
behaviour, the possibility that cognitive biases of those involved in investigations may
play a role in this tendency was explored. Study 2 was an experimental scenario study,
designed to examine the impact of outcome bias on judgements of staff responsibility,
incident avoidability, importance of investigating and recommendation selection.
Outcome bias occurs when the ultimate outcome of a past event is given excessive
weight, in comparison to other information, when judging the preceding actions or
decisions. The results of this study indicated that outcome bias had significant impact
of judgement and responses when investigating incidents, with higher ratings of staff
responsibility, importance of investigating and higher likelihood of punitive
recommendations when patients came to greater harm. While expertise in safety
reduced this impact it did not entirely eliminate it.
Secondly, Study 1 findings suggested difficulties in judging recommendations’ quality
or effectiveness, and that there was no consistent approach in the literature. Before
attempting to improve recommendations, it is first necessary to define what a ‘good
recommendation’ is. Study 3 was a modified Delphi study that aimed to achieve consensus on what ‘good’ looks like in investigation and recommendation generation.
As recommendations are closely linked to the findings and activities of the
investigation, it was decided to attempt to gain consensus on criteria to judge both the
quality of an investigation and recommendations. Ninety-two evidenced-based criteria
were drafted with the help of an expert steering group. Following three rounds of the
Delphi process, consensus was achieved for 92 criteria, which were then ranked by
their ratings and level of expert agreement. Further work is needed to understand how
these criteria could be used to judge and improve the quality of investigations and
recommendations.
Taken together, this evidence suggests that the generation of recommendations is a
complex, and that the current evidence does not sufficiently describe how this
important work is achieved in everyday healthcare practice. What is evident is that
despite the increasing awareness of systems factors in the incidence of adverse events,
outcome bias is a significant influence on the generation of recommendations and the
assignment of responsibility. With the potentially far-reaching impact of cognitive
biases on investigations and recommendation generation, further work is needed to
examine the impact as well as mitigation strategies. The generation of
recommendations is further complicated by the lack of guidance about what best
practice might be in the investigation and recommendation generation process. Indeed,
cognitive bias identification and mitigation is but one of the criteria formulated from the
modified Delphi study. These criteria will be useful in measuring the effectiveness of
investigations and recommendations within future research but also for front-line
teams in patient safety. However, further research will be needed to understand how
these criteria can be operationalised for systematic application by healthcare staff, to
improve their processes for learning from patient safety events.