A Randomised Simulation Study to Assess the Impact of a Focussed Crew Resource Management Course on UK Medical Students

Authors: S. MacDougall-Davis, S. Wysling, R. Willmore

Abstract:

Background: The application of good non-technical skills, also known as crew resource management (CRM), is central to the delivery of safe, effective healthcare. The authors have been running remote trauma courses for over 10 years, primarily focussing on developing participants’ CRM in time-critical, high-stress clinical situations. The course has undergone an iterative process over the past 10 years. We employ a number of experiential learning techniques for improving CRM, including small group workshops, military command tasks, high fidelity simulations with reflective debriefs, and a ‘flipped classroom’, where participants are asked to create their own simulations and assess and debrief their colleagues’ CRM. We created a randomised simulation study to assess the impact of our course on UK medical students’ CRM, both at an individual and a teams level. Methods: Sixteen students took part. Four clinical scenarios were devised, designed to be of similar urgency and complexity. Professional moulage effects and experienced clinical actors were used to increase fidelity and to further simulate high-stress environments. Participants were block randomised into teams of 4; each team was randomly assigned to one pre-course simulation. They then underwent our 5 day remote trauma CRM course. Post-course, students were re-randomised into four new teams; each was randomly assigned to a post-course simulation. All simulations were videoed. The footage was reviewed by two independent CRM-trained assessors, who were blinded to the before/after the status of the simulations. Assessors used the internationally validated team emergency assessment measure (TEAM) to evaluate key areas of team performance, as well as a global outcome rating. Prior to the study, assessors had scored two unrelated scenarios using the same assessment tool, demonstrating 89% concordance. Participants also completed pre- and post-course questionnaires. Likert scales were used to rate individuals’ perceived NTS ability and their confidence to work in a team in time-critical, high-stress situations. Results: Following participation in the course, a significant improvement in CRM was observed in all areas of team performance. Furthermore, the global outcome rating for team performance was markedly improved (40-70%; mean 55%), thus demonstrating an impact at Level 4 of Kirkpatrick’s hierarchy. At an individual level, participants’ self-perceived CRM improved markedly after the course (35-70% absolute improvement; mean 55%), as did their confidence to work in a team in high-stress situations. Conclusion: Our study demonstrates that with a short, cost-effective course, using easily reproducible teaching sessions, it is possible to significantly improve participants’ CRM skills, both at an individual and, perhaps more importantly, at a teams level. The successful functioning of multi-disciplinary teams is vital in a healthcare setting, particularly in high-stress, time-critical situations. Good CRM is of paramount importance in these scenarios. The authors believe that these concepts should be introduced from the earliest stages of medical education, thus promoting a culture of effective CRM and embedding an early appreciation of the importance of these skills in enabling safe and effective healthcare.

Keywords: crew resource management, non-technical skills, training, simulation