Enhancing emergency medical responder training with extended reality

In a mass casualty incident, it is crucial that medical first responders promptly classify victims according to the severity of their injuries. Creating realistic scenarios to train these responders can help better prepare them for such crises.

However, real-life mass casualty simulation drills require an immense amount of planning, manpower, time, and space. Virtual reality (VR) simulation drills can help ease the amount of resources needed compared to a physical drills, but most VR simulations involve point-and-click actions using gaming controllers, or they simply offer users a visual simulation of an incident with no interactivity. These virtual simulations lack the direct and natural tactile interaction that stimulates cognitive and muscle memory.

Targeting the limitations of these conventional training methods, researchers at the Keio-NUS CUTE Center under the NUS Smart Systems Institute, together with the Singapore Armed Forces (SAF), developed the Multi-Sensory Extended Reality (MS-XR) Medical Crisis Management System which offers medical first responders a more realistic triage training experience. The crisis management system has been used in the SAF Medical Training Institute’s (SMTI) medical triaging training module. SMTI has trained their Emergency Medical Technicians trainees using the new training approach and feedback from these trainees has been positive. SMTI plans to use the new training module for all its trainees and graduates across the various SAF units every year.

Linking physical and virtual spaces

At the heart of the system is an MS-XR middleware platform - a software that brings together virtual reality with tangible multi-sensorial hardware interfaces. In the MS-XR Medical Crisis Management System, the middleware links a physical smart manikin with a virtually simulated casualty to create synchronised responses between the physical and virtual elements.

Using the system, trainees are able to perform life-saving interventions and interact with casualties as they would in reality. These include conducting triage assessments such as checking on breath, pulse and mental state, as well as airway repositioning. The interactions are performed on a smart manikin developed with smart sensors and haptic devices to recreate the tangible feeling of handling a casualty. Users also put on proprietary tracker gloves that provide tactile feedback to the virtual simulation without the need for another intermediary controller. The simulation, on the other hand, indicates a casualty’s vital signs, such as pulse.

Trainees are immersed in a realistic situation from the get-go. They are required to move quickly through the virtual scene while physically triaging casualties efficiently. At the end of each simulation exercise, each trainee receives timely assessment with the breakdown of their actions recorded for analysis and debrief.

The middleware is capable of integrating the other senses of smell, taste, heat and airflow into virtual reality simulations, which has the potential to provide even more hyper-realistic training situations. With the established middleware, training content can also be easily expanded to include more casualty types without requiring any hardware changes.

Creating real-world impact

In recognition of this innovative approach in enhancing learning outcomes, the MS-XR Medical Crisis Management System received the Gold Award in the “Virtual/Augmented Reality” category at the Wharton-QS Reimagine Education Awards in December 2021.

“This medical crisis management system builds on Keio-NUS CUTE Center’s existing work on multi-sensory simulation system and assembly. In this collaboration, we worked closely with SMTI to design a training scenario that is highly applicable for their personnel. Given the customisable nature of our MS-XR middleware, we look forward to expanding the practical applications of this innovation,” said Associate Professor Yen Ching-Chiuan, who is the Co-Director of the Keio-NUS CUTE Centre and is also from the NUS College of Design and Engineering.

The Keio-NUS CUTE Center is looking at further collaboration with SMTI to develop systems which provide core medical skills training for individual and team-based competencies. This includes developing MS-XR based training for pre-hospital care scenarios.