How Does Human-Centred Extended Reality Support Healthcare Students’ Learning in Clinical Conditions?

It is critical to promptly address issues contributing to the attrition of healthcare students and staff members, as we cannot afford to lose additional health professionals. WHO [1] and UN [2] predict international shortages will increase to 9.9 million healthcare workers by 2030. Higher education in healthcare enhances professionals’ competence, which can positively impact staff retention and the overall quality of healthcare services [3]. For example, according to Directive 2013/55/EU [4], total nurses’ education includes up to 50% of clinical practice, which is dysfunctional and ineffective because of the lack of healthcare staff to supervise students [5, 6]. As students cannot access necessary training because of a lack of healthcare supervisors at clinical placements, the pipeline of qualified healthcare professionals is compromised, exacerbating the broader healthcare workforce shortage. Addressing these challenges requires a comprehensive approach that involves increasing the healthcare workforce, enhancing support for educators and mentors, and finding innovative solutions to provide students with the necessary clinical experiences.

Healthcare education needs to be reformed to sustain quality, faster response to crises and ensure a rapid and efficient graduation path for future healthcare professionals. According to previous evidence, simulation-based extended reality (XR) education can partly replace healthcare clinical practice by supporting students’ learning experiences in safe learning environments [7, 8]. Hybrid intelligence (including extended reality (XR) and artificial intelligence (AI)) is seen to be ‘an effective way to augment traditional forms of pedagogy’ [9] and offer new pedagogical methods for training opportunities.

Human-centred XR will empower end users and continuously involve healthcare students and educators in co-design and decision-making by aiming to respond to the objectives. This will be achieved by developing a) an intuitive extended reality (XR) virtual simulation environment with realistic scenarios, b) user experience analysis and real-time adaptation through a combination of neurophysiological and behavioural data collected by wearable sensors, and c) the creation of convincing digital humans (patients and mentors).

RQ1.: Does a human-centred XR support healthcare students’ learning in clinical conditions?

RQ2.: How do students’ cognitive and neural responses, as measured by fNIRS and EEG, differ during learning tasks in XR simulations compared to traditional learning?

H1: Human-centred XR will support students’ learning in clinical conditions statistically significantly (p ≤ 0.05).

Two human-centred XR cases have been developed by integrating self-regulated learning [10], immersive technology principles [11] and interprofessional mentor competence framework [12]. The cases contained scenarios with an XR environment: a) discharging a patient by providing instructions and a home care plan, b) assessing a patient in anaphylactic reaction after administering intervenors antibiotics. In the XR technology, we have used game engine integration, e.g., rendering the full-body avatar of patients and nurses, implementing the single-user interaction design, integrating physical props, and the graphical content and scenario-specific sequences to comprise the whole simulation. The cases contained natural communication between real humans and avatar meta-human patients, making interaction seamless (see Picture 1). The development of XR technologies involved user-centred design and co-design by involving students, educators, and health professionals from the beginning until the testing phase. Expert panels have been used to assess the feasibility of the intervention, using design-thinking workshops to understand human needs, re-frame problems, and create ideas through brainstorming sessions and iterative low-cost rapid prototyping (Fig. 1).

In total, 30 subjects participated in the pilot study representing two universities of applied sciences in the Nordic part of Finland. 61% were first-year students, 36%- second year and 3% - third year. Preliminary results have shown that students’ clinical reasoning improved statistically significantly (varying between p = 0.028 to p < 0.001) in all areas but two, focusing on collecting patient information and evaluating patient condition improvement. The highest p-value area reached (p < 0.001) was in identifying and communicating vital information clearly to the doctors based on the patient's current situation. Students evaluated that XR learning has increased their confidence in learning in all areas, mostly reporting enjoying how the instructor taught the simulation (mean 4.5, SD 0.73) and teaching methods used in the simulation were helpful and effective (mean 4.4., SD 0.62). Students evaluated simulation pedagogical design as having the highest instructor’s support in their learning (mean 4.83, SD 0.46) and a clear understanding of the purpose and objectives of their learning (mean 4.50, SD 0.68). Students evaluated the lowest in the simulation, allowing them to analyse their behaviour and actions (mean 3.87, SD 0.86) and the opportunity to reflect with teachers to build their knowledge to another level (mean 3.93, SD 0.69).

Spectral entropy measures the complexity of the signal in the frequency domain; the higher the value, the more complex the signal. Figures 2, 3, and 4 show the response of oxy-, deoxy-, and total haemoglobin also with cerebrospinal fluid (CSF) in three different bands, very low frequency (0.008–0.1 Hz), respiratory (0.1–0.6 Hz), and cardiac (0.6–5 Hz) bands. P1 represents phase 1 and acts as a baseline for the measurement. P2 and P3 indicate reading and XR tasks. As the subjects only had a little knowledge of Finnish, they looked like they were struggling, and the spectral entropy scores during P2 were relatively high. Although the XR task seemed quite demanding in specific scenarios, the overall experience did not induce complexity in the signal.

In our study, we aimed to develop an intuitive XR virtual simulation environment with realistic scenarios and metahuman avatars, enabling team interaction to test and analyse participants’ real-time adaptation through a combination of neuro-physiological and behavioural data collected by wearable sensors. Until nowadays, formal healthcare training has focused predominantly on developing discipline-based knowledge, clinical expertise, and technical skills in non-adaptive designs [19]. Contemporary healthcare education needs to develop further solutions to enhance students’ and future professionals’ team-oriented competencies, such as non-technical skills, defined as situation awareness, decision-making, communication and teamwork.

Our study has shown that a human-centred XR learning environment already in short usage increased students’ clinical reasoning statistically significantly. Students also evaluated XR and simulation design to support their learning positively by increasing their confidence in competence development. In this paper, we present preliminary data analysis prior to the synchronisation of the data into multimodal data analysis, which will be further used to develop adaptive learning to support students’ self-regulatory learning mechanisms to master their clinical competencies with the support of XR. By developing human-centred XR with adaptive learning and mentoring, the new knowledge can bring breakthrough evidence of novel learning/teaching methods to enhance clinical training in healthcare. Students’ clinical competence can be developed by complementing existing teaching methods, reducing the number of training hours with actual patients while increasing patient safety. The human-centred XR will allow students to practice clinical skills alone and/or with peers from a distance and in different locations, widening opportunities for clinical and interprofessional training in healthcare education.