DEVELOPMENT AND EFFECTIVENESS OF AUGMENTED REALITY-BASED LEARNING FOR HEALTH SCIENCE STUDENTS: A SYSTEMATIC REVIEW
Lia Artika Sari1, Muhammad Rusdi2, Asrial2, Herlambang2
1 Doctorate student in Education MIPA Jambi University, Indonesia
2 Jambi University, Indonesia
Corresponding author: Lia Artika Sari Jl. Prof DR GA Siwabessy No.42, Buluran Kenali, Kec. Telanaipura, Kota Jambi, Jambi 36122, Indonesia, Tel: +6282196687959, Email: email@example.com, Orcid : https://orcid.org/0000-0002-5285-5356
Cite this article
Background and Objective. The rapid development of technology makes it easier for teachers to continue to be interactively connected with students, for example, by using Augmented Reality technology. We conducted this review intending to investigate the diffusion and the effectiveness of AR technology as a learning media for students from various health fields.
Materials and Method. This systematic review was conducted using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols) Checklist. We used some databases including PubMed, Google Scholar, Wiley Online Library, and Sciencedirect to search relevant literature with eligibility criteria, namely articles published in the period 201-2021, and discuss the development of Augmented Reality -based applications for learning students in the field of health
Results. The studies included are on the development of AR-based learning applications carried out to improve the clinical skills of health students (Medicine, Nursing, and Midwifery). Various types of application development are carried out including anatomy, Endotracheal Intubation, AR Prototype for Medical Surgery, Intravascular Neurosurgery, injection skills, and Laparoscopic.
Conclusion. The use of Augmented Reality as a learning medium really helps improve the understanding and skills of students majoring in health sciences.
Keywords: Development, Augmented Reality, Health-Science, Students
The use of technology in the education of health science students has evolved over the years. These trends are mainly evolving in response to the challenges facing health education . The use of simulation in health education has been applied in the last 50 years . Augmented reality technology is an example of virtual reality technology developing rapidly in nursing education .
Augmented Reality (AR) technology refers to virtual elements to display the actual physical environment to create mixed-reality files in real-time. It complements and enhances the perceptions that humans acquire through their senses in the real world . AR provides various levels of understanding and interaction, which can help students in e-learning activities . For example, in an AR learning environment, motivational factors related to attention and learning satisfaction are rated higher than slide-based learning . Today’s development of smartphone technology makes AR technology more accessible to students and lecturers; for example, mobile learning (m-learning) using AR has become a trend .
Simulations using AR technology can replicate real-world aspects so that a safe learning environment is available for students where they can practice until the expected skill competencies are achieved . Simulation has become an integral part of nursing curricula , which involves using patient simulators, trained people, real-life virtual environments, and role play .
Technological advances over time have increased the realism and authenticity of the simulated environment, leading to increased reactions, satisfaction, learning attitudes, cognitive and affective outcomes among health students in general .
Clinical health services have also used AR because it provides an internal picture of the patient, without the need for invasive procedures [12–15]. Medical students and professionals need more situational experience in clinical care, especially for patient safety, so this shows that there is a real need to continue developing the use of AR in health education.
The focus of studies on AR in recent years [16,17] has highlighted the belief that AR provides medical students with rich contextual learning to help achieve core competencies, such as decision making, work for effective teams, and creative adaptation of global resources to address local priorities , AR provides more authentic and engaging learning opportunities for various learning styles, providing students with a more personalized and exploratory learning experience . The security of the patient will also be awake if an error occurs during skills training with AR .
This review was conducted to describe the development of AR technology as a learning medium for students from various health fields. This study is expected to be a reference material for teachers in learning strategies.
The research design is a Systematic Review, using the PRISMA-P 2009 (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols) Checklist.
To search for literature using the PubMed database, Google Scholar, Wiley Online Library, and Sciencedirect using the keywords “Developing” AND “Augmented Reality” AND “Clinical practice” AND (Medical OR Nurse OR Midewifery) “College student”.
We categorize the search into five categories that are considered to represent the topic of Augmented Reality development, namely AR typology, AR features and advantages, AR user perceptions, AR effectiveness in supporting learning, and AR design. Each category was analyzed to identify the best lessons, experiences, and evidence related to the design and development of AR.
The articles included in this review use the development method, with the subject of the trial being health students. In addition, the articles used are in English and full text, published in the period 2010–2020. Furthermore, the data obtained are then analyzed using quantitative descriptive methods and a narrative is produced that explains the study results.
The study results were documented to identify the effectiveness of using augmented reality in student health learning.
The studies included in the criteria for this review are only limited to studies on the development
of Augmented Reality technology for student learning in the health sector. Articles entered are in English, full text, and is not a thesis or dissertation.
Type of Participant/Population Target
The participants used were health students (Medicine, Nursing, Midwifery) who did clinical practicum (Clinical Skill). There are no restrictions on age, gender, level/semester, as long as participants do clinical practicum learning (clinical skills).
Quality assessment was carried out on six journals that met the inclusion and exclusion criteria using the JBI Critical Appraisal Checklist criteria. Journals are good if they meet at least 80%, moderate if they meet 50–80% and weak if they meet less than 50% of the criteria. Articles are used in good to moderate categories for further data synthesis, namely, grouping similar extracted data according to the results to be measured to conclude.
Literature Identification and Selection
There were 319 articles identified from four databases (Pubmed, Google Scholar, ScienceDirect, and Wiley Online Library) relevant to the review topic, where the assessment or screening was based on the title and abstract of the articles obtained. 66 studies were removed because they were duplicate. After screening the title and abstract, 219 studies were removed due to irrelevant theme, not AR topic, and proceeding types. At the eligibility stage, 28 studies were not fit the inclusion criterias.
Based on the JBI Critical Appraisal Checklist, six pieces of literature are in the excellent category, and two pieces of literature are in the weak category.
To maintain the quality of the literature studies made, this review only uses six good-quality journals, and then data extraction will be carried out (Figure 1).
Figure 1. PRISMA Flowchart: Strategy for Searching for Development of Augmented Reality in Educational Situations for Health-Science Students
After bearing the assessment, screening, and feasibility, the authors agreed to include six studies in this systematic review of the literature. Furthermore, the extraction of data from each of the included literature we describe in the following table displays the critical information needed with the theme of the study.
Table 1. Data Extraction on Included Articles
Characteristics of the studies included
The articles included in the inclusion criteria were six from several countries, including the USA as many as two articles, Canada 1 article, Sweden 1 article, Ireland 1 article, and Japan 1 article. Overall, the article taken is a study on the development of AR-based learning applications carried out to improve the clinical skills of health students (Medicine, Nursing, and Midwifery). Various types of application development are carried out including anatomy, Endotracheal Intubation, AR Prototype for Medical Surgery, Intravascular Neurosurgery, injection skills, and Laparoscopic.
Based on the JBI Critical Appraisal Checklist, six pieces of literature are in the excellent category, and two pieces of literature are in the weak category. To maintain the quality of the literature studies made, this review only uses six good-quality journals, and then data extraction will be carried out.
Table 2. Summary of Critical appraisal based on JBI checklist
AR system design
In Majimas’ work, the learners can learn experts’ nursing skills without moving their lines of sight. When practicing skills training, learners can learn skills by following and imitating (tracing) the images of experts’ techniques that are dis-played transparently in front of them in real time. The prototype system verified that training is possible by overlaying images on a simulation arm model.
Chien and colleague The system is based on a complete structure of the skull which can be decomposed and reassembled. To be an effective training tool, the system has to provide correct information to the students, the skull includes zygomatic bone, temporal bone, sphenoid bone, mandible, maxilla, ethimoid bone, parietal bone, frontal bone, occipital bone, nasal bone, lacrimal bone, palatine, vomer, and inferior nasal concha.
Torregrosa and team developed an ARBOOK which includes a standard part of descriptive anatomy of the lower limb including osteology, arthrology, myology, nerve and vascular supply. Each part of the book includes bi-dimensional images and text about the muscles: origin insertion, vascular and nerve supply or action. It also includes a card for each anatomical figure that can be recognized by a digital webcam connected to a computer. The users can modify the actual position of the virtual structure by moving the card. To develop the ARBOOK, more than 100 TC images were needed and the images were processed by OsiriX software and 3D constructed. LabHuman and VMV3D companies performed the animation.
Drapkin study, an open-source T1 and T2 weighted simulated MRI dataset of a normal human brain constructed from a composite of 27 volumetric datasets of the same living subject was obtained from the BrainWeb simulated brain database. This dataset was viewed using GEHC MicroView software, version 2.1.2 (General Electric Healthcare, Little Chalfont, Buckinghamshire, UK). 3D models were constructed using MicroView to create isosurfaces based on gray scale values within a given region of interest to create a 3D mesh approximating the shape of a given internal brain structure. These computer graphic object composites were exported as a VTK PolyData file and edited using Maya software, version 2010 (Autodesk, San Rafael, CA) and were examined by two neuroanatomists and one neurologist for accuracy and compared to the Netter’s Atlas of Human Neuroscience. The final edited versions were imported back into MicroView 2.1.2 as Wavefront OBJ files and overlaid on top of the original MRI dataset. The final product was a set of digital 3D models of internal brain structures that can be freely rotated and zoomed by the user. To fabricate the 3D-printed models in Licci study, anonymized CT data set of a patient with enlarged CSF spaces was first downloaded from the picture archiving and communication system (PACS) and further processed with the medical segmentation software Materialise Mimics (Mimics Innovation Suite v20; Materialise). The DICOM CT data set consisted of native cross-sectional slices of bone and soft-tissue windows to display the relevant anatomical features. Further processing and segmentation of several anatomical structures according to tissue density (Hounsfield units) was worked out. The virtual cranial vault was designed with the help of the modeling software Materialise 3-Matics to be removable and equipped with realistic, neurosurgical burr holes for endoscopic access. The osseous skull was printed completely (2 parts) with a consumer Replicator+ 3D printer (MakerBot Industries) from polylactic acid (PLA; light gray), and the corresponding ventricle spaces were divided into 2 parts with a wall thickness of 3 mm in transparent PLA material. After printing a total of 5 skull models, the support structures were manually removed, and the two halves of the ventricular system were glued together. These were inserted into the skull model, and the cavity between the ventricular system and the bony skull was filled with 2-component silicone for stabilization.
In the Islam study, they proposed a novel video-based approach for observing continuous, long sequence of surgeon’s hand and surgical tool movements in both surgical operation or surgical training, and then modeling and evaluating the skill demonstrated in the observation. Hand movement of entire surgical procedure is captured using inexpensive video camera. Video data of the tool movement can also be obtained for minimal invasive surgery (MIS). Both of the video data are analyzed using computer vision algorithm and then integrated to correlate with user’s skill level.
For modeling the surgical skill, a stochastic approach is proposed that uses simple arithmetic mean and standard deviation of the processed data. Using this technique, observer-independent models can be developed through objective and quantitative measurement of surgical skills. Because of the non-contact nature of the tracking technique, the system is free from sterile issue and there is minimal interference with the skill execution, unlike other methods that employ instrumented gloves or sensor-based surgical tools.
AR for Nursing skills
There is one study that developed the teaching skills of nurses using AR technology. The skill learned in the study was performing intravenous injections .
AR for Anatomy learning
Three studies  developed learning methods based on AR technology. AR technology was used to create an interactive learning environment, which allows students to understand the 3D skull structure with visual support . One of the studies gave their app the name ARBOOK, which can be presented in both, printed or electronic version. ARBOOK includes a standard part of descriptive anatomy of the lower limb including osteology, arthrology, myology, nerve and vascular supply . Another study developed 3D Neuroanatomy Teaching Tool. The models were created of the ventricular system, thalamus, hypothalamus, pituitary gland, hippocam-pus, amygdala, fornix, caudate, putamen, globus pallidus, brainstem, cerebral peduncles, and cerebellar peduncles .
AR for Surgical training
There are two studies that develop training based on AR technology. The first study involved a neuroendoscopic ventricular lesion removal training , and the second study provided two laparoscopic graspers and performed the pegboard transfer exercise on the FLS .
It is undeniable that the advancement of Augmented Reality technology has had a significant impact on the health sciences. Professions requiring high precision and good psychomotor abilities certainly require more time to practice carrying out their actions. The presence of Augmented Reality technology in its various forms is proven to increase students’ abilities and interests in dealing with the learning process.
Under certain conditions, especially during pandemic times where large-scale restrictions are imposed, direct meetings to carry out laboratory practicums are deemed possible, so there must be changes in strategies or effective learning methods for students in dealing with curriculum demands related to learning outcomes. A total of 6 eligible articles have been extracted to provide an overview of the development of Augmented Reality technology-based tools/tools in many health science fields, including Medicine, Nursing,/Midwifery. From the article, the discussion will be described based on the field of development, software and hardware used,
Two articles develop applications for learning body anatomy based on Augmented Reality . Tried to develop a 3D interactive learning environment of bone structure with visual support. This application is equipped with pop up labels and interactive displays in 3D to make it easier for users to see the position of each bone at various angles. In addition, users are also facilitated with the help of each label with information about the bone so that students no longer need to open books to look for information about the designated bone. To use this 3D application, students/users need hardware devices such as laptops/PCs equipped with cameras and pointers. For testing this device, Chien and colleagues used 30 medical students who had never taken anatomy courses to hope that the participants’ responses to this application would be of better quality. At the evaluation stage, participants revealed that the developed application was fascinating because it could provide a complete picture of the displayed bone structure and explain each pop-up label, making it easier to understand and memorize. In addition, another exciting thing is that the reassembled function in the application allows students to see the inner structure of the bone.
Another application developed by Torregrosa and colleagues in 2014 called ARBOOK (Augmented Reality Book) focuses on the anatomical structure of the lower extremities. For its development, 100 TC photos/images are needed, then the images are processed using OsiriX software and 3D object creation. For validation, the questionnaire compiled for the ARBOOK evaluation consists of the categories of task motivation and attention, autonomous work, comprehensive spatial orientation, and 3D interpretation. . Next, an expert assessment will be carried out. Application testing involves first-year health students who have never taken an anatomy course. The test results show a significant difference between learning using ARBOOK and conventional learning. As has been stated in previous studies that the use of virtual materials in anatomy learning can provide good benefits for student learning achievement, especially regarding motivation and independence [27,28].
Augmented Reality technology was also developed in Neuroanatomy learning for MRI exercises developed by Drapkin and colleagues in 2015. The developed application makes the brain image display into a 3D shape. This 3D model begins by using MicroView to form a primary image in the form of isosurfaces and then form a 3D model similar to the shape of the actual brain. The graph is then exported in VTK PolyData file format and edited using Maya software. The editing results are then given to neuroanatomists and neuroscientists to assess the accuracy of the image shape and compared with images on the ATLAS neuroscience Netter. The final image is then placed on top of the actual brain image from the MRI. Next, we entered the pilot phase, which was conducted on participants who were medical students at level 1. The trials showed that this 3D neuroanatomy teaching tool effectively trains medical students to read brain MRI and effectively teach students to identify internal brain structures.
In contrast to learning the body’s anatomical structure, surgical skills in surgery require hand-eye coordination, which can be achieved with continuous practice . In surgery, one is not enough to see what other people are doing when performing surgery; that is, to become skilled, it is necessary to “watch and do” .
One of the six articles included in this review is an Augmented Reality-based simulation development study for Neuroendoscopic Ventricular Removal exercises . In this development study, a 3D-printed model of synthetic body tissue was created. The idea is based on the limited material for practical surgery such as tumour removal. By using this 3D-printed model, it is hoped that it can accommodate all residents to do exercises repeatedly because this model is reusable.
Overall, the surveyed participants agreed or strongly agreed (Likert scores of 4 and 5) on the realistic nature of the anatomical model of the skull and ventricular system, the technical suitability of the model, the camera view, which was similar to the actual surgical view. Participants also agreed or strongly agreed that the content validity of the simulator is a valuable tool for enhancing surgical competence for neuro-endoscopic procedures that helps develop coordinating skills and represent an excellent practical exercise tool for ventricular tumour removal.
Other Augmented Reality-based surgical simulations are also included in this study. The development study conducted by Islam et al.  aims to create a video-based approach to observing surgeon hands and surgical instrument movements in surgery and surgical training. The data is captured with a video camera and then explored using a computer vision algorithm. Furthermore, by analyzing the basic statistical parameters, observer-independent performs objective and quantitative measurements of the surgical skills of the trainees. Computer vision is done through two steps, namely Glove/object detection and motion capture. This application is very suitable for remote assessment of student skills. Between the rater and the assessed, it is possible not to be in the room together; this allows the assessed participants to be calmer in the face of the assessment. Students can also receive virtual and interactive demonstrations of surgical procedures with surgeons carrying out the surgery so that students can experience real situations in the operating room.
Majima, et all  developed a practicum learning system for nursing students based on Augmented Reality, especially in the act of taking blood specimens. In certain types of blood vessels, beginners find it difficult to insert the needle. It is the basis for this research. Through this development, beginners can learn the “art” in the veins and imitate the images displayed in front of them. In injection skills education, both instructors and students are usually very interested in holding a syringe. However, in reality, the teaching given is limited to fixation, and the left finger technique is taught, which is tailored to the characteristics of each patient’s blood vessels that are difficult to insert a needle. How to repair and lengthen unstable blood vessels has not been entirely taught.
When practising skills training, students can learn skills by following and imitating (tracing) expert technical drawings transparently displayed in front of them in real-time. The prototype system verifies that training can be performed by overlaying the image on a simulated arm model.
The use of Augmented Reality as a learning medium really helps improve the understanding and skills of students majoring in health sciences. The many choices of models in application development provide opportunities for researchers to continue to innovate. Augmented Reality-based learning applications in the future become an absolute thing along with the increasing development of technology.
Many databases not used in this review, such as Scopus, Ebsco, IEEE, and others, are very
credible for searching literature/articles. It is due to limited access to these databases. The use of gray literature such as google scholar conducted carefully with agreement of all authors.
The author also has limitations in understanding the software and programming languages used in the articles reviewed, so the authors cannot further discuss the application development process in the six articles reviewed.
This study provides a broad overview of the Augmented Reality-based application development process so that it can be a reference material for future teachers or researchers to be able to innovate in the development of Augmented Reality-based learning applications, for example, in the process of guiding final project students, or multiplying nursing action tutorials that are currently available. Not yet fully available in the form of an Augmented Reality application.
This systematic review does not get funding.
Conflict of Interest
The author declares there is no conflict of interest in this study.
- Birt J, Stromberga Z, Cowling M, Moro C. Mobile mixed reality for experiential learning and simulation in medical and health sciences education. Information. 2018;9(2):31.
- Jones F, Passos-Neto CE, Braghiroli OFM. Simulation in medical education: brief history and methodology. Principles and practice of clinical research. 2015;1(2).
- Foronda CL, Alfes CM, Dev P, Kleinheksel AJ, Nelson Jr DA, O’Donnell JM, et al. Virtually nursing: Emerging technologies in nursing education. Nurse educator. 2017;42(1):14–7.
- Arena F, Collotta M, Pau G, Termine F. An Overview of Augmented Reality. Computers. 2022;11(2):28.
- Chiang THC, Yang SJH, Hwang G-J. Students’ online interactive patterns in augmented reality-based inquiry activities. Computers & Education. 2014;78:97–108.
- Di Serio Á, Ibáñez MB, Kloos CD. Impact of an augmented reality system on students’ motivation for a visual art course. Computers & Education. 2013;68:586–96.
- Khan T, Johnston K, Ophoff J. The impact of an augmented reality application on learning motivation of students. Advances in Human-Computer Interaction. 2019;2019.
- Jeffries PR. A framework for designing, implementing, and evaluating: Simulations used as teaching strategies in nursing. Nursing education perspectives. 2005;26(2):96–103.
- Aebersold M, Voepel-Lewis T, Cherara L, Weber M, Khouri C, Levine R, et al. Interactive anatomy-augmented virtual simulation training. Clinical simulation in nursing. 2018;15:34–41.
- Scalese RJ, Issenberg SB. Effective use of simulations for the teaching and acquisition of veterinary professional and clinical skills. Journal of veterinary medical education. 2005;32(4):461–7.
- Hamacher A, Kim SJ, Cho ST, Pardeshi S, Lee SH, Eun S-J, et al. Application of virtual, augmented, and mixed reality to urology. International neurourology journal. 2016;20(3):172.
- Bajura M, Fuchs H, Ohbuchi R. Merging virtual objects with the real world: Seeing ultrasound imagery within the patient. ACM SIGGRAPH Computer Graphics. 1992;26(2):203–10.
- Cameron C. How augmented reality helps doctors save lives. Read Write Web. 2010.
- De Paolis LT, Ricciardi F, Dragoni AF, Aloisio G. An augmented reality application for the radio frequency ablation of the liver tumors. In: International Conference on Computational Science and Its Applications. Springer; 2011. p. 572–81.
- Pandya A, Siadat M-R, Auner G. Design, implementation and accuracy of a prototype for medical augmented reality. Computer Aided Surgery. 2005;10(1):23–35.
- Rolland JP, Biocca F, Hamza-Lup F, Ha Y, Martins R. Development of head-mounted projection displays for distributed, collaborative, augmented reality applications. Presence: Teleoperators & Virtual Environments. 2005;14(5):528–49.
- Sielhorst T, Obst T, Burgkart R, Riener R, Navab N. An augmented reality delivery simulator for medical training. In: International workshop on augmented environments for medical imaging-MICCAI Satellite Workshop. 2004. p. 11–20.
- Frenk J, Chen L, Bhutta ZA, Cohen J, Crisp N, Evans T, et al. Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. The lancet. 2010;376(9756):1923–58.
- Parmaxi A, Demetriou AA. Augmented reality in language learning: A state‐of‐the‐art review of 2014–2019. Journal of Computer Assisted Learning. 2020;36(6):861–75.
- Kobayashi L, Zhang XC, Collins SA, Karim N, Merck DL. Exploratory application of augmented reality/mixed reality devices for acute care procedure training. Western Journal of Emergency Medicine. 2018;19(1):158.
- Majima Y, Masuda S, Matsuda T. Development of Augmented Reality in Learning for Nursing Skills. In: MEDINFO 2019: Health and Wellbeing e-Networks for All. IOS Press; 2019. p. 1720–1.
- Chien C-H, Chen C-H, Jeng T-S. An interactive augmented reality system for learning anatomy structure. In: proceedings of the international multiconference of engineers and computer scientists. International Association of Engineers Hong Kong, China; 2010. p. 17–9.
- Ferrer-Torregrosa J, Torralba J, Jimenez MA, García S, Barcia JM. ARBOOK: Development and assessment of a tool based on augmented reality for anatomy. Journal of Science Education and Technology. 2015;24(1):119–24.
- Drapkin ZA, Lindgren KA, Lopez MJ, Stabio ME. Development and assessment of a new 3D neuroanatomy teaching tool for MRI training. Anatomical sciences education. 2015;8(6):502–9.
- Licci M, Thieringer FM, Guzman R, Soleman J. Development and validation of a synthetic 3D-printed simulator for training in neuroendoscopic ventricular lesion removal. Neurosurgical focus. 2020;48(3):E18.
- Islam G, Kahol K, Ferrara J, Gray R. Development of computer vision algorithm for surgical skill assessment. In: International Conference on Ambient Media and Systems. Springer; 2011. p. 44–51.
- Fairén M, Moyés J, Insa E. VR4Health: Personalized teaching and learning anatomy using VR. Journal of medical systems. 2020;44(5):1–11.
- Pringle RM, Dawson K, Ritzhaupt AD. Integrating science and technology: Using technological pedagogical content knowledge as a framework to study the practices of science teachers. Journal of Science Education and Technology. 2015;24(5):648–62.
- Rao A, Tait I, Alijani A. Systematic review and meta-analysis of the role of mental training in the acquisition of technical skills in surgery. The American Journal of Surgery. 2015;210(3):545–53.
- Herrlin S V, Wange PO, Lapidus G, Hållander M, Werner S, Weidenhielm L. Is arthroscopic surgery beneficial in treating non-traumatic, degenerative medial meniscal tears? A five year follow-up. Knee Surgery, Sports Traumatology, Arthroscopy. 2013;21(2):358–64.