An Interactive Holographic Curriculum for Urogynecologic Surgery (original) (raw)
Medical Research Archives, 2023
ABSTRACT: Image-guided surgery using XR (extended reality: VR/AR/MR) technology has the potential to revolutionize the field of surgery by improving surgical accuracy, reducing procedure time, and enhancing communication and collaboration among the surgical team. We have developed a web-based system, Holoeyes, integrating XR, AI, and metaverse technology to facilitate holographic image- guided surgery. Holoeyes extracts organ shape data from CT or MRI scans and renders them with positional information to obtain X, Y, and Z coordinates. These coordinates are then converted into polygonal information for use in XR technology. The medical device, Holoeyes MD, was developed to create XR applications for surgical planning and navigation. It provides an immersive experience for the surgical team, improving both accuracy and efficiency. The integration of the metaverse in surgery allows for spatial conferencing and review of training, and the avatars replicate the hand and eye movements of the actual surgical procedure. Our Holoeyes system has already been utilized in numerous institutions for pre-and post-operative conferences, surgical planning, and surgical records, with multiple people wearing the headset and sharing information about the pathology, extent of resection, and layers of dissection from all directions. We conducted a systematic review of the literature to investigate the effectiveness of Holoeyes, focusing on the use of XR and the metaverse in surgery. We believe that Holoeyes has the potential to become an indispensable tool in the field of surgery, and we encourage further research and development in this field.
The Virtual Pelvic Floor, a tele-immersive educational environment
Proceedings / AMIA ... Annual Symposium. AMIA Symposium, 1999
This paper describes the development of the Virtual Pelvic Floor, a new method of teaching the complex anatomy of the pelvic region utilizing virtual reality and advanced networking technology. Virtual reality technology allows improved visualization of three-dimensional structures over conventional media because it supports stereo vision, viewer-centered perspective, large angles of view, and interactivity. Two or more ImmersaDesk systems, drafting table format virtual reality displays, are networked together providing an environment where teacher and students share a high quality three-dimensional anatomical model, and are able to converse, see each other, and to point in three dimensions to indicate areas of interest. This project was realized by the teamwork of surgeons, medical artists and sculptors, computer scientists, and computer visualization experts. It demonstrates the future of virtual reality for surgical education and applications for the Next Generation Internet.
EAI Endorsed Trans. Pervasive Health Technol., 2021
INTRODUCTION: Mixed reality (MR) allows surgeons to pre-operatively assess patients’ anatomy (e.g., tissue to be removed). However, medical students have limited access to this technology, and express both the desire to try it and suspicious attitudes. OBJECTIVES: To assess students’ experience with traditional vs. innovative technology for pre-operative planning. METHODS: 11 medical students analyzed a lung cancer case using CT scans or a 3D hologram (MR) and assessed the technology in terms of mental workload, emotions and formative value. RESULTS: MR resulted in less cognitive load and effort, shorter response time and more positive emotions. No differences emerged in formative value, but the students expressed the desire to be trained both in traditional and innovative technology for pre-operative planning. CONCLUSION: Medical students respond positively to “hands-on” experiences of technology for pre-operative planning. The time may be ripe to include MR in medical formation.
Journal of Cardiovascular Development and Disease, 2023
We sought to determine the role of the patient-specific, three-dimensional (3D) holographic vascular model in patient medical knowledge and its influence on obtaining a more conscious informed consent process for percutaneous balloon angioplasty (PTA). Patients with peripheral arterial disease who had been scheduled for PTA were enrolled in the study. Information regarding the primary disease, planned procedure, and informed consent was recorded in typical fashion. Subsequently, the disease and procedure details were presented to the patient, showing the patients their individual model. A patient and medical supervisor equipped with mixed reality headsets could both simultaneously manipulate the hologram using gestures. The holographic 3D model had been created on a scale of 1:1 based on computed tomography scans. The patient's knowledge was tested by the completion of a questionnaire before and after the interaction in a mixed reality environment. Seventy-nine patients manipulated arterial holograms in mixed reality head-mounted devices. Before the 3D holographic artery model interaction, the mean ± standard deviation score of the knowledge test was 2.95 ± 1.21 points. After the presentation, the score had increased to 4.39 ± 0.82, with a statistically significant difference (p = 0.0000) between the two scores. Using a Likert scale from 1 to 5, the patients had scored the use of the 3D holographic model at 3.90 points regarding its usefulness in comprehending their medical condition; at 4.04 points regarding the evaluation of the holograms as helpful in understanding the course of surgery; and rated the model at 1.99 points in reducing procedure-related stress. Using a nominal scale (know or don't know), the patients had self-assessed their knowledge of the procedure before and after the 3D model presentation, with a score of 6.29 ± 2.01 and 8.39 ± 1.54, respectively. The study group tolerated the use of head-mounted devices. Only one patient had nausea and dizziness, while four patients experienced transient eye pain. The 3D holographic arterial model aided in the understanding of patients' knowledge regarding the disease and procedure, making the informed consent process more conscious. The holograms improved the patient's self-consciousness. Mixed reality headset-related complications were rare and within acceptable rates.
Interactive Holograms and Tutorials in Healthcare Education: Case Studies from the e-REAL Experience
International Journal of Advanced Corporate Learning (iJAC), 2016
With both portable and permanent fixtures available, the e-REAL lab encompasses users in an entirely interactive and immersive ecosystem; advanced medical simulation reaches its best thanks to interactive 3D holographic visualization. Using a number of tools, e-REAL enables not only face-to-face training, but also e-learning and remote communication across the globe.
2011
Pelvic floor anatomy is complex and its three-dimensional organization is often difficult to understand for both undergraduate and postgraduate students. Here, we focused on several critical points that need to be considered when teaching the perineum. We have to deal with a mixed population of students and with a variety of interest. Yet, a perfect knowledge of the pelvic floor is the basis for any gynecologist and for any surgical intervention. Our objectives are several-fold; i) to establish the objectives and the best way of teaching, ii) to identify and localize areas in the female pelvic floor that are susceptible to generate problems in understanding the three-dimensional organization, iii) to create novel approaches by respecting the anatomical surroundings, and iv) prospectively, to identify elements that may create problems during surgery i.e. to have a closer look at nerve trajectories and on compression sites that may cause neuralgia or postoperative pain. A feedback from students concludes that they have difficulties to assimilate this much information, especially the different imaging techniques. Eventually, this will lead to a severe selection of what has to be taught and included in lectures or practicals. Another consequence is that more time to study prosected pelves needs to be given.
Utilisation of the HoloLens mixed-reality device in minimally invasive surgery
European Urology Supplements, 2019
Background The advent of Virtual Reality technologies presents new opportunities for enhancing current surgical practice. Studies suggest that current techniques in endoscopic surgery are prone to disturbance of a surgeon's visual-motor axis, influencing performance, ergonomics and iatrogenic injury rates. The Microsoft ® HoloLens is a novel head-mounted display that has not been explored within surgical innovation research. This study aims to evaluate the HoloLens as a potential alternative to conventional monitors in endoscopic surgery. Materials and methods This prospective, observational and comparative study recruited 72 participants consisting of novices (n = 28), intermediate-level (n = 24) and experts (n = 20). Participants performed ureteroscopy, within an inflatable operating environment, using a validated training model and the HoloLens mixed-reality device as a monitor. Novices also completed the assigned task using conventional monitors; whilst the experienced groups did not, due to their extensive familiarity. Outcome measures were procedural completion time and performance evaluation (OSATS) score. A final evaluation survey was distributed amongst all participants. Results The HoloLens facilitated improved outcomes for procedural times (absolute difference, − 73 s; 95% CI − 115 to − 30; P = 0.0011) and OSAT scores (absolute difference, 4.1 points; 95% CI 2.9-5.3; P < 0.0001) compared to conventional monitors. Feedback evaluation demonstrated 97% of participants agreed or strongly agreed that the HoloLens will have a role in surgical education (mean rating, 4.6 of 5; 95% CI 4.5-4.8). Furthermore, 95% of participants agreed or strongly agreed that the HoloLens is feasible to introduce clinically and will have a role within surgery (mean rating, 4.4 of 5; 95% CI 4.2-4.5). Conclusion This study demonstrates that the device facilitated improved outcomes of performance in novices and was widely accepted as a surgical visual aid by all groups. The HoloLens represents a feasible alternative to the conventional setup, possibly by aligning the surgeon's visual-motor axis.
Surgical simulator for hysteroscopy: a case study of visualization in surgical training
Proceedings Visualization, 2001. VIS '01.
Computer-based surgical simulation promises to provide a broader scope of clinical training through the introduction of anatomic variation, simulation of untoward events, and collection of performance data. We present a haptically-enabled surgical simulator for the most common techniques in diagnostic and operative hysteroscopy-cervical dilation, endometrial resection and ablation, and lesion excision. Engineering tradeoffs in developing a real-time, haptic-rate simulator are discussed.
Highly-realistic, immersive training environment for hysteroscopy
Studies in Health Technology and Informatics, 2006
The primary driving application of our current research is the development of a generic surgical training simulator for hysteroscopy. A key target is to go beyond rehearsal of basic manipulative skills, and enable training of procedural skills like decision making and problem solving. In this respect, the sense of presence plays an important role in the achievable training effect. To enable user immersion into the training environment, the surrounding and interaction metaphors should be the same as during the real intervention. To this end, we replicated an OR in our lab, provided standard hysteroscopic tools for interaction, and generate a new virtual scene for every session. In this setting, the training starts, as soon as the trainees enter the OR, and ends when they leave the room.
A real-time graphic environment for a urological operation training simulator
2004
Abstract An OpenGL/C++ real-time graphic environment, part of a training simulator for urological operations, is presented. The graphic environment simulates endoscope insertion in a small diameter deformable tube and is used with a low-force 5-dof force-feedback haptic mechanism. Piecewise Bezier interpolations are used for smooth urethra deformations. A novel particle-based model computes the forces and torques fed to the haptics.