Surface reconstruction of abdominal organs using laparoscopic structured light for augmented reality (original) (raw)
Related papers
Surface reconstruction of abdominal organs using laparoscopic structured light for augmented reality
Three-Dimensional Image Capture and Applications V, 2002
Creation of accurate surface models of abdominal organs is essential for many developing technologies in medicine and surgery. One application we are working towards is augmented reality (AR) visualization for laparoscopic surgery. Our current system meets some, but not all, of the requirements. We use two custom built laparoscopes, a custom built miniature projector, a standard camera, and a standard video capture and processing card to implement a laparoscopic structured light range acquisition system. We will briefly show the custom hardware but will emphasize the structured light depth extraction techniques used for the unique properties of surfaces inside the body, particularly dealing with specular reflections. In early experiments, we studied the effectiveness of our algorithm in highly specular environments by creating range images acquired from fresh animal organs. These experiments used a large projector, open abdomens, and offline image processing. We report the results of experiments using our miniature projector, and on line processing.
Augmented reality visualization for laparoscopic surgery
1998
We present the design and a prototype implementation of a three-dimensional visualization system to assist with laparoscopic surgical procedures. The system uses 3D visualization, depth extraction from laparoscopic images, and six degree-of-freedom head and laparoscope tracking to display a merged real and synthetic image in the surgeon's video-see-through head-mounted display. We also introduce a custom design for this display.
Surgical Structured Light for 3D Minimally Invasive Surgical Imaging
Surgeons perform minimally invasive surgery using an image delivered by a laparoscope and a camera system that provides a high definition 2D image, but this leaves the surgeon without 3D depth perception. The lack of depth perception can slow the surgeon, increase the risk of misidentifying structures, and/or inadvertently cause unwanted injury to tissues surrounding the surgical site. To address the lack of depth perception, we propose a Surgical Structured Light (SSL) system that includes a 3D sensor capable of measuring and modeling the surgical site during a procedure. The 3D information provided by this system can enable the surgeon to: 1) improve the navigation of tools based on precise localization of instruments in relation to structures in the surgical site, 2) allow 3D visualizations side-by-side with a standard 2D color image, and 3) precisely measure sizes of structures (e.g., tumors) and distances between structures with simple mouse clicks. We demonstrate the accuracy of our SSL system using ex-vivo data on both a cylinder calibration object as well as various plastic organs.
Challenging requirements and optical depth estimation techniques in laparoscopy
Current Directions in Biomedical Engineering, 2022
Minimally invasive surgery has many advantages and cannot be missed nowadays. It leads to faster recovery and less surgical trauma. In laparoscopic surgery years of training are required to optimize patient outcomes. To visually support the surgeon during stomach surgery a 3D-reconstruction of the whole organ shall be created prior and during the procedure. Results might be beneficial for various applications such as before-and-after documentation, navigation support and autonomous robotic surgery. The technical implementation of 3D-reconstruction requires depth estimation which is challenged due to the environmental conditions and surgical constraints that exist in the human body during minimally invasive surgery. This paper focuses on the requirements of 3D-reconstruction in laparoscopy, reveals current research challenges and proposes an evaluation framework for optical depth estimation techniques. Eight methods were included in the evaluation. Scores considering the requirements were established and assigned to each method. The methods Deformable Shape-from-Motion, Stereoscopy, Shape-from-Motion, Simultaneous Localization and Mapping, Structured Light and Light-Field Technology were shown to partially fulfill the requirements for laparoscopic 3D-reconstruction. Shape-from-Shading and Time-of-Flight need extensive modifications to be applicable. In conclusion it can be stated that currently no method exists to realize a real time highresolution 3D-reconstruction of inner organs during laparoscopy.
A System to Support Laparoscopic Surgery by Augmented Reality Visualization
Lecture Notes in Computer Science, 2001
This paper describes the development of an augmented reality system for intra-operative laparoscopic surgery support. The goal of this system is to reveal structures, otherwise hidden within the laparoscope view. To allow flexible movement of the laparoscope we use optical tracking to track both patient and laparoscope. The necessary calibration and registration procedures were developed and bundled where possible in order to facilitate integration in a current laparoscopic procedure. Care was taken to achieve high accuracy by including radial distortion components without compromising real time speed. Finally a visual error assessment is performed, the usefulness is demonstrated within a test setup and some preliminary quantitative evaluation is done.
Proceedings of SPIE - The International Society for Optical Engineering
Gathering depth information through an endoscope or laparoscope during surgical or other procedures is quite difficult. There are stereo laparoscopes but generating three-dimensional models with them is very difficult. Accurate real-time generation of three-dimensional models through a laparoscope is a needed technology to enable a wide range of surgical applications. We have designed a miniature laparoscopic optical system consisting of a single laser whose pattern is modulated and uses the laparoscope as the optical display path into the body. Two cameras, one sensitive to the laser light and the other for full color imaging share this same tube as the laser projector but use the light from the opposite direction. The images gathered by the laser sensitive camera are used to generate a three dimensional map, and the color image is used to acquire the corresponding texture map. High-speed image processing hardware is used to generate 3D information using a structured light techniqu...
Framework for augmented reality in Minimally Invasive laparoscopic surgery
2015 17th International Conference on E-health Networking, Application & Services (HealthCom), 2015
This article presents a framework for fusing preoperative data and intra-operative data for surgery guidance. This framework is employed in the context of Minimally Invasive Surgery (MIS) of the liver. From stereoscopic images a three dimensional point cloud is reconstructed in real-time. This point cloud is then used to register a patient-specific biomechanical model derived from Computed Tomography images onto the laparoscopic view. In this way internal structures such as vessels and tumors can be visualized to help the surgeon during the procedure. This is particularly relevant since abdominal organs undergo large deformations in the course of the surgery, making it difficult for surgeons to correlate the laparoscopic view with the pre-operative images. Our method has the potential to reduce the duration of the operation as the biomechanical model makes it possible to estimate the in-depth position of tumors and vessels at any time of the surgery, which is essential to the surgical decision process. Results show that our method can be successfully applied during laparoscopic procedure without interfering with the surgical work flow.
Medical Imaging 2014: Image-Guided Procedures, Robotic Interventions, and Modeling, 2014
In laparoscopic surgery, live video provides visualization of the exposed organ surfaces in the surgical field, but is unable to show internal structures beneath those surfaces. The laparoscopic ultrasound is often used to visualize the internal structures, but its use is limited to intermittent confirmation because of the need for an extra hand to maneuver the ultrasound probe. Other limitations of using ultrasound are the difficulty of interpretation and the need for an extra port. The size of the ultrasound transducer may also be too large for its usage in small children. In this paper, we report on an augmented reality (AR) visualization system that features continuous hands-free volumetric ultrasound scanning of the surgical anatomy and video imaging from a stereoscopic laparoscope. The acquisition of volumetric ultrasound image is realized by precisely controlling a back-and-forth movement of an ultrasound transducer mounted on a linear slider. Furthermore, the ultrasound volume is refreshed several times per minute. This scanner will sit outside of the body in the envisioned use scenario and could be even integrated into the operating table. An overlay of the maximum intensity projection (MIP) of ultrasound volume on the laparoscopic stereo video through geometric transformations features an AR visualization system particularly suitable for children, because ultrasound is radiation-free and provides higher-quality images in small patients. The proposed AR representation promises to be better than the AR representation using ultrasound slice data.