Gregory Hager - Profile on Academia.edu (original) (raw)

Papers by Gregory Hager

The ability to accurately recognize elementary surgical gestures is a step- ping stone to automat... more The ability to accurately recognize elementary surgical gestures is a step- ping stone to automated surgical assessment and surgical training. However, as the pool of subjects increases, variation in surgical techniqu es and unanticipated mo- tion increases the challenge of creating robust statistica l models of gestures. This paper examines the applicability of advanced modeling techniques from automated speech recognition

Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006., 2006

Virtual fixtures are virtual constraints that enhance human performance in motion tasks. They can... more Virtual fixtures are virtual constraints that enhance human performance in motion tasks. They can either confine and/or guide a user's motion. In this paper, we use a commercially available motion platform to explore the portability and applicability of virtual fixtures and document how people interact with them. Two micromanipulation tasks are analyzed and the effects of similarly designed virtual fixtures are discussed. One task simulates a medical task, retinal vein cannulation, and the other simulates a manufacturing task, fine leads soldering. Preliminary experimental results show that the virtual fixtures increase the accuracy of both medical and manufacturing tasks, lending support to its portability and applicability across unrelated tasks.

In retinal surgery, surgeons face difficulties such as indirect visualization of surgical targets... more In retinal surgery, surgeons face difficulties such as indirect visualization of surgical targets, physiological tremor and lack of tactile feedback. Such difficulties increase the risks of incorrect surgical gestures which may cause retinal damage. In this context, robotic assistance has the potential to overcome current technical limitations and increase surgical safety. In this paper we present a method for robustly tracking surgical tools in retinal surgery for detecting proximity between surgical tools and the retinal surface. An image similarity function based on weighted mutual information is specially tailored for tracking under critical illumination variations, lens distortions, and rapid motion. The proposed method was tested on challenging conditions using a phantom eye and recorded human in vivo data acquired by an ophthalmic stereo microscope.

We present a system architecture for robust target following with a mobile robot. The system is b... more We present a system architecture for robust target following with a mobile robot. The system is based on tracking multiple cues in binocular stereo images using the XVision toolkit. Fusion of complementary information in the images, including texture, color and depth, combined with a fast optimized processing reduces the possibility of loosing the tracked object in a dynamic scene with

Abstract, We present a platform for human-machine in-terfaces that provides functionality for rob... more Abstract, We present a platform for human-machine in-terfaces that provides functionality for robust, unencumbered interaction: the 4D Touchpad (4DT). The goal is direct in-teraction with interface components through intuitive actions and gestures. The 4DT is based on the 3D-2D Projection-based mode of the VICs framework. The fundamental idea behind VICs is that expensive global image processing with user modeling and

Application of task-level augmentation for cooperative fine manipulation tasks in surgery

Medical Image Computing and Computer-Assisted Intervention, 2001

We report on applications of an augmentation system for cooperative fine manipulation using the s... more We report on applications of an augmentation system for cooperative fine manipulation using the steady hand manipulation paradigm. Using the "steady hand" robot as the experimental platform we investigate using such a system for creating composite retinal images using a GRIN lens endoscope and for puncturing small blood vessels in the retina.

2008 2nd IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics, 2008

The real-time detection of surgeon's action could be very useful for intelligent intraoperative a... more The real-time detection of surgeon's action could be very useful for intelligent intraoperative assistance. Robotic assisted surgery provides quantified information that can be used to detect these actions. In this paper we present our attempt at spotting tool-tissue interactions from real surgery data, in the context of robotic assisted laparoscopy with the daVinci TM system, using both kinematic information and visual information. Our early results are encouraging in this very challenging context.

2009 IEEE International Conference on Robotics and Automation, 2009

We describe a general methodology for tracking 3-dimensional objects in monocular and stereo vide... more We describe a general methodology for tracking 3-dimensional objects in monocular and stereo video that makes use of GPU-accelerated filtering and rendering in combination with machine learning techniques. The method operates on targets consisting of kinematic chains with known geometry. The tracked target is divided into one or more areas of consistent appearance. The appearance of each area is represented by a classifier trained to assign a class-conditional probability to image feature vectors. A search is then performed on the configuration space of the target to find the maximum likelihood configuration. In the search, candidate hypotheses are evaluated by rendering a 3D model of the target object and measuring its consistency with the class probability map. The method is demonstrated for tool tracking on videos from two surgical domains, as well as in a human hand-tracking task.

2007 IEEE Conference on Computer Vision and Pattern Recognition, 2007

In this paper, we address two closely related visual tracking problems: 1) localizing a target's ... more In this paper, we address two closely related visual tracking problems: 1) localizing a target's position in low or moderate resolution videos and 2) segmenting a target's image support in moderate to high resolution videos. Both tasks are treated as an online binary classification problem using dynamic foreground/background appearance models. Our major contribution is a novel nonparametric approach that successfully maintains a temporally changing appearance model for both foreground and background. The appearance models are formulated as "bags of image patches" that approximate the true two-class appearance distributions. They are maintained using a temporaladaptive importance resampling procedure that is based on simple nonparametric statistics of the appearance patch bags. The overall framework is independent of an specific foreground/background classification process and thus offers the freedom to use different classifiers. We demonstrate the effectiveness of our approach with extensive comparative experimental results on sequences from previous visual tracking and video matting [4] work as well as our own data.

Two decades after the first reported robotic surgical procedure , surgical robots are just beginn... more Two decades after the first reported robotic surgical procedure , surgical robots are just beginning to be widely used in the operating room or interventional suite. The da Vinci telerobotic system (Intuitive Surgical, Inc.), for example, has recently become more widely employed for minimally invasive surgery . This article, the first in a three-part series, examines the core concepts underlying surgical and interventional robots, including the potential benefits and technical approaches, followed by a summary of the technical challenges in sensing, manipulation, user interfaces, and system design. The article concludes with a review of key design aspects, particularly in the areas of risk analysis and safety design. Note that medical care can be delivered in a surgical suite (operating room) or an interventional suite, but for convenience, we will henceforth use the term surgical to refer to both the surgical and interventional domains.

Proceedings 2007 IEEE International Conference on Robotics and Automation, 2007

This paper presents a new visual servoing method that is able to stabilize a moving area of soft ... more This paper presents a new visual servoing method that is able to stabilize a moving area of soft tissue within an ultrasound B-mode imaging plane. The approach consists of moving the probe in order to minimize the relative position between a target imaging plane and the ultrasound plane observed by the probe of the moving tissue target. The problem is decoupled into motion out-of-plane and motion within plane. For the former, a new original method based on the speckle information contained in the images is developed. For the latter, an image region tracker is used to provide the in-plane motion. A visual servoing control scheme is then developed to perform the tracking robotic task. The method is validated on simulated motions of a probe on a static ultrasound volume acquired from a phantom.

Visual Tracking of Surgical Tools for Proximity Detection in Retinal Surgery

Lecture Notes in Computer Science, 2011

We present the design and implementation of a vi- sion-based system for cooperative manipulation ... more We present the design and implementation of a vi- sion-based system for cooperative manipulation at millimeter to micrometer scales. The system is based on an admittance control algorithm that implements a broad class of guidance modes called virtual fixtures. A virtual fixture, like a real fixture, limits the mo- tion of a tool to a prescribed class or range of

Automated Diagnosis of Barrett’s Esophagus with Endoscopic Images

IFMBE Proceedings, 2009

In this paper, we describe current progress on the development of a Computer Assisted Diagnosis S... more In this paper, we describe current progress on the development of a Computer Assisted Diagnosis System (CAD) for the classification of Barrett’s esophagus and associated neoplasia. Barrett’s esophagus is a condition in which normal squamous mucosa is replaced by columnar epithelium, which is similar to the lining of the intestine. Barrett’s esophagus as a known precancerous condition leading to esophageal

Feature-Based Visual Servoing and its Application to Telerobotics

Intelligent Robots and Systems, 1995

Recent advances in visual servoing theory and practice now make it possible to accurately and rob... more Recent advances in visual servoing theory and practice now make it possible to accurately and robustly position a robot manipulator relative to a target, Both the vision and con-trol algorithms are extremely simple, however they must be initialized on task-relevant features in order ...

Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171), 1998

This paper outlines a set of problems associated with constructing a robust, domain-independent v... more This paper outlines a set of problems associated with constructing a robust, domain-independent visionbased navigation system suitable for both structured and unstructured environments. The system utilizes visual tracking to monitor a set of automatically selected image features (markers), and employs visionbased control to guide the motion of the robot from the image trajectory of a set of markers. An environment is represented as a graph which may be constructed either under human control (e.g. by giving the system a tour) or autonomously as the system explores. In this paper, we review the system architecture and present two image-based mobile robot controllers for following visually-de ned trajectories.

Surgical Gesture Segmentation and Recognition

Lecture Notes in Computer Science, 2013

Automatic surgical gesture segmentation and recognition can provide useful feedback for surgical ... more Automatic surgical gesture segmentation and recognition can provide useful feedback for surgical training in robotic surgery. Most prior work in this field relies on the robot&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;#39;s kinematic data. Although recent work [1,2] shows that the robot&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;#39;s video data can be equally effective for surgical gesture recognition, the segmentation of the video into gestures is assumed to be known. In this paper, we propose a framework for joint segmentation and recognition of surgical gestures from kinematic and video data. Unlike prior work that relies on either frame-level kinematic cues, or segment-level kinematic or video cues, our approach exploits both cues by using a combined Markov/semi-Markov conditional random field (MsM-CRF) model. Our experiments show that the proposed model improves over a Markov or semi-Markov CRF when using video data alone, gives results that are comparable to state-of-the-art methods on kinematic data alone, and improves over state-of-the-art methods when combining kinematic and video data.

Decidability of robot positioning tasks using stereo vision systems

Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171), 1998

Page 1. FM02 11:30 Proceedings of the 37th IEEE Conference on Decision & Control Tampa, F... more Page 1. FM02 11:30 Proceedings of the 37th IEEE Conference on Decision & Control Tampa, Florida USA December 1998 Decidability of Robot Positioning Tasks Using Stereo Vision Systems* JP Hespanha Z. Dodds G. D. Hager A. S. Morse ...

Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146), 1998

Over the last several years, there has been an increasing appreciation of the impact of control a... more Over the last several years, there has been an increasing appreciation of the impact of control architecture on the accuracy of visual servoing systems. In particular, it is generally acknowledged that so-called image-based methods provide the highest guarantees of accuracy on inaccurately calibrated hand-eye systems.