John McGuirl - Academia.edu (original) (raw)

Papers by John McGuirl

Research paper thumbnail of Supporting trust calibration through the presentation of dynamic reliability information /

Research paper thumbnail of Seeing is believing?: The effects of real-time, image- based feedback on emergency management decision- making

Emergency management personnel often face feedback delays and a lack of reliable information. To ... more Emergency management personnel often face feedback delays and a lack of reliable information. To address this problem, new information technologies have been developed that can provide real-time, image-based feedback. While potentially useful, this trend represents a fundamental shift in both the timing and format of the information used by incident commanders (ICs). Eight ICs took part in a simulation exercise to determine the potential impact of real-time imaging on their decision-making. Nearly all of the ICs failed to detect important changes in the situation that were not captured in the imaging but that were available via other, more traditional data sources. It appears that the ICs placed an inappropriately high level of trust in the imaging data, resulting in reduced data search activities and hypothesis generation. This research helps practitioners anticipate and guard against undesirable effects of introducing similar technologies on training and operational procedures.

Research paper thumbnail of The Effects of Real-Time Image-Based Feedback on Data Gathering and Analysis: The Case of Emergency Management Decision Making

Several recent events, such as 9/11 and hurricane Katrina, have placed an increased emphasis on s... more Several recent events, such as 9/11 and hurricane Katrina, have placed an increased emphasis on supporting decision-makers during disaster recovery missions. To this end advanced information technologies have been developed that provide real-time feedback to practitioners who have traditionally been forced to operate in a feed-forward control mode due to a lack of timely information about the system they are controlling. In particular, a number of new remote sensing technologies have been introduced that present remotely located decision-makers with real-time imaging of the area of interest. This new data channel promises several benefits to decision-makers at all levels of command and control. It can aid in assessing and comprehending the overall situation while also allowing planners to consider challenges faced by local operators (e.g., restricted access, potential dangers) when they are developing their response strategy. As the mission evolves, reviewing earlier and the most recent imaging data can serve as a tool for comparative analysis and for assessing the effectiveness of the current strategy. It can raise the decision-maker's awareness of constraints and possibilities, aid them in making informed decisions, and serve as an evaluation and re-planning tool. While potentially useful, past experience has shown that introducing technologies that change the type, amount, and nature of information available to a decision-maker can also have v Dedicated to my mother and father vi ACKNOWLEDGMENTS

Research paper thumbnail of How are we doing?: Presenting System Confidence Information to Support Trust Calibration and Adaptive Function Allocation

PsycEXTRA Dataset, 2003

The "strong but silent" design of many decision support systems (DSS) has contributed to problems... more The "strong but silent" design of many decision support systems (DSS) has contributed to problems such as automation bias and trust miscalibration. The present study examined whether these difficulties can be overcome by providing continually updated information regarding a system's confidence in its own ability to perform its task(s) accurately and reliably. The application domain for this research was in-flight icing. Two groups of pilots flew a motion-based simulator in simulated icing conditions and were assisted by a neural net-based DSS that detected, and identified the location of, ice accretion. One group of pilots received information about overall system reliability only, whereas a second group was presented with a trend display of system confidence. Pilots in the latter group were better calibrated in terms of when to follow the system's advice. They were also more likely to reverse their actions when system-recommended actions were unsuccessful. Consequently, they experienced fewer icing-induced stalls. The findings from this research will be discussed in terms of their implications for the design of decision aids and for adaptive function allocation in human-machine teams.

Research paper thumbnail of Effects of Real-time Imaging on Decision-Making in a Simulated Incident Command Task

International Journal of Information Systems for Crisis Response and Management, 2009

Eight Incident Commanders (ICs) took part in a simulation exercise to determine the impact of rea... more Eight Incident Commanders (ICs) took part in a simulation exercise to determine the impact of real-time imaging feedback on situation assessment and decision-making in an uncertain and high-tempo environment. The imaging feedback simulated the video feed from an unmanned aerial vehicle (UAV) that allows incident command centers to monitor developments at the crisis site. Nearly all of the ICs failed to detect important changes in the situation that were not captured in the imaging but that were available via other, more traditional data sources. It appears that the ICs placed an inappropriately high level of trust in the imaging data, resulting in a narrowing of their data search activities and limited cross-checking between the data sources being used. This research helps anticipate and guard against undesirable effects of introducing similar technologies on training and operational procedures in a variety of domains.

Research paper thumbnail of Presenting in-flight icing information: a comparison of visual and tactile cues

20th DASC. 20th Digital Avionics Systems Conference (Cat. No.01CH37219), 2001

In-flight icing continues to represent a major safety hazard for a large segment of the aviation ... more In-flight icing continues to represent a major safety hazard for a large segment of the aviation industry. A new system, the so-called Ice Management System (IMS), is currently under development to support pilots in detecting, diagnosing, and responding to icing conditions in a timely and appropriate manner. The IMS will provide crews with information about the onset and development of ice accretion as well as its effects on flight dynamics and aircraft performance [1]. The present study was conducted as part of the development of the pilot-IMS interface. It examines the effectiveness of presenting icing-related information via different sensory channels in an attempt to minimize cross-modal interference. In particular, it compares the use of tactile and visual cues for alerting pilots to the onset of ice accretion and for indicating the nature of the icing condition. Twenty-four pilots were assigned to one of three experimental groups. They received information on the onset, location (wing versus tail-plane), and severity (light, moderate, and severe) of ice accretion from one of two different visual displays or from a tactile interface. Pilots were asked to fly two 25-minute scenarios on a desktop simulator. The scenarios differed in terms of workload and the frequency of icing updates. Participants were asked to report the location and severity of ice accretion, and they had to indicate whether the most recent icing cues represented a change from the previous condition. Another secondary task involved monitoring for the occurrence of an out-of-range value on one of two oil pressure gauges. There was no significant difference between the three experimental conditions in terms of icing detection and identification. All three groups showed almost perfect performance on those two tasks. However, pilots in the tactile condition were much better able to recall and compare the previous and current icing condition. They also detected significantly more of the of oil pressure deviations and of icing cues that were presented simultaneously with air traffic control clearances. These results suggest that the introduction of tactile cues, and thus the distribution of information across sensory channels, represents a promising approach to providing pilots with icingrelated information. These cues do not differ from visual cues in terms of supporting the detection and identification of icing conditions; however, they help avoid the resource competition and scanning costs associated with adding yet more visual information to the flight deck.

Research paper thumbnail of Complex Social Systems: Rising Complexity in Business Environments

Attempts to simulate the effects of environmental complexity changes in business yielded the real... more Attempts to simulate the effects of environmental complexity changes in business yielded the realization that much exploratory work needs to be done to (a) demonstrate that complexity has increased universally, and (b) to elucidate what contributory factors are at work in this environment. Accordingly, the team decided to undertake exploratory work to see if universal factors that increase complexity could be identified. The method involved brainstorming ideas, formulating a hypothesis, and conducting a literature review and analysis. We conclude that the effects of globalization and the widespread adoption of technology are indeed universal factors that profoundly increase complexity. This may be considered a partial list, and much more work in this area remains to be done.

Research paper thumbnail of Robot-Assisted Versus Laparoscopic Gastric Bypass: Comparison of Short-Term Outcomes

Obesity Surgery, 2013

Roux-en-Y gastric bypass is an effective treatment for severe obesity and obesity-related comorbi... more Roux-en-Y gastric bypass is an effective treatment for severe obesity and obesity-related comorbidities. Presently, gastric bypass is performed most often laparoscopically, although a robotic-assisted procedure is the preferred approach for an increasing number of bariatric surgeons. This retrospective study compared the results of 100 Roux-en-Y gastric bypass operations using the da Vinci robot and 100 laparoscopic Roux-en-Y gastric bypasses performed laparoscopically. Short-term outcomes were determined by evaluating mortality, length of stay, length of operation, return to the operating room within 90 days of operation, conversions to open procedure, leaks, strictures, transfusions, and hospital readmissions. There was no mortality, pulmonary embolus, or conversion to open procedure in either group. Both the laparoscopic and robotic operative times decreased progressively, although the robotic operation time was longer (mean, 144 versus 87 min, P < 0.001). The length of stay was shorter for the robotic-assisted group (37 versus 52 h, P < 0.001), and 60% of these patients were discharged after one night's stay (P < 0.001). There were fewer transfusions (P = 0.005) and readmissions (P = .560) in the robotic group. The stricture rate was higher in the first 50 robotic procedures (17 mm gastrotomy) but resolved in the second 50 procedures (21 mm gastrotomy). There was no difference in the rate of leak and return to the operating room between groups (both P > 0.05). These results indicate that Roux-en-Y gastric bypass can be performed safely with robotic assistance, even during the first 100 cases.

Research paper thumbnail of Supporting Decision Making and Action Selection under Time Pressure and Uncertainty: The Case of In-Flight Icing

Human Factors: The Journal of the Human Factors and Ergonomics Society, 2001

Pilots often need to make decisions and select and execute appropriate actions under time pressur... more Pilots often need to make decisions and select and execute appropriate actions under time pressure and uncertainty. One possible way to support them in these tasks is the introduction of decision support systems. This manuscript describes a recent study that examined the benefits and disadvantages of two major forms of decision support, status and command displays, in the context of inflight icing. Status displays provide information on the current situation only and are intended to help the operator with the detection and diagnosis of a problem. Command displays, on the other hand, recommend appropriate responses to the problem and thus support primarily the action selection stage. Twenty-seven pilots, equally divided into three groups (baseline, status, and command), each flew a total of 20 approaches involving some form of inflight icing on a motion-base simulator. Factors such as the accuracy of the decision aid and pilots' familiarity with the icing situation were varied. The impact of these factors on pilots' performance in terms of detecting and handling the icing encounter, recovering from incorrect actions and assessments, and performing concurrent tasks (such as detection of a glideslope failure) was examined. Overall, accurate information from either type of decision aid led to better performance than the baseline condition. However, inaccurate information, especially when presented on the command display, resulted in considerable performance costs, even when traditional cues were present.

Research paper thumbnail of Seeing is believing?: The effects of real-time, image-based feedback on emergency management decision-making

Emergency management personnel often face feedback delays and a lack of reliable information. To ... more Emergency management personnel often face feedback delays and a lack of reliable information. To address this problem, new information technologies have been developed that can provide real-time, image-based feedback. While potentially useful, this trend represents a fundamental shift in both the timing and format of the information used by incident commanders (ICs). Eight ICs took part in a simulation exercise to determine the potential impact of real-time imaging on their decision-making. Nearly all of the ICs failed to detect important changes in the situation that were not captured in the imaging but that were available via other, more traditional data sources. It appears that the ICs placed an inappropriately high level of trust in the imaging data, resulting in reduced data search activities and hypothesis generation. This research helps practitioners anticipate and guard against undesirable effects of introducing similar technologies on training and operational procedures.

Research paper thumbnail of Supporting Trust Calibration and the Effective Use of Decision Aids by Presenting Dynamic System Confidence Information

Research paper thumbnail of Incident Command Situation Assessment Utilizing Video Feeds from UAVs: New Risks for Decision Making Breakdowns

Past experience has shown that introducing new Information Technologies can have unintended and u... more Past experience has shown that introducing new Information Technologies can have unintended and undesirable consequences, such as new forms of errors and a narrowing of data search activities. Eight Incident Commanders (ICs) took part in a simulated disaster response exercise to determine how the availability of real-time image feeds from a UAV impact on situation assessment and decision-making.
The exercise simulated the video feed from an unmanned aerial vehicle (UAV) that allows incident command centers to monitor developments at a crisis site. The results showed that information from the video image channel dominated information available from other channels or in other forms. Nearly all of the ICs failed to detect important changes in the situation that were not captured in the imaging channel but that were available via other, more traditional data sources. The dominance of the image feed resulted in ICs narrowing their data search activities and reducing cross-checking across diverse data sources. This study confirms anecdotal reports that users can over-rely on video feeds from UAVs.

Research paper thumbnail of Supporting trust calibration through the presentation of dynamic reliability information /

Research paper thumbnail of Seeing is believing?: The effects of real-time, image- based feedback on emergency management decision- making

Emergency management personnel often face feedback delays and a lack of reliable information. To ... more Emergency management personnel often face feedback delays and a lack of reliable information. To address this problem, new information technologies have been developed that can provide real-time, image-based feedback. While potentially useful, this trend represents a fundamental shift in both the timing and format of the information used by incident commanders (ICs). Eight ICs took part in a simulation exercise to determine the potential impact of real-time imaging on their decision-making. Nearly all of the ICs failed to detect important changes in the situation that were not captured in the imaging but that were available via other, more traditional data sources. It appears that the ICs placed an inappropriately high level of trust in the imaging data, resulting in reduced data search activities and hypothesis generation. This research helps practitioners anticipate and guard against undesirable effects of introducing similar technologies on training and operational procedures.

Research paper thumbnail of The Effects of Real-Time Image-Based Feedback on Data Gathering and Analysis: The Case of Emergency Management Decision Making

Several recent events, such as 9/11 and hurricane Katrina, have placed an increased emphasis on s... more Several recent events, such as 9/11 and hurricane Katrina, have placed an increased emphasis on supporting decision-makers during disaster recovery missions. To this end advanced information technologies have been developed that provide real-time feedback to practitioners who have traditionally been forced to operate in a feed-forward control mode due to a lack of timely information about the system they are controlling. In particular, a number of new remote sensing technologies have been introduced that present remotely located decision-makers with real-time imaging of the area of interest. This new data channel promises several benefits to decision-makers at all levels of command and control. It can aid in assessing and comprehending the overall situation while also allowing planners to consider challenges faced by local operators (e.g., restricted access, potential dangers) when they are developing their response strategy. As the mission evolves, reviewing earlier and the most recent imaging data can serve as a tool for comparative analysis and for assessing the effectiveness of the current strategy. It can raise the decision-maker's awareness of constraints and possibilities, aid them in making informed decisions, and serve as an evaluation and re-planning tool. While potentially useful, past experience has shown that introducing technologies that change the type, amount, and nature of information available to a decision-maker can also have v Dedicated to my mother and father vi ACKNOWLEDGMENTS

Research paper thumbnail of How are we doing?: Presenting System Confidence Information to Support Trust Calibration and Adaptive Function Allocation

PsycEXTRA Dataset, 2003

The "strong but silent" design of many decision support systems (DSS) has contributed to problems... more The "strong but silent" design of many decision support systems (DSS) has contributed to problems such as automation bias and trust miscalibration. The present study examined whether these difficulties can be overcome by providing continually updated information regarding a system's confidence in its own ability to perform its task(s) accurately and reliably. The application domain for this research was in-flight icing. Two groups of pilots flew a motion-based simulator in simulated icing conditions and were assisted by a neural net-based DSS that detected, and identified the location of, ice accretion. One group of pilots received information about overall system reliability only, whereas a second group was presented with a trend display of system confidence. Pilots in the latter group were better calibrated in terms of when to follow the system's advice. They were also more likely to reverse their actions when system-recommended actions were unsuccessful. Consequently, they experienced fewer icing-induced stalls. The findings from this research will be discussed in terms of their implications for the design of decision aids and for adaptive function allocation in human-machine teams.

Research paper thumbnail of Effects of Real-time Imaging on Decision-Making in a Simulated Incident Command Task

International Journal of Information Systems for Crisis Response and Management, 2009

Eight Incident Commanders (ICs) took part in a simulation exercise to determine the impact of rea... more Eight Incident Commanders (ICs) took part in a simulation exercise to determine the impact of real-time imaging feedback on situation assessment and decision-making in an uncertain and high-tempo environment. The imaging feedback simulated the video feed from an unmanned aerial vehicle (UAV) that allows incident command centers to monitor developments at the crisis site. Nearly all of the ICs failed to detect important changes in the situation that were not captured in the imaging but that were available via other, more traditional data sources. It appears that the ICs placed an inappropriately high level of trust in the imaging data, resulting in a narrowing of their data search activities and limited cross-checking between the data sources being used. This research helps anticipate and guard against undesirable effects of introducing similar technologies on training and operational procedures in a variety of domains.

Research paper thumbnail of Presenting in-flight icing information: a comparison of visual and tactile cues

20th DASC. 20th Digital Avionics Systems Conference (Cat. No.01CH37219), 2001

In-flight icing continues to represent a major safety hazard for a large segment of the aviation ... more In-flight icing continues to represent a major safety hazard for a large segment of the aviation industry. A new system, the so-called Ice Management System (IMS), is currently under development to support pilots in detecting, diagnosing, and responding to icing conditions in a timely and appropriate manner. The IMS will provide crews with information about the onset and development of ice accretion as well as its effects on flight dynamics and aircraft performance [1]. The present study was conducted as part of the development of the pilot-IMS interface. It examines the effectiveness of presenting icing-related information via different sensory channels in an attempt to minimize cross-modal interference. In particular, it compares the use of tactile and visual cues for alerting pilots to the onset of ice accretion and for indicating the nature of the icing condition. Twenty-four pilots were assigned to one of three experimental groups. They received information on the onset, location (wing versus tail-plane), and severity (light, moderate, and severe) of ice accretion from one of two different visual displays or from a tactile interface. Pilots were asked to fly two 25-minute scenarios on a desktop simulator. The scenarios differed in terms of workload and the frequency of icing updates. Participants were asked to report the location and severity of ice accretion, and they had to indicate whether the most recent icing cues represented a change from the previous condition. Another secondary task involved monitoring for the occurrence of an out-of-range value on one of two oil pressure gauges. There was no significant difference between the three experimental conditions in terms of icing detection and identification. All three groups showed almost perfect performance on those two tasks. However, pilots in the tactile condition were much better able to recall and compare the previous and current icing condition. They also detected significantly more of the of oil pressure deviations and of icing cues that were presented simultaneously with air traffic control clearances. These results suggest that the introduction of tactile cues, and thus the distribution of information across sensory channels, represents a promising approach to providing pilots with icingrelated information. These cues do not differ from visual cues in terms of supporting the detection and identification of icing conditions; however, they help avoid the resource competition and scanning costs associated with adding yet more visual information to the flight deck.

Research paper thumbnail of Complex Social Systems: Rising Complexity in Business Environments

Attempts to simulate the effects of environmental complexity changes in business yielded the real... more Attempts to simulate the effects of environmental complexity changes in business yielded the realization that much exploratory work needs to be done to (a) demonstrate that complexity has increased universally, and (b) to elucidate what contributory factors are at work in this environment. Accordingly, the team decided to undertake exploratory work to see if universal factors that increase complexity could be identified. The method involved brainstorming ideas, formulating a hypothesis, and conducting a literature review and analysis. We conclude that the effects of globalization and the widespread adoption of technology are indeed universal factors that profoundly increase complexity. This may be considered a partial list, and much more work in this area remains to be done.

Research paper thumbnail of Robot-Assisted Versus Laparoscopic Gastric Bypass: Comparison of Short-Term Outcomes

Obesity Surgery, 2013

Roux-en-Y gastric bypass is an effective treatment for severe obesity and obesity-related comorbi... more Roux-en-Y gastric bypass is an effective treatment for severe obesity and obesity-related comorbidities. Presently, gastric bypass is performed most often laparoscopically, although a robotic-assisted procedure is the preferred approach for an increasing number of bariatric surgeons. This retrospective study compared the results of 100 Roux-en-Y gastric bypass operations using the da Vinci robot and 100 laparoscopic Roux-en-Y gastric bypasses performed laparoscopically. Short-term outcomes were determined by evaluating mortality, length of stay, length of operation, return to the operating room within 90 days of operation, conversions to open procedure, leaks, strictures, transfusions, and hospital readmissions. There was no mortality, pulmonary embolus, or conversion to open procedure in either group. Both the laparoscopic and robotic operative times decreased progressively, although the robotic operation time was longer (mean, 144 versus 87 min, P < 0.001). The length of stay was shorter for the robotic-assisted group (37 versus 52 h, P < 0.001), and 60% of these patients were discharged after one night's stay (P < 0.001). There were fewer transfusions (P = 0.005) and readmissions (P = .560) in the robotic group. The stricture rate was higher in the first 50 robotic procedures (17 mm gastrotomy) but resolved in the second 50 procedures (21 mm gastrotomy). There was no difference in the rate of leak and return to the operating room between groups (both P > 0.05). These results indicate that Roux-en-Y gastric bypass can be performed safely with robotic assistance, even during the first 100 cases.

Research paper thumbnail of Supporting Decision Making and Action Selection under Time Pressure and Uncertainty: The Case of In-Flight Icing

Human Factors: The Journal of the Human Factors and Ergonomics Society, 2001

Pilots often need to make decisions and select and execute appropriate actions under time pressur... more Pilots often need to make decisions and select and execute appropriate actions under time pressure and uncertainty. One possible way to support them in these tasks is the introduction of decision support systems. This manuscript describes a recent study that examined the benefits and disadvantages of two major forms of decision support, status and command displays, in the context of inflight icing. Status displays provide information on the current situation only and are intended to help the operator with the detection and diagnosis of a problem. Command displays, on the other hand, recommend appropriate responses to the problem and thus support primarily the action selection stage. Twenty-seven pilots, equally divided into three groups (baseline, status, and command), each flew a total of 20 approaches involving some form of inflight icing on a motion-base simulator. Factors such as the accuracy of the decision aid and pilots' familiarity with the icing situation were varied. The impact of these factors on pilots' performance in terms of detecting and handling the icing encounter, recovering from incorrect actions and assessments, and performing concurrent tasks (such as detection of a glideslope failure) was examined. Overall, accurate information from either type of decision aid led to better performance than the baseline condition. However, inaccurate information, especially when presented on the command display, resulted in considerable performance costs, even when traditional cues were present.

Research paper thumbnail of Seeing is believing?: The effects of real-time, image-based feedback on emergency management decision-making

Emergency management personnel often face feedback delays and a lack of reliable information. To ... more Emergency management personnel often face feedback delays and a lack of reliable information. To address this problem, new information technologies have been developed that can provide real-time, image-based feedback. While potentially useful, this trend represents a fundamental shift in both the timing and format of the information used by incident commanders (ICs). Eight ICs took part in a simulation exercise to determine the potential impact of real-time imaging on their decision-making. Nearly all of the ICs failed to detect important changes in the situation that were not captured in the imaging but that were available via other, more traditional data sources. It appears that the ICs placed an inappropriately high level of trust in the imaging data, resulting in reduced data search activities and hypothesis generation. This research helps practitioners anticipate and guard against undesirable effects of introducing similar technologies on training and operational procedures.

Research paper thumbnail of Supporting Trust Calibration and the Effective Use of Decision Aids by Presenting Dynamic System Confidence Information

Research paper thumbnail of Incident Command Situation Assessment Utilizing Video Feeds from UAVs: New Risks for Decision Making Breakdowns

Past experience has shown that introducing new Information Technologies can have unintended and u... more Past experience has shown that introducing new Information Technologies can have unintended and undesirable consequences, such as new forms of errors and a narrowing of data search activities. Eight Incident Commanders (ICs) took part in a simulated disaster response exercise to determine how the availability of real-time image feeds from a UAV impact on situation assessment and decision-making.
The exercise simulated the video feed from an unmanned aerial vehicle (UAV) that allows incident command centers to monitor developments at a crisis site. The results showed that information from the video image channel dominated information available from other channels or in other forms. Nearly all of the ICs failed to detect important changes in the situation that were not captured in the imaging channel but that were available via other, more traditional data sources. The dominance of the image feed resulted in ICs narrowing their data search activities and reducing cross-checking across diverse data sources. This study confirms anecdotal reports that users can over-rely on video feeds from UAVs.