Chris W Schwarz | The University of Iowa (original) (raw)
Papers by Chris W Schwarz
Proceedings of the 8th International Driving Symposium on Human Factors in Driver Assessment, Training, and Vehicle Design: driving assessment 2015, 2015
With the introduction of more advanced vehicle technology, it is paramount to assess its safety b... more With the introduction of more advanced vehicle technology, it is paramount to assess its safety benefit. Advanced driver assistance systems (ADAS) can reduce crashes and mitigate crash severity, if designed appropriately. Driver behavior models are integral to the ADAS design process, complementing time and resource intensive human participant experiments. We introduce a method to model driver responses to forward collision events by quantifying multivariate behavior with copulas and Monte Carlo simulation. This approach capitalizes on the data from small samples of crash events observed in naturalistic or simulator studies. Copulas summarize data by capturing the underlying joint distribution of variables, and Monte Carlo methods can be used to repeatedly sample from these distributions. A driver model can be parameterized with these samples, and run on a desktop driving simulation environment.
Traffic Injury Prevention
Objective: Drowsiness is a major cause of driver impairment leading to crashes and fatalities. Re... more Objective: Drowsiness is a major cause of driver impairment leading to crashes and fatalities. Research has established the ability to detect drowsiness with various kinds of sensors. We studied drowsy driving in a high-fidelity driving simulator and evaluated the ability of an automotive production-ready driver monitoring system (DMS) to detect drowsy driving. Additionally, this feature was compared to and combined with signals from vehicle-based sensors. Methods: The National Advanced Driving Simulator was used to expose drivers to long, monotonous drives. Twenty participants drove for about 4 h in the simulator between 10 p.m. and 2 a.m. They were allowed to use cruise control and traffic was sparse and semirandom, with both slowerand faster-moving vehicles. Observational ratings of drowsiness (ORDs) were used as the ground truth for drowsiness, and several dependent measures were calculated from vehicle and DMS signals. Drowsiness classification models were created that used only vehicle signals, only driver monitoring signals, and a combination of the 2 sources. Results: The model that used DMS signals performed better than the one that used only vehicle signals; however, the combination of the two performed the best. The models were effective at discriminating low levels of drowsiness from moderate to severe drowsiness; however, they were not effective at telling the difference between moderate and severe levels. A binary model that lumped drowsiness into 2 classes had an area under the receiver operating characteristic (ROC) curve of 0.897. Conclusions: Blinks and saccades have been shown to be predictive of microsleeps; however, it may be that detection of microsleeps and lane departures occurs too late. Therefore, it is encouraging that the model was able to distinguish mild from moderate drowsy driving. The use of automation may make vehicle-based signals useless for characterizing driver states, providing further motivation for a DMS. Future improvements in impairment detection systems may be expected through a combination of improved hardware, physiological measures from unobtrusive sensors and wearables, and the intelligent integration of environmental variables like time of day and time on task.
Cognition, Technology & Work
Accident; analysis and prevention, 2018
This study designs and evaluates a contextual and temporal algorithm for detecting drowsiness-rel... more This study designs and evaluates a contextual and temporal algorithm for detecting drowsiness-related lane. The algorithm uses steering angle, pedal input, vehicle speed and acceleration as input. Speed and acceleration are used to develop a real-time measure of driving context. These measures are integrated with a Dynamic Bayesian Network that considers the time dependencies in transitions between drowsiness and awake states. The Dynamic Bayesian Network algorithm is validated with data collected from 72 participants driving the National Advanced Driving Simulator. The algorithm has a significantly lower false positive rate than PERCLOS-the current gold standard-and baseline, non-contextual, algorithms under design parameters that prioritize drowsiness detection. Under these parameters, the algorithm reduces false positive rate in highway and rural environments, which are typically problematic for vehicle-based detection algorithms. This algorithm is a promising new approach to dri...
Accident; analysis and prevention, Jan 22, 2017
Advanced driver assistance systems (ADAS) have the potential to prevent crashes and reduce their ... more Advanced driver assistance systems (ADAS) have the potential to prevent crashes and reduce their severity. Forward collision warnings (FCW) are quickly becoming standard across vehicle lineups and may prevent frontal crashes by alerting drivers. Previous research has demonstrated the effectiveness of FCW for distracted drivers, but their effectiveness for other types of impairment remains unknown. Like distraction, drowsiness can impair driver response time and lead to crashes. The goal of the present study was to evaluate the effectiveness of FCW for moderately and severely drowsy drivers using a high-fidelity driving simulator. Drowsy drivers were divided into three warning conditions during a revealed stop vehicle forward collision event: An auditory alert, a haptic seat vibration, and a no warning baseline. Results indicate that FCW were effective at speeding drowsy driver response, but only when the drowsy drivers were looking away from the forward roadway at the onset of the e...
Transportation Research Record: Journal of the Transportation Research Board
Traffic injury prevention, May 21, 2017
Objective Driver drowsiness contributes to a substantial number of fatal and non-fatal crashes, w... more Objective Driver drowsiness contributes to a substantial number of fatal and non-fatal crashes, with recent estimates attributing up to 21% of fatal crashes to drowsiness. This paper describes recent NHTSA research on in-vehicle drowsiness countermeasures. Recent advances in technology and state detection algorithms have shown success in detecting drowsiness using a variety of data sources, including camera-based eye tracking, steering wheel position, yaw rate, and vehicle lane position. However, detection is just the first step in reducing drowsy driving crashes. Countermeasures are also needed to provide feedback to the driver, modify driver behavior, and prevent crashes. The goal of this study was to evaluate the effectiveness of in-vehicle drowsiness countermeasures in reducing drowsy lane departures. The tested countermeasures included different warning modalities in either a discrete or staged interface. Methods Data were collected from 72 young adult drivers (age 21-32) in th...
Transportation Research Part F: Traffic Psychology and Behaviour, 2014
ABSTRACT Time to collision (TTC) has been a key vehicle safety metric for decades. With the incre... more ABSTRACT Time to collision (TTC) has been a key vehicle safety metric for decades. With the increasing prevalence of advanced driver assistance systems and vehicle automation, TTC and many related metrics are being applied to the analysis of more complicated scenarios, as well as being integrated into automation algorithms. While the TTC metric was originally conceived to be inclusive of generic two-dimensional situations, its applications has been mostly limited to one-dimensional scenarios. This paper derives general equations and algorithms using two-dimensional information. Additionally, methods from computational geometry, a field that didn’t exist when TTC was first used, are employed for the general case of computing TTC between bounding boxes. Parametric equations for lines play a prominent role and offer an elegant way to express the geometry of the scenarios described in this paper. Throughout, the approach is not to derive specific algebraic conditions as in previous efforts. Rather, the focus in on developing general algorithms for computation. The techniques presented are not necessary for traditional car following scenarios; but offer options for more complex situations that trade off analytic solutions for computational flexibility.
Handbook of Driving Simulation for Engineering, Medicine, and Psychology, 2011
SAE Technical Paper Series, 2016
SAE Technical Paper Series, 2016
We are moving towards an age of autonomous vehicles. Cycles of innovation initiated in the public... more We are moving towards an age of autonomous vehicles. Cycles of innovation initiated in the public and private sectors have led one into another since the 1990s; and out of these efforts have sprung a variety of Advanced Driver Assistance Systems and several functioning autonomous vehicles. The challenges that face autonomous vehicle are still significant. There is still technical work to be done to make sensors, algorithms, control schemes, and intelligence more effective and more reliable. As automation in vehicles increases, the associated human factors challenges become more complex. Then, there are a host of socioeconomic issues. Are autonomous vehicles legal; and who is liable if one crashes? How can we ensure privacy and security of data and automation systems? Finally, how might the wide adoption of autonomous vehicles affect society at large? It is hoped that when they appear, they will bring with them the promised benefits of safety, mobility, efficiency, and societal change.
SAE Technical Paper Series, 2007
In order to remain on the forefront of driving simulation technology, it is necessary to continua... more In order to remain on the forefront of driving simulation technology, it is necessary to continually upgrade and expand the capabilities of the NADS. One extension that is suggested by recent trends in the automotive marketplace is the ability to simulate hybrid electric vehicles. Support of hybrid electric vehicles by the NADS is motivated by the engineering and human factors challenges of designing and testing advanced concepts for vehicles and user interfaces. This paper discusses the extension of the vehicle dynamics to include a split hybrid electric four wheel drive power train. The new power train is based on Daimler Chrysler's Dodge Durango TTR hybrid. The vehicle dynamics subsystem is based on the University of Iowa's real time recursive dynamics (RTRD). The RTRD enables efficient simulation of general rigid multi-body systems. Additional force generating subroutines are included to model an extensive array of vehicle subsystems, such as power train, steering, brakes, aerodynamics, and tires. The new hybrid power train design includes new electromechanical component models, which are described in detail. A discussion of appropriate applications of the models based on their fidelity is included. Models of electromechanical components can bring with them greater bandwidth requirements. Bandwidth/performance tradeoffs are discussed; and suitable models for driver-in-the-loop simulation are selected.
With the introduction of more advanced vehicle technology, it is paramount to assess its safety b... more With the introduction of more advanced vehicle technology, it is paramount to assess its safety benefit. Advanced driver assistance systems (ADAS) can reduce crashes and mitigate crash severity, if designed appropriately. Driver behavior models are integral to the ADAS design process, complementing time and resource intensive human participant experiments. We introduce a method to model driver responses to forward collision events by quantifying multivariate behavior with copulas and Monte Carlo simulation. This approach capitalizes on the data from small samples of crash events observed in naturalistic or simulator studies. Copulas summarize data by capturing the underlying joint distribution of variables, and Monte Carlo methods can be used to repeatedly sample from these distributions. A driver model can be parametrized with these samples, and run on a desktop driving simulation environment.
A simulator fidelity study was performed on the NADS that compared drives with and without motion... more A simulator fidelity study was performed on the NADS that compared drives with and without motion and scenarios with varying levels of visual fidelity. Simulator sickness levels were higher than average, resulting in above average drop-out rates. Though the goal of the study was not to study sim sickness, the factors in the study make it an interesting and informative case study. The occurrence of simulator sickness is complicated; but one generally accepted reason for sim sickness is cue mismatch between the vestibular and ocular systems. Also important are aspects of the visual system such as field-of-view. However, driver susceptibility is also an important factor that causes some people to become symptomatic very quickly, while others never present symptoms. We consider the presence of motion cues, the visual scene, the sickness scores, and the final disposition of the driver (dropped out or completed); and look at several driver performance dependent measures. The goal was to determine if and how the presence of sickness affects driver inputs and other measures of driver performance. This paper presents some interesting and significant findings.
Traffic injury prevention, 2015
In 2012 in the United States, pedestrian injuries accounted for 3.3% of all traffic injuries but,... more In 2012 in the United States, pedestrian injuries accounted for 3.3% of all traffic injuries but, disproportionately, pedestrian fatalities accounted for roughly 14% of traffic-related deaths (NHTSA 2014 ). In many other countries, pedestrians make up more than 50% of those injured and killed in crashes. This research project examined driver response to crash-imminent situations involving pedestrians in a high-fidelity, full-motion driving simulator. This article presents a scenario development method and discusses experimental design and control issues in conducting pedestrian crash research in a simulation environment. Driving simulators offer a safe environment in which to test driver response and offer the advantage of having virtual pedestrian models that move realistically, unlike test track studies, which by nature must use pedestrian dummies on some moving track. An analysis of pedestrian crash trajectories, speeds, roadside features, and pedestrian behavior was used to crea...
The risk of drivers engaging in distracting activies is increasing as in-vehicle technology and c... more The risk of drivers engaging in distracting activies is increasing as in-vehicle technology and carried-in devices become increasingly common and complicated. Consequently, distraction and inattention contribute to crash risk and are likely to have an increasing influence on driving safety. Analysis of police-reported crash data from 2008 showed that distractions contributed to an estimated 5,870 fatalities and 515,000 injuries. This paper assesses the extent to which vision-based algorithms can detect different types of driver distraction under different driving conditions. Data were collected on the National Advanced Driving Simulator from 32 volunteer drivers between the ages of 25 and 50. Participants drove through representative situations on three types of roadways (urban, freeway, and rural) twice: once with and once without distraction tasks. The order of the drives was counterbalanced. The three distraction tasks included a reaching task, a visual-manual task and a cognitive task which were repeated eight times throughout the drive. Four different vision-based algorithms were evaluated. All of them performed significantly better than chance (random) performance. There was little difference between the approaches for the visual-manual bug task which required the most eyes-offroad time. The algorithm that estimated level of distraction by combining percent of glances to the road and long glances away from the road performed best for the arrows task, and was also the only algorithm that detected cognitive impairment. Differences across road types were also observed. Trade-offs exist between ensuring distraction detection and avoiding false alarms that complicate determining the most promising algorithm for detecting distraction. The differences in the algorithms' abilities across evaluation criteria, road type, and distraction task type demonstrate critical trade-offs in capabilities that need to be considered. Depending on how feedback is presented to drivers, high false alarm rates may undermine drivers' acceptance of the system. The study shows the importance of designing and testing algorithms with a variety of challenges to assess performance across a range of representative road and task types.
Proceedings of the 8th International Driving Symposium on Human Factors in Driver Assessment, Training, and Vehicle Design: driving assessment 2015, 2015
With the introduction of more advanced vehicle technology, it is paramount to assess its safety b... more With the introduction of more advanced vehicle technology, it is paramount to assess its safety benefit. Advanced driver assistance systems (ADAS) can reduce crashes and mitigate crash severity, if designed appropriately. Driver behavior models are integral to the ADAS design process, complementing time and resource intensive human participant experiments. We introduce a method to model driver responses to forward collision events by quantifying multivariate behavior with copulas and Monte Carlo simulation. This approach capitalizes on the data from small samples of crash events observed in naturalistic or simulator studies. Copulas summarize data by capturing the underlying joint distribution of variables, and Monte Carlo methods can be used to repeatedly sample from these distributions. A driver model can be parameterized with these samples, and run on a desktop driving simulation environment.
Traffic Injury Prevention
Objective: Drowsiness is a major cause of driver impairment leading to crashes and fatalities. Re... more Objective: Drowsiness is a major cause of driver impairment leading to crashes and fatalities. Research has established the ability to detect drowsiness with various kinds of sensors. We studied drowsy driving in a high-fidelity driving simulator and evaluated the ability of an automotive production-ready driver monitoring system (DMS) to detect drowsy driving. Additionally, this feature was compared to and combined with signals from vehicle-based sensors. Methods: The National Advanced Driving Simulator was used to expose drivers to long, monotonous drives. Twenty participants drove for about 4 h in the simulator between 10 p.m. and 2 a.m. They were allowed to use cruise control and traffic was sparse and semirandom, with both slowerand faster-moving vehicles. Observational ratings of drowsiness (ORDs) were used as the ground truth for drowsiness, and several dependent measures were calculated from vehicle and DMS signals. Drowsiness classification models were created that used only vehicle signals, only driver monitoring signals, and a combination of the 2 sources. Results: The model that used DMS signals performed better than the one that used only vehicle signals; however, the combination of the two performed the best. The models were effective at discriminating low levels of drowsiness from moderate to severe drowsiness; however, they were not effective at telling the difference between moderate and severe levels. A binary model that lumped drowsiness into 2 classes had an area under the receiver operating characteristic (ROC) curve of 0.897. Conclusions: Blinks and saccades have been shown to be predictive of microsleeps; however, it may be that detection of microsleeps and lane departures occurs too late. Therefore, it is encouraging that the model was able to distinguish mild from moderate drowsy driving. The use of automation may make vehicle-based signals useless for characterizing driver states, providing further motivation for a DMS. Future improvements in impairment detection systems may be expected through a combination of improved hardware, physiological measures from unobtrusive sensors and wearables, and the intelligent integration of environmental variables like time of day and time on task.
Cognition, Technology & Work
Accident; analysis and prevention, 2018
This study designs and evaluates a contextual and temporal algorithm for detecting drowsiness-rel... more This study designs and evaluates a contextual and temporal algorithm for detecting drowsiness-related lane. The algorithm uses steering angle, pedal input, vehicle speed and acceleration as input. Speed and acceleration are used to develop a real-time measure of driving context. These measures are integrated with a Dynamic Bayesian Network that considers the time dependencies in transitions between drowsiness and awake states. The Dynamic Bayesian Network algorithm is validated with data collected from 72 participants driving the National Advanced Driving Simulator. The algorithm has a significantly lower false positive rate than PERCLOS-the current gold standard-and baseline, non-contextual, algorithms under design parameters that prioritize drowsiness detection. Under these parameters, the algorithm reduces false positive rate in highway and rural environments, which are typically problematic for vehicle-based detection algorithms. This algorithm is a promising new approach to dri...
Accident; analysis and prevention, Jan 22, 2017
Advanced driver assistance systems (ADAS) have the potential to prevent crashes and reduce their ... more Advanced driver assistance systems (ADAS) have the potential to prevent crashes and reduce their severity. Forward collision warnings (FCW) are quickly becoming standard across vehicle lineups and may prevent frontal crashes by alerting drivers. Previous research has demonstrated the effectiveness of FCW for distracted drivers, but their effectiveness for other types of impairment remains unknown. Like distraction, drowsiness can impair driver response time and lead to crashes. The goal of the present study was to evaluate the effectiveness of FCW for moderately and severely drowsy drivers using a high-fidelity driving simulator. Drowsy drivers were divided into three warning conditions during a revealed stop vehicle forward collision event: An auditory alert, a haptic seat vibration, and a no warning baseline. Results indicate that FCW were effective at speeding drowsy driver response, but only when the drowsy drivers were looking away from the forward roadway at the onset of the e...
Transportation Research Record: Journal of the Transportation Research Board
Traffic injury prevention, May 21, 2017
Objective Driver drowsiness contributes to a substantial number of fatal and non-fatal crashes, w... more Objective Driver drowsiness contributes to a substantial number of fatal and non-fatal crashes, with recent estimates attributing up to 21% of fatal crashes to drowsiness. This paper describes recent NHTSA research on in-vehicle drowsiness countermeasures. Recent advances in technology and state detection algorithms have shown success in detecting drowsiness using a variety of data sources, including camera-based eye tracking, steering wheel position, yaw rate, and vehicle lane position. However, detection is just the first step in reducing drowsy driving crashes. Countermeasures are also needed to provide feedback to the driver, modify driver behavior, and prevent crashes. The goal of this study was to evaluate the effectiveness of in-vehicle drowsiness countermeasures in reducing drowsy lane departures. The tested countermeasures included different warning modalities in either a discrete or staged interface. Methods Data were collected from 72 young adult drivers (age 21-32) in th...
Transportation Research Part F: Traffic Psychology and Behaviour, 2014
ABSTRACT Time to collision (TTC) has been a key vehicle safety metric for decades. With the incre... more ABSTRACT Time to collision (TTC) has been a key vehicle safety metric for decades. With the increasing prevalence of advanced driver assistance systems and vehicle automation, TTC and many related metrics are being applied to the analysis of more complicated scenarios, as well as being integrated into automation algorithms. While the TTC metric was originally conceived to be inclusive of generic two-dimensional situations, its applications has been mostly limited to one-dimensional scenarios. This paper derives general equations and algorithms using two-dimensional information. Additionally, methods from computational geometry, a field that didn’t exist when TTC was first used, are employed for the general case of computing TTC between bounding boxes. Parametric equations for lines play a prominent role and offer an elegant way to express the geometry of the scenarios described in this paper. Throughout, the approach is not to derive specific algebraic conditions as in previous efforts. Rather, the focus in on developing general algorithms for computation. The techniques presented are not necessary for traditional car following scenarios; but offer options for more complex situations that trade off analytic solutions for computational flexibility.
Handbook of Driving Simulation for Engineering, Medicine, and Psychology, 2011
SAE Technical Paper Series, 2016
SAE Technical Paper Series, 2016
We are moving towards an age of autonomous vehicles. Cycles of innovation initiated in the public... more We are moving towards an age of autonomous vehicles. Cycles of innovation initiated in the public and private sectors have led one into another since the 1990s; and out of these efforts have sprung a variety of Advanced Driver Assistance Systems and several functioning autonomous vehicles. The challenges that face autonomous vehicle are still significant. There is still technical work to be done to make sensors, algorithms, control schemes, and intelligence more effective and more reliable. As automation in vehicles increases, the associated human factors challenges become more complex. Then, there are a host of socioeconomic issues. Are autonomous vehicles legal; and who is liable if one crashes? How can we ensure privacy and security of data and automation systems? Finally, how might the wide adoption of autonomous vehicles affect society at large? It is hoped that when they appear, they will bring with them the promised benefits of safety, mobility, efficiency, and societal change.
SAE Technical Paper Series, 2007
In order to remain on the forefront of driving simulation technology, it is necessary to continua... more In order to remain on the forefront of driving simulation technology, it is necessary to continually upgrade and expand the capabilities of the NADS. One extension that is suggested by recent trends in the automotive marketplace is the ability to simulate hybrid electric vehicles. Support of hybrid electric vehicles by the NADS is motivated by the engineering and human factors challenges of designing and testing advanced concepts for vehicles and user interfaces. This paper discusses the extension of the vehicle dynamics to include a split hybrid electric four wheel drive power train. The new power train is based on Daimler Chrysler's Dodge Durango TTR hybrid. The vehicle dynamics subsystem is based on the University of Iowa's real time recursive dynamics (RTRD). The RTRD enables efficient simulation of general rigid multi-body systems. Additional force generating subroutines are included to model an extensive array of vehicle subsystems, such as power train, steering, brakes, aerodynamics, and tires. The new hybrid power train design includes new electromechanical component models, which are described in detail. A discussion of appropriate applications of the models based on their fidelity is included. Models of electromechanical components can bring with them greater bandwidth requirements. Bandwidth/performance tradeoffs are discussed; and suitable models for driver-in-the-loop simulation are selected.
With the introduction of more advanced vehicle technology, it is paramount to assess its safety b... more With the introduction of more advanced vehicle technology, it is paramount to assess its safety benefit. Advanced driver assistance systems (ADAS) can reduce crashes and mitigate crash severity, if designed appropriately. Driver behavior models are integral to the ADAS design process, complementing time and resource intensive human participant experiments. We introduce a method to model driver responses to forward collision events by quantifying multivariate behavior with copulas and Monte Carlo simulation. This approach capitalizes on the data from small samples of crash events observed in naturalistic or simulator studies. Copulas summarize data by capturing the underlying joint distribution of variables, and Monte Carlo methods can be used to repeatedly sample from these distributions. A driver model can be parametrized with these samples, and run on a desktop driving simulation environment.
A simulator fidelity study was performed on the NADS that compared drives with and without motion... more A simulator fidelity study was performed on the NADS that compared drives with and without motion and scenarios with varying levels of visual fidelity. Simulator sickness levels were higher than average, resulting in above average drop-out rates. Though the goal of the study was not to study sim sickness, the factors in the study make it an interesting and informative case study. The occurrence of simulator sickness is complicated; but one generally accepted reason for sim sickness is cue mismatch between the vestibular and ocular systems. Also important are aspects of the visual system such as field-of-view. However, driver susceptibility is also an important factor that causes some people to become symptomatic very quickly, while others never present symptoms. We consider the presence of motion cues, the visual scene, the sickness scores, and the final disposition of the driver (dropped out or completed); and look at several driver performance dependent measures. The goal was to determine if and how the presence of sickness affects driver inputs and other measures of driver performance. This paper presents some interesting and significant findings.
Traffic injury prevention, 2015
In 2012 in the United States, pedestrian injuries accounted for 3.3% of all traffic injuries but,... more In 2012 in the United States, pedestrian injuries accounted for 3.3% of all traffic injuries but, disproportionately, pedestrian fatalities accounted for roughly 14% of traffic-related deaths (NHTSA 2014 ). In many other countries, pedestrians make up more than 50% of those injured and killed in crashes. This research project examined driver response to crash-imminent situations involving pedestrians in a high-fidelity, full-motion driving simulator. This article presents a scenario development method and discusses experimental design and control issues in conducting pedestrian crash research in a simulation environment. Driving simulators offer a safe environment in which to test driver response and offer the advantage of having virtual pedestrian models that move realistically, unlike test track studies, which by nature must use pedestrian dummies on some moving track. An analysis of pedestrian crash trajectories, speeds, roadside features, and pedestrian behavior was used to crea...
The risk of drivers engaging in distracting activies is increasing as in-vehicle technology and c... more The risk of drivers engaging in distracting activies is increasing as in-vehicle technology and carried-in devices become increasingly common and complicated. Consequently, distraction and inattention contribute to crash risk and are likely to have an increasing influence on driving safety. Analysis of police-reported crash data from 2008 showed that distractions contributed to an estimated 5,870 fatalities and 515,000 injuries. This paper assesses the extent to which vision-based algorithms can detect different types of driver distraction under different driving conditions. Data were collected on the National Advanced Driving Simulator from 32 volunteer drivers between the ages of 25 and 50. Participants drove through representative situations on three types of roadways (urban, freeway, and rural) twice: once with and once without distraction tasks. The order of the drives was counterbalanced. The three distraction tasks included a reaching task, a visual-manual task and a cognitive task which were repeated eight times throughout the drive. Four different vision-based algorithms were evaluated. All of them performed significantly better than chance (random) performance. There was little difference between the approaches for the visual-manual bug task which required the most eyes-offroad time. The algorithm that estimated level of distraction by combining percent of glances to the road and long glances away from the road performed best for the arrows task, and was also the only algorithm that detected cognitive impairment. Differences across road types were also observed. Trade-offs exist between ensuring distraction detection and avoiding false alarms that complicate determining the most promising algorithm for detecting distraction. The differences in the algorithms' abilities across evaluation criteria, road type, and distraction task type demonstrate critical trade-offs in capabilities that need to be considered. Depending on how feedback is presented to drivers, high false alarm rates may undermine drivers' acceptance of the system. The study shows the importance of designing and testing algorithms with a variety of challenges to assess performance across a range of representative road and task types.