Yasmeen Abdrabou | Universität der Bundeswehr München (original) (raw)

Papers by Yasmeen Abdrabou

Proceedings of the ACM on human-computer interaction, May 17, 2023

Although eye tracking brings many benefits to users of mobile devices and developers of mobile ap... more Although eye tracking brings many benefits to users of mobile devices and developers of mobile applications, it poses significant privacy risks to both: the users of mobile devices, and the bystanders that surround users, are within the front-facing camera's field of view. Recent research demonstrates that tracking an individual's gaze reveals personal and sensitive information. This paper presents an investigation of the privacy perceptions and the subjective acceptance of users towards eye tracking on handheld mobile devices. In a four-phase user study (N=17), participants used a smartphone eye tracking app, were interviewed before and after viewing a video showing the amount of sensitive and personal data that could be derived from eye movements, and had their privacy concerns measured. Our findings 1) show factors that influence users' and bystanders' attitudes toward eye tracking on mobile devices such as the algorithms' transparency and the developers' credibility and 2) support designing mechanisms to allow for privacy-aware eye tracking solutions on mobile-devices. CCS Concepts: • Security and privacy → Human and societal aspects of security and privacy; • Human-centered computing → Empirical studies in HCI.

arXiv (Cornell University), May 26, 2023

Users are the last line of defense as phishing emails pass filter mechanisms. At the same time, p... more Users are the last line of defense as phishing emails pass filter mechanisms. At the same time, phishing emails are designed so that they are challenging to identify by users. To this end, attackers employ techniques, such as eliciting stress, targeting helpfulness, or exercising authority, due to which users often miss being manipulated out of malicious intent. This work builds on the assumption that manipulation techniques, even if going unnoticed by users, still lead to changes in their behavior. In this work, we present the outcomes of an online study in which we collected gaze and mouse movement data during an email sorting task. Our findings show that phishing emails lead to significant differences across behavioral features but depend on the nature of the email. We discuss how our findings can be leveraged to build security mechanisms protecting users and companies from phishing.

Lecture Notes in Computer Science

Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Proceedings of the 2nd Empathy-Centric Design Workshop

Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Figure 1: We evaluate the performance of the three widely used gaze-based interaction methods: Dw... more Figure 1: We evaluate the performance of the three widely used gaze-based interaction methods: Dwell time (A), Pursuits (B) and Gaze gestures (C), for target selections on handheld mobile devices while sitting (left) and while walking (right). All participants performed all selections using the three different techniques while sitting and while walking. The red arrow in (B) illustrates the direction in which a yellow dot stimuli was rotating around a selectable target. The red arrows in (C) indicate the directions in which the user could perform a gaze gesture. All arrows are for illustration and were not shown to participants.

Proceedings 2023 Symposium on Usable Security

We propose an approach to identify users' exposure to fake news from users' gaze and mouse moveme... more We propose an approach to identify users' exposure to fake news from users' gaze and mouse movement behavior. Our approach is meant as an enabler for interventions that make users aware of engaging with fake news while not being consciously aware of this. Our work is motivated by the rapid spread of fake news on the web (in particular, social media) and the difficulty and effort required to identify fake content, either technically or by means of a human fact checker. To this end, we set out with conducting a remote online study (N = 54) in which participants were exposed to real and fake social media posts while their mouse and gaze movements were recorded. We identify the most predictive gaze and mouse movement features and show that fake news can be predicted with 68.4% accuracy from users' gaze and mouse movement behavior. Our work is complemented by discussing the implications of using behavioral features for mitigating the spread of fake news on social media.

Proceedings of the 2022 International Conference on Advanced Visual Interfaces

In this work, we explore attacker behavior during shoulder surfing. As such behavior is often opp... more In this work, we explore attacker behavior during shoulder surfing. As such behavior is often opportunistic and difficult to observe in real world settings, we leverage the capabilities of virtual reality (VR). We recruited 24 participants and observed their behavior in two virtual waiting scenarios: at a bus stop and in an open office space. In both scenarios, participants shoulder surfed private screens displaying different types of content. From the results we derive an understanding of factors influencing shoulder surfing behavior, reveal common attack patterns, and sketch a behavioral shoulder surfing model. Our work suggests directions for future research on shoulder surfing and can serve as a basis for creating novel approaches to mitigate shoulder surfing. CCS CONCEPTS • Security and privacy → Human and societal aspects of security and privacy;

CHI Conference on Human Factors in Computing Systems, Apr 29, 2022

A significant drawback of text passwords for end-user authentication is password reuse. We propos... more A significant drawback of text passwords for end-user authentication is password reuse. We propose a novel approach to detect password reuse by leveraging gaze as well as typing behavior and study its accuracy. We collected gaze and typing behavior from 49 users while creating accounts for 1) a webmail client and 2) a news website. While most participants came up with a new password, 32% reported having reused an old password when setting up their accounts. We then compared different ML models to detect password reuse from the collected data. Our models achieve an accuracy of up to 87.7% in detecting password reuse from gaze, 75.8% accuracy from typing, and 88.75% when considering both types of behavior. We demonstrate that using gaze, password reuse can already be detected during the registration process, before users entered their password. Our work paves the road for developing novel interventions to prevent password reuse. CCS CONCEPTS • Security and privacy → Usability in security and privacy.

Proceedings of the Second African Conference for Human Computer Interaction: Thriving Communities, 2018

Recent research involves biometrics in authentication as they establish a natural way of communic... more Recent research involves biometrics in authentication as they establish a natural way of communication. Accordingly, in this paper, we describe the implementation of an Android-based authentication application and we focus on human factor impostor attacks on gait based systems. We used the smartphone's built-in accelerometer to record gait cycles while walking. Results proved that imitating other's gait cycles is possible and can be a huge threat. CCS CONCEPTS • Security and privacy → Social aspects of security and privacy; Usability in security and privacy; • Human-centered computing → HCI design and evaluation methods;

IOP Conference Series: Materials Science and Engineering, 2018

The Internet of Things (Iot) field has been expanding in various domains with cultural heritage b... more The Internet of Things (Iot) field has been expanding in various domains with cultural heritage being no exception. Nevertheless, cultural heritage sites are still often managed in traditional ways and a prominent gap remains between tour managers and visitors in correctly identifying visitors interest to specific sites and monuments. Previous related works focused on tackling mood detection separately from particular touristic locations which lead to overly generalized assumptions. Accordingly, in this paper, we propose a comprehensive platform named CHEOPS to enhance the touristic experience by detecting the tourists interest in the sites. The proposed platform provides a customized tour through touristic sites by using indoor localization techniques coupled with information about smartphone activity. Accordingly, the visitor is directed to a more interesting location. In addition to interest detection, CHEOPS implements several techniques for interaction with the touristic site such as gaze detection, gesture recognition, and augmented reality. A prototype of the proposed platform is implemented and tested in real scenarios to detect the usability of the system. The output accurately detects the interest of the users, which can then be used to fill the gap between the site manages and the tourists.

Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications, 2019

In this paper, we propose a calibration-free gaze-based text entry system that uses smooth pursui... more In this paper, we propose a calibration-free gaze-based text entry system that uses smooth pursuit eye movements. We report on our implementation, which improves over prior work on smooth pursuit text entry by 1) eliminating the need of calibration using motion correlation, 2) increasing input rate from 3.34 to 3.41 words per minute, 3) featuring text suggestions that were trained on 10,000 lexicon sentences recommended in the literature. We report on a user study (N=26) which shows that users are able to eye type at 3.41 words per minutes without calibration and without user training. Qualitative feedback also indicates that users positively perceive the system. Our work is of particular benefit for disabled users and for situations when voice and tactile input are not feasible (e.g., in noisy environments or when the hands are occupied). CCS CONCEPTS • Human-centered computing → Human computer interaction (HCI); Text input.

Proceedings of the 17th International Conference on Mobile and Ubiquitous Multimedia, 2018

Most of the current adaptive systems support single task activities. The rise in the number of da... more Most of the current adaptive systems support single task activities. The rise in the number of daily interactive devices and sources of information made multitasking an integral activity in our daily life. Affect-aware systems show exciting potential to support the user, however, they focus on the induced effect of an additional task in terms of cognitive load and stress, rather than the influence of the number of tasks i.e. multitasking. This paper presents indicators of the number of tasks being performed by the user using a set of bio-sensors. A preliminary user study was conducted with two follow-up explorations. Our findings imply that we can distinguish between the number of tasks performed based on high-end as well as cheap Heart Rate sensors. Additionally, tasks number correlates with other signals, namely wrist and forehead temperature. We provide empirical evidence showing how to differentiate between single-and dual-tasking activities.

ACM Symposium on Eye Tracking Research and Applications, 2021

We investigate how problems in understanding text-specifically a word or a sentence-while filling... more We investigate how problems in understanding text-specifically a word or a sentence-while filling in questionnaires are reflected in gaze behaviour. To identify text comprehension problems, while filling a questionnaire, and their correlation with the gaze features, we collected data from 42 participant. In a follow-up study (N=30), we evoked comprehension problems and features they affect and quantified users' gaze behaviour. Our findings implies that comprehension problems could be reflected in a set of gaze features, namely, in the number of fixations, duration of fixations, and number of regressions. Our findings not only demonstrate the potential of eye tracking for assessing reading comprehension but also pave the way for researchers and designers to build novel questionnaire tools that instantly mitigate problems in reading comprehension. CCS CONCEPTS • Human-centered computing → HCI design and evaluation methods.

Intelligent Human Systems Integration, 2017

Brain signals are a reliable information source because human beings have limited voluntary contr... more Brain signals are a reliable information source because human beings have limited voluntary control over. We examine EEG readings as a reporting tool concerning human emotions. We examine whether readings from an eye tracker, can enhance the results. We conducted an experiment on 25 users to measure their EEG signals in response to emotional stimuli. All sensors used were off-the-shelf, to test our method using cheap sensors. We used pleasant and unpleasant videos content to elicit emotional responses. Along with Self-Assessment Mannequin (SAM), Alpha symmetry index readings, and pupil diameter, were recorded. Our results show a significant difference in the video clips eliciting different emotions. This implies that EEG can be a valid way to detect emotional state, especially when combined with eye-tracker. We conclude from our findings that EEG can be used as a platform, upon which reliable affect-aware systems and applications can be built.

ACM Symposium on Eye Tracking Research and Applications, 2021

We investigate the use of gaze behaviour as a means to assess password strength as perceived by u... more We investigate the use of gaze behaviour as a means to assess password strength as perceived by users. We contribute to the effort of making users choose passwords that are robust against guessing-attacks. Our particular idea is to consider also the users' understanding of password strength in security mechanisms. We demonstrate how eye tracking can enable this: by analysing people's gaze behaviour during password creation, its strength can be determined. To demonstrate the feasibility of this approach, we present a proof of concept study (N = 15) in which we asked participants to create weak and strong passwords. Our findings reveal that it is possible to estimate password strength from gaze behaviour with an accuracy of 86% using Machine Learning. Thus, we enable research on novel interfaces that consider users' understanding with the ultimate goal of making users choose stronger passwords. CCS CONCEPTS • Human-centered computing → Empirical studies in HCI; Interactive systems and tools.

Human-Computer Interaction – INTERACT 2021, 2021

We investigate the effectiveness of thermal attacks against input of text with different characte... more We investigate the effectiveness of thermal attacks against input of text with different characteristics; we study text entry on a smartphone touchscreen and a laptop keyboard. First, we ran a study (N=25) to collect a dataset of thermal images of short words, websites, complex strings (special characters, numbers, letters), passphrases and words with duplicate characters. Afterwards, 20 different participants visually inspected the thermal images to attempt to identify the text input. We found that long and complex strings are less vulnerable to thermal attacks, that visual inspection of thermal images reveals different parts of the entered text (36% on average and up to 82%) even if the attack is not fully successful, and that entering text on laptops is more vulnerable to thermal attacks than on smartphones. We conclude with three learned lessons and recommendations to resist thermal attacks.

Proceedings of the ACM on human-computer interaction, May 17, 2023

Although eye tracking brings many benefits to users of mobile devices and developers of mobile ap... more Although eye tracking brings many benefits to users of mobile devices and developers of mobile applications, it poses significant privacy risks to both: the users of mobile devices, and the bystanders that surround users, are within the front-facing camera's field of view. Recent research demonstrates that tracking an individual's gaze reveals personal and sensitive information. This paper presents an investigation of the privacy perceptions and the subjective acceptance of users towards eye tracking on handheld mobile devices. In a four-phase user study (N=17), participants used a smartphone eye tracking app, were interviewed before and after viewing a video showing the amount of sensitive and personal data that could be derived from eye movements, and had their privacy concerns measured. Our findings 1) show factors that influence users' and bystanders' attitudes toward eye tracking on mobile devices such as the algorithms' transparency and the developers' credibility and 2) support designing mechanisms to allow for privacy-aware eye tracking solutions on mobile-devices. CCS Concepts: • Security and privacy → Human and societal aspects of security and privacy; • Human-centered computing → Empirical studies in HCI.

arXiv (Cornell University), May 26, 2023

Users are the last line of defense as phishing emails pass filter mechanisms. At the same time, p... more Users are the last line of defense as phishing emails pass filter mechanisms. At the same time, phishing emails are designed so that they are challenging to identify by users. To this end, attackers employ techniques, such as eliciting stress, targeting helpfulness, or exercising authority, due to which users often miss being manipulated out of malicious intent. This work builds on the assumption that manipulation techniques, even if going unnoticed by users, still lead to changes in their behavior. In this work, we present the outcomes of an online study in which we collected gaze and mouse movement data during an email sorting task. Our findings show that phishing emails lead to significant differences across behavioral features but depend on the nature of the email. We discuss how our findings can be leveraged to build security mechanisms protecting users and companies from phishing.

Lecture Notes in Computer Science

Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Proceedings of the 2nd Empathy-Centric Design Workshop

Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Figure 1: We evaluate the performance of the three widely used gaze-based interaction methods: Dw... more Figure 1: We evaluate the performance of the three widely used gaze-based interaction methods: Dwell time (A), Pursuits (B) and Gaze gestures (C), for target selections on handheld mobile devices while sitting (left) and while walking (right). All participants performed all selections using the three different techniques while sitting and while walking. The red arrow in (B) illustrates the direction in which a yellow dot stimuli was rotating around a selectable target. The red arrows in (C) indicate the directions in which the user could perform a gaze gesture. All arrows are for illustration and were not shown to participants.

Proceedings 2023 Symposium on Usable Security

We propose an approach to identify users' exposure to fake news from users' gaze and mouse moveme... more We propose an approach to identify users' exposure to fake news from users' gaze and mouse movement behavior. Our approach is meant as an enabler for interventions that make users aware of engaging with fake news while not being consciously aware of this. Our work is motivated by the rapid spread of fake news on the web (in particular, social media) and the difficulty and effort required to identify fake content, either technically or by means of a human fact checker. To this end, we set out with conducting a remote online study (N = 54) in which participants were exposed to real and fake social media posts while their mouse and gaze movements were recorded. We identify the most predictive gaze and mouse movement features and show that fake news can be predicted with 68.4% accuracy from users' gaze and mouse movement behavior. Our work is complemented by discussing the implications of using behavioral features for mitigating the spread of fake news on social media.

Proceedings of the 2022 International Conference on Advanced Visual Interfaces

In this work, we explore attacker behavior during shoulder surfing. As such behavior is often opp... more In this work, we explore attacker behavior during shoulder surfing. As such behavior is often opportunistic and difficult to observe in real world settings, we leverage the capabilities of virtual reality (VR). We recruited 24 participants and observed their behavior in two virtual waiting scenarios: at a bus stop and in an open office space. In both scenarios, participants shoulder surfed private screens displaying different types of content. From the results we derive an understanding of factors influencing shoulder surfing behavior, reveal common attack patterns, and sketch a behavioral shoulder surfing model. Our work suggests directions for future research on shoulder surfing and can serve as a basis for creating novel approaches to mitigate shoulder surfing. CCS CONCEPTS • Security and privacy → Human and societal aspects of security and privacy;

CHI Conference on Human Factors in Computing Systems, Apr 29, 2022

A significant drawback of text passwords for end-user authentication is password reuse. We propos... more A significant drawback of text passwords for end-user authentication is password reuse. We propose a novel approach to detect password reuse by leveraging gaze as well as typing behavior and study its accuracy. We collected gaze and typing behavior from 49 users while creating accounts for 1) a webmail client and 2) a news website. While most participants came up with a new password, 32% reported having reused an old password when setting up their accounts. We then compared different ML models to detect password reuse from the collected data. Our models achieve an accuracy of up to 87.7% in detecting password reuse from gaze, 75.8% accuracy from typing, and 88.75% when considering both types of behavior. We demonstrate that using gaze, password reuse can already be detected during the registration process, before users entered their password. Our work paves the road for developing novel interventions to prevent password reuse. CCS CONCEPTS • Security and privacy → Usability in security and privacy.

Proceedings of the Second African Conference for Human Computer Interaction: Thriving Communities, 2018

Recent research involves biometrics in authentication as they establish a natural way of communic... more Recent research involves biometrics in authentication as they establish a natural way of communication. Accordingly, in this paper, we describe the implementation of an Android-based authentication application and we focus on human factor impostor attacks on gait based systems. We used the smartphone's built-in accelerometer to record gait cycles while walking. Results proved that imitating other's gait cycles is possible and can be a huge threat. CCS CONCEPTS • Security and privacy → Social aspects of security and privacy; Usability in security and privacy; • Human-centered computing → HCI design and evaluation methods;

IOP Conference Series: Materials Science and Engineering, 2018

The Internet of Things (Iot) field has been expanding in various domains with cultural heritage b... more The Internet of Things (Iot) field has been expanding in various domains with cultural heritage being no exception. Nevertheless, cultural heritage sites are still often managed in traditional ways and a prominent gap remains between tour managers and visitors in correctly identifying visitors interest to specific sites and monuments. Previous related works focused on tackling mood detection separately from particular touristic locations which lead to overly generalized assumptions. Accordingly, in this paper, we propose a comprehensive platform named CHEOPS to enhance the touristic experience by detecting the tourists interest in the sites. The proposed platform provides a customized tour through touristic sites by using indoor localization techniques coupled with information about smartphone activity. Accordingly, the visitor is directed to a more interesting location. In addition to interest detection, CHEOPS implements several techniques for interaction with the touristic site such as gaze detection, gesture recognition, and augmented reality. A prototype of the proposed platform is implemented and tested in real scenarios to detect the usability of the system. The output accurately detects the interest of the users, which can then be used to fill the gap between the site manages and the tourists.

Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications, 2019

In this paper, we propose a calibration-free gaze-based text entry system that uses smooth pursui... more In this paper, we propose a calibration-free gaze-based text entry system that uses smooth pursuit eye movements. We report on our implementation, which improves over prior work on smooth pursuit text entry by 1) eliminating the need of calibration using motion correlation, 2) increasing input rate from 3.34 to 3.41 words per minute, 3) featuring text suggestions that were trained on 10,000 lexicon sentences recommended in the literature. We report on a user study (N=26) which shows that users are able to eye type at 3.41 words per minutes without calibration and without user training. Qualitative feedback also indicates that users positively perceive the system. Our work is of particular benefit for disabled users and for situations when voice and tactile input are not feasible (e.g., in noisy environments or when the hands are occupied). CCS CONCEPTS • Human-centered computing → Human computer interaction (HCI); Text input.

Proceedings of the 17th International Conference on Mobile and Ubiquitous Multimedia, 2018

Most of the current adaptive systems support single task activities. The rise in the number of da... more Most of the current adaptive systems support single task activities. The rise in the number of daily interactive devices and sources of information made multitasking an integral activity in our daily life. Affect-aware systems show exciting potential to support the user, however, they focus on the induced effect of an additional task in terms of cognitive load and stress, rather than the influence of the number of tasks i.e. multitasking. This paper presents indicators of the number of tasks being performed by the user using a set of bio-sensors. A preliminary user study was conducted with two follow-up explorations. Our findings imply that we can distinguish between the number of tasks performed based on high-end as well as cheap Heart Rate sensors. Additionally, tasks number correlates with other signals, namely wrist and forehead temperature. We provide empirical evidence showing how to differentiate between single-and dual-tasking activities.

ACM Symposium on Eye Tracking Research and Applications, 2021

We investigate how problems in understanding text-specifically a word or a sentence-while filling... more We investigate how problems in understanding text-specifically a word or a sentence-while filling in questionnaires are reflected in gaze behaviour. To identify text comprehension problems, while filling a questionnaire, and their correlation with the gaze features, we collected data from 42 participant. In a follow-up study (N=30), we evoked comprehension problems and features they affect and quantified users' gaze behaviour. Our findings implies that comprehension problems could be reflected in a set of gaze features, namely, in the number of fixations, duration of fixations, and number of regressions. Our findings not only demonstrate the potential of eye tracking for assessing reading comprehension but also pave the way for researchers and designers to build novel questionnaire tools that instantly mitigate problems in reading comprehension. CCS CONCEPTS • Human-centered computing → HCI design and evaluation methods.

Intelligent Human Systems Integration, 2017

Brain signals are a reliable information source because human beings have limited voluntary contr... more Brain signals are a reliable information source because human beings have limited voluntary control over. We examine EEG readings as a reporting tool concerning human emotions. We examine whether readings from an eye tracker, can enhance the results. We conducted an experiment on 25 users to measure their EEG signals in response to emotional stimuli. All sensors used were off-the-shelf, to test our method using cheap sensors. We used pleasant and unpleasant videos content to elicit emotional responses. Along with Self-Assessment Mannequin (SAM), Alpha symmetry index readings, and pupil diameter, were recorded. Our results show a significant difference in the video clips eliciting different emotions. This implies that EEG can be a valid way to detect emotional state, especially when combined with eye-tracker. We conclude from our findings that EEG can be used as a platform, upon which reliable affect-aware systems and applications can be built.

ACM Symposium on Eye Tracking Research and Applications, 2021

We investigate the use of gaze behaviour as a means to assess password strength as perceived by u... more We investigate the use of gaze behaviour as a means to assess password strength as perceived by users. We contribute to the effort of making users choose passwords that are robust against guessing-attacks. Our particular idea is to consider also the users' understanding of password strength in security mechanisms. We demonstrate how eye tracking can enable this: by analysing people's gaze behaviour during password creation, its strength can be determined. To demonstrate the feasibility of this approach, we present a proof of concept study (N = 15) in which we asked participants to create weak and strong passwords. Our findings reveal that it is possible to estimate password strength from gaze behaviour with an accuracy of 86% using Machine Learning. Thus, we enable research on novel interfaces that consider users' understanding with the ultimate goal of making users choose stronger passwords. CCS CONCEPTS • Human-centered computing → Empirical studies in HCI; Interactive systems and tools.

Human-Computer Interaction – INTERACT 2021, 2021

We investigate the effectiveness of thermal attacks against input of text with different characte... more We investigate the effectiveness of thermal attacks against input of text with different characteristics; we study text entry on a smartphone touchscreen and a laptop keyboard. First, we ran a study (N=25) to collect a dataset of thermal images of short words, websites, complex strings (special characters, numbers, letters), passphrases and words with duplicate characters. Afterwards, 20 different participants visually inspected the thermal images to attempt to identify the text input. We found that long and complex strings are less vulnerable to thermal attacks, that visual inspection of thermal images reveals different parts of the entered text (36% on average and up to 82%) even if the attack is not fully successful, and that entering text on laptops is more vulnerable to thermal attacks than on smartphones. We conclude with three learned lessons and recommendations to resist thermal attacks.