Andreas Sonderegger | University of Fribourg (original) (raw)
Papers by Andreas Sonderegger
Multimodal Interactive Systems Management, 2014
We present in this paper a new multimodal corpus of spontaneous collaborative and affective inter... more We present in this paper a new multimodal corpus of spontaneous collaborative and affective interactions in French: RECOLA, which is being made available to the research community. Participants were recorded in dyads during a video conference while completing a task requiring collaboration. Different multimodal data, i.e., audio, video, ECG and EDA, were recorded continuously and synchronously. In total, 46 participants took part in the test, for which the first 5 minutes of interaction were kept to ease annotation. In addition to these recordings, 6 annotators measured emotion continuously on two dimensions: arousal and valence, as well as social behavior labels on five dimensions. The corpus allowed us to take self-report measures of users during task completion. Methodologies and issues related to affective corpus construction are briefly reviewed in this paper. We further detail how the corpus was constructed, i.e., participants, procedure and task, the multimodal recording setup, the annotation of data and some analysis of the quality of these annotations.
This study examined the utility of the concept of expressive aesthetics by testing websites that ... more This study examined the utility of the concept of expressive aesthetics by testing websites that did or did not match this concept. A website scoring highly on this concept was created and was then compared to websites that either was non-aesthetic or corresponded to the concept of classical aesthetics. 60 website users of a broad age range (18 -60 yrs) were allocated to three experimental groups (expressive, classical, and non-aesthetic) and asked to complete a series of information search tasks. During the experiment, measures were taken of performance, perceived usability, perceived aesthetics, emotion, and trustworthiness. The results showed that expressive aesthetics can be considered a distinct concept. It also emerged that the website scoring high on expressive aesthetics shows a similar pattern of results to classical aesthetics. Both aesthetically appealing websites received higher ratings of perceived usability and trustworthiness than the non-aesthetic website. The effects of website aesthetics on subjective measures were not moderated by age. Thielsch (2010) presented a model that differentiates between four factors in the perception of visual aesthetics: simplicity (reflecting aspects that facilitate perception and the processing of a layout, such as clarity, orderliness, homogeneity, grouping, and balance), diversity (reflecting visual richness, dynamics, variety, creativity, and novelty), colourfulness (reflecting aspects related to the evaluation of individual colours and their composition), and craftsmanship (reflecting whether the site was designed with skill and care using modern technologies). Another distinction is made between classical and expressive aesthetics (Lavie and Tractinsky, 2004). Since this distinction represents the theoretical framework of the present study, it is explored in more detail.
This paper examines how emotion feedback influences emotion awareness and gaze behavior. Simulati... more This paper examines how emotion feedback influences emotion awareness and gaze behavior. Simulating a videoconference setup, 36 participants watched 12 emotional video sequences that were selected from the SEMAINE database. All participants wore an eye-tracker to measure gaze behavior and were asked to rate the perceived emotion for each video sequence. 3 conditions were tested: (c1) no feedback, i.e., the original video-sequences, (c2) correct feedback, i.e., an emoticon is integrated in the video to show the emotion depicted by the person in the video and (c3) random feedback, i.e., the emoticon displays at random an emotional state that may or may not correspond to the one of the person. The results showed that emotion feedback had a significant influence on gaze behavior, e.g., over time random feedback led to a decrease in the frequency of episodes of gaze. No effect of emotion display was observed for emotion recognition. However, experiments on the automatic emotion recognition using gaze behavior provided good performance, with better score on arousal than valence, and a very good performance was obtained in the automatic recognition of the correctness of the emotion feedback.
This article examines the influence of mood feedback on different outcomes of teamwork in two dif... more This article examines the influence of mood feedback on different outcomes of teamwork in two different collaborative work environments. Employing a 2 x 2 between-subjects design, mood feedback (present vs. not present) and communication mode (face-to-face vs. video conferencing) were manipulated experimentally. We used a newly developed collaborative communication environment, called EmotiBoard, which is a large vertical interactive screen, with which team members can interact in a face-to-face discussion or as a spatially distributed team. To support teamwork, this tool provides visual feedback of each team member's emotional state. Thirty-five teams comprising 3 persons each (with a confederate in each team) completed three different tasks, measuring mood, performance, subjective workload, and team satisfaction. Results indicated that the evaluation of the other team members' emotional state was more accurate when the mood feedback was presented. In addition, mood feedback influenced team performance positively in the video conference condition and negatively in the face-to-face condition. Furthermore, participants in the video conference condition were more satisfied after task completion than participants in the face-to-face condition. Findings indicate that the mood feedback toolis helpful for teams to gain a more accurate understanding of team members' emotional states in different work situations.
An experimental study examined whether the effects of aesthetic appeal on usability test outcomes... more An experimental study examined whether the effects of aesthetic appeal on usability test outcomes are moderated by usage domain. The aesthetic appeal of a cell phone was experimentally manipulated in both home-and work-based usage domains. The two usage domains were modeled in a usability laboratory. 60 participants completed a series of typical cell phone user tasks. Dependent measures such as performance, perceived usability, and emotion were taken. The results showed that aesthetic appeal had a positive effect on perceived usability but a negative effect on performance. The effects of aesthetic appeal on usability test outcomes were not moderated by usage domain. The results of this study imply that it may be sufficient to test dual-domain products in only one of their usage domains.
Proceedings of the 32nd annual ACM conference on Human factors in computing systems - CHI '14, 2014
ABSTRACT An experimental study examined whether the effects of aesthetic appeal on usability test... more ABSTRACT An experimental study examined whether the effects of aesthetic appeal on usability test outcomes are moderated by usage domain. The aesthetic appeal of a cell phone was experimentally manipulated in both home- and work-based usage domains. The two usage domains were modeled in a usability laboratory. 60 participants completed a series of typical cell phone user tasks. Dependent measures such as performance, perceived usability, and emotion were taken. The results showed that aesthetic appeal had a positive effect on perceived usability but a negative effect on performance. The effects of aesthetic appeal on usability test outcomes were not moderated by usage domain. The results of this study imply that it may be sufficient to test dual-domain products in only one of their usage domains.
A longitudinal field experiment was carried out over a period of two weeks to examine the influen... more A longitudinal field experiment was carried out over a period of two weeks to examine the influence of product aesthetics and inherent product usability. A 2 x 2 x 3 mixed design was used in the study, with product aesthetics (high / low) and usability (high / low) being manipulated as between-subjects variables and exposure time as a repeated-measures variable (3 levels). A sample of 60 mobile phone users was tested during a multiple-session usability test. A range of outcome variables was measured, including performance, perceived usability, perceived aesthetics, and emotion. A major finding was that the positive effect of an aesthetically appealing product on perceived usability, reported in many previous studies, began to wane with increasing exposure time. The data provided similar evidence for emotion, which also showed changes as a function of exposure time. The study has methodological implications for the future design of usability tests, notably suggesting the need for longitudinal approaches in usability research.
This article examines the influence of socio-cultural background and product value on different o... more This article examines the influence of socio-cultural background and product value on different outcomes of usability tests. A study was conducted in two different socio-cultural regions, Switzerland and East Germany, which differed in a number of aspects (e.g. economic power, price sensitivity and culture). Product value (high vs. low) was varied by manipulating the price of the product. Sixty-four test participants were asked to carry out five typical user tasks in the context of coffee machine usage, measuring performance, perceived usability, and emotion. The results showed that in Switzerland, high-value products were rated higher in usability than low-value products whereas in East Germany, high-value products were evaluated lower in usability. A similar interaction effect of socio-cultural background and product value was observed for user emotion. Implications are that the outcomes of usability tests do not allow for a simple transfer across cultures and that the mediating influence of perceived product value needs to be taken into consideration.
This article examines the influences of situational factors on user behaviour in usability tests.... more This article examines the influences of situational factors on user behaviour in usability tests. Sixty participants carried out two tasks on a computer-simulated prototype of a mobile phone. Employing a 3 6 2 mixed experimental design, laboratory set-up was varied as a between-subjects variable (presence of facilitator and two non-interactive observers, presence of facilitator or no person present) while task difficulty was manipulated as a within-subjects variable (low vs. high). Performance data, subjective measures and physiological parameters (e.g. heart rate variability) were taken. The results showed that the presence of non-interactive observers during a usability test led to a physiological stress response, decreased performance on some measures and affected the emotional state of test participants. The presence of a facilitator (i.e. a participating observer) also influenced the emotional state of the test participant. Practitioners involved in usability testing need to be aware of undue influences of observers, in particular, if the observers are non-interactive. The findings presented in this paper have implications for the practice of usability testing. They indicated a considerable influence of observers on test participants (physiology and emotions) and on the outcomes of usability tests (performance measures). This should be considered when selecting the set-up of a usability testing procedure.
This article examines the utility of two commonly used approaches in the evaluation of interactiv... more This article examines the utility of two commonly used approaches in the evaluation of interactive consumer products: lab-based testing and single task scenarios. These are compared to two more complex and resource-demanding approaches (field-based testing and dual task scenarios) with regard to the test results they produce. An experiment with N ¼ 80 users was carried out, employing a 2 (laboratory vs. field) by 2 (single task vs. dual task scenario) by 2 (on-product information: present vs. absent) between-subjects design. On-product information (advising users to save water and electricity during kettle usage) represented the intervention, of which the effects on user behaviour were compared under the different experimental conditions. The main finding was that the impact of on-product information on user behaviour was strongest in the lab-based testing environment using a single task scenario (i.e., most economical testing condition), compared to the three other experimental conditions. The work found similar effects for self-report measures. The findings of the study point to the risk that the effects of system redesign on user behaviour may be overestimated if low-fidelity testing approaches are employed. The relevance of these findings for other application areas is also discussed (e.g., design of warnings).
This article examined the effects of product aesthetics on several outcome variables in usability... more This article examined the effects of product aesthetics on several outcome variables in usability tests. Employing a computer simulation of a mobile phone, 60 adolescents (14-17 yrs) were asked to complete a number of typical tasks of mobile phone users. Two functionally identical mobile phones were manipulated with regard to their visual appearance (highly appealing vs not appealing) to determine the influence of appearance on perceived usability, performance measures and perceived attractiveness. The results showed that participants using the highly appealing phone rated their appliance as being more usable than participants operating the unappealing model. Furthermore, the visual appearance of the phone had a positive effect on performance, leading to reduced task completion times for the attractive model. The study discusses the implications for the use of adolescents in ergonomic research.
European Journal of Neuroscience, 2014
Microsaccades are involuntary, small-magnitude saccadic eye movements that occur during attempted... more Microsaccades are involuntary, small-magnitude saccadic eye movements that occur during attempted visual fixation. Recent research has found that attention can modulate microsaccade dynamics, but few studies have addressed the effects of task difficulty on microsaccade parameters, and those have obtained contradictory results. Further, no study to date has investigated the influence of task difficulty on microsaccade production during the performance of non-visual tasks. Thus, the effects of task difficulty on microsaccades, isolated from sensory modality, remain unclear. Here we investigated the effects of task difficulty on microsaccades during the performance of a non-visual, mental arithmetic task with two levels of complexity. We found that microsaccade rates decreased and microsaccade magnitudes increased with increased task difficulty. We propose that changes in microsaccade rates and magnitudes with task difficulty are mediated by the effects of varying attentional inputs on the rostral superior colliculus activity map.
An empirical study examined the impact of prototype fidelity on user behaviour, subjective user e... more An empirical study examined the impact of prototype fidelity on user behaviour, subjective user evaluation and emotion. The independent factors of prototype fidelity (paper prototype, computer prototype, fully operational appliance) and aesthetics of design (high vs. moderate) were varied in a betweensubjects design. The 60 participants of the experiment were asked to complete two typical tasks of mobile phone usage: sending a text message and suppressing a phone number. Both performance data and a number of subjective measures were recorded. The results suggested that task completion time may be overestimated when a computer prototype is being used. Furthermore, users appeared to compensate for deficiencies in aesthetic design by overrating the aesthetic qualities of reduced fidelity prototypes. Finally, user emotions were more positively affected by the operation of the more attractive mobile phone than by the less appealing one.
Multimodal Interactive Systems Management, 2014
We present in this paper a new multimodal corpus of spontaneous collaborative and affective inter... more We present in this paper a new multimodal corpus of spontaneous collaborative and affective interactions in French: RECOLA, which is being made available to the research community. Participants were recorded in dyads during a video conference while completing a task requiring collaboration. Different multimodal data, i.e., audio, video, ECG and EDA, were recorded continuously and synchronously. In total, 46 participants took part in the test, for which the first 5 minutes of interaction were kept to ease annotation. In addition to these recordings, 6 annotators measured emotion continuously on two dimensions: arousal and valence, as well as social behavior labels on five dimensions. The corpus allowed us to take self-report measures of users during task completion. Methodologies and issues related to affective corpus construction are briefly reviewed in this paper. We further detail how the corpus was constructed, i.e., participants, procedure and task, the multimodal recording setup, the annotation of data and some analysis of the quality of these annotations.
This study examined the utility of the concept of expressive aesthetics by testing websites that ... more This study examined the utility of the concept of expressive aesthetics by testing websites that did or did not match this concept. A website scoring highly on this concept was created and was then compared to websites that either was non-aesthetic or corresponded to the concept of classical aesthetics. 60 website users of a broad age range (18 -60 yrs) were allocated to three experimental groups (expressive, classical, and non-aesthetic) and asked to complete a series of information search tasks. During the experiment, measures were taken of performance, perceived usability, perceived aesthetics, emotion, and trustworthiness. The results showed that expressive aesthetics can be considered a distinct concept. It also emerged that the website scoring high on expressive aesthetics shows a similar pattern of results to classical aesthetics. Both aesthetically appealing websites received higher ratings of perceived usability and trustworthiness than the non-aesthetic website. The effects of website aesthetics on subjective measures were not moderated by age. Thielsch (2010) presented a model that differentiates between four factors in the perception of visual aesthetics: simplicity (reflecting aspects that facilitate perception and the processing of a layout, such as clarity, orderliness, homogeneity, grouping, and balance), diversity (reflecting visual richness, dynamics, variety, creativity, and novelty), colourfulness (reflecting aspects related to the evaluation of individual colours and their composition), and craftsmanship (reflecting whether the site was designed with skill and care using modern technologies). Another distinction is made between classical and expressive aesthetics (Lavie and Tractinsky, 2004). Since this distinction represents the theoretical framework of the present study, it is explored in more detail.
This paper examines how emotion feedback influences emotion awareness and gaze behavior. Simulati... more This paper examines how emotion feedback influences emotion awareness and gaze behavior. Simulating a videoconference setup, 36 participants watched 12 emotional video sequences that were selected from the SEMAINE database. All participants wore an eye-tracker to measure gaze behavior and were asked to rate the perceived emotion for each video sequence. 3 conditions were tested: (c1) no feedback, i.e., the original video-sequences, (c2) correct feedback, i.e., an emoticon is integrated in the video to show the emotion depicted by the person in the video and (c3) random feedback, i.e., the emoticon displays at random an emotional state that may or may not correspond to the one of the person. The results showed that emotion feedback had a significant influence on gaze behavior, e.g., over time random feedback led to a decrease in the frequency of episodes of gaze. No effect of emotion display was observed for emotion recognition. However, experiments on the automatic emotion recognition using gaze behavior provided good performance, with better score on arousal than valence, and a very good performance was obtained in the automatic recognition of the correctness of the emotion feedback.
This article examines the influence of mood feedback on different outcomes of teamwork in two dif... more This article examines the influence of mood feedback on different outcomes of teamwork in two different collaborative work environments. Employing a 2 x 2 between-subjects design, mood feedback (present vs. not present) and communication mode (face-to-face vs. video conferencing) were manipulated experimentally. We used a newly developed collaborative communication environment, called EmotiBoard, which is a large vertical interactive screen, with which team members can interact in a face-to-face discussion or as a spatially distributed team. To support teamwork, this tool provides visual feedback of each team member's emotional state. Thirty-five teams comprising 3 persons each (with a confederate in each team) completed three different tasks, measuring mood, performance, subjective workload, and team satisfaction. Results indicated that the evaluation of the other team members' emotional state was more accurate when the mood feedback was presented. In addition, mood feedback influenced team performance positively in the video conference condition and negatively in the face-to-face condition. Furthermore, participants in the video conference condition were more satisfied after task completion than participants in the face-to-face condition. Findings indicate that the mood feedback toolis helpful for teams to gain a more accurate understanding of team members' emotional states in different work situations.
An experimental study examined whether the effects of aesthetic appeal on usability test outcomes... more An experimental study examined whether the effects of aesthetic appeal on usability test outcomes are moderated by usage domain. The aesthetic appeal of a cell phone was experimentally manipulated in both home-and work-based usage domains. The two usage domains were modeled in a usability laboratory. 60 participants completed a series of typical cell phone user tasks. Dependent measures such as performance, perceived usability, and emotion were taken. The results showed that aesthetic appeal had a positive effect on perceived usability but a negative effect on performance. The effects of aesthetic appeal on usability test outcomes were not moderated by usage domain. The results of this study imply that it may be sufficient to test dual-domain products in only one of their usage domains.
Proceedings of the 32nd annual ACM conference on Human factors in computing systems - CHI '14, 2014
ABSTRACT An experimental study examined whether the effects of aesthetic appeal on usability test... more ABSTRACT An experimental study examined whether the effects of aesthetic appeal on usability test outcomes are moderated by usage domain. The aesthetic appeal of a cell phone was experimentally manipulated in both home- and work-based usage domains. The two usage domains were modeled in a usability laboratory. 60 participants completed a series of typical cell phone user tasks. Dependent measures such as performance, perceived usability, and emotion were taken. The results showed that aesthetic appeal had a positive effect on perceived usability but a negative effect on performance. The effects of aesthetic appeal on usability test outcomes were not moderated by usage domain. The results of this study imply that it may be sufficient to test dual-domain products in only one of their usage domains.
A longitudinal field experiment was carried out over a period of two weeks to examine the influen... more A longitudinal field experiment was carried out over a period of two weeks to examine the influence of product aesthetics and inherent product usability. A 2 x 2 x 3 mixed design was used in the study, with product aesthetics (high / low) and usability (high / low) being manipulated as between-subjects variables and exposure time as a repeated-measures variable (3 levels). A sample of 60 mobile phone users was tested during a multiple-session usability test. A range of outcome variables was measured, including performance, perceived usability, perceived aesthetics, and emotion. A major finding was that the positive effect of an aesthetically appealing product on perceived usability, reported in many previous studies, began to wane with increasing exposure time. The data provided similar evidence for emotion, which also showed changes as a function of exposure time. The study has methodological implications for the future design of usability tests, notably suggesting the need for longitudinal approaches in usability research.
This article examines the influence of socio-cultural background and product value on different o... more This article examines the influence of socio-cultural background and product value on different outcomes of usability tests. A study was conducted in two different socio-cultural regions, Switzerland and East Germany, which differed in a number of aspects (e.g. economic power, price sensitivity and culture). Product value (high vs. low) was varied by manipulating the price of the product. Sixty-four test participants were asked to carry out five typical user tasks in the context of coffee machine usage, measuring performance, perceived usability, and emotion. The results showed that in Switzerland, high-value products were rated higher in usability than low-value products whereas in East Germany, high-value products were evaluated lower in usability. A similar interaction effect of socio-cultural background and product value was observed for user emotion. Implications are that the outcomes of usability tests do not allow for a simple transfer across cultures and that the mediating influence of perceived product value needs to be taken into consideration.
This article examines the influences of situational factors on user behaviour in usability tests.... more This article examines the influences of situational factors on user behaviour in usability tests. Sixty participants carried out two tasks on a computer-simulated prototype of a mobile phone. Employing a 3 6 2 mixed experimental design, laboratory set-up was varied as a between-subjects variable (presence of facilitator and two non-interactive observers, presence of facilitator or no person present) while task difficulty was manipulated as a within-subjects variable (low vs. high). Performance data, subjective measures and physiological parameters (e.g. heart rate variability) were taken. The results showed that the presence of non-interactive observers during a usability test led to a physiological stress response, decreased performance on some measures and affected the emotional state of test participants. The presence of a facilitator (i.e. a participating observer) also influenced the emotional state of the test participant. Practitioners involved in usability testing need to be aware of undue influences of observers, in particular, if the observers are non-interactive. The findings presented in this paper have implications for the practice of usability testing. They indicated a considerable influence of observers on test participants (physiology and emotions) and on the outcomes of usability tests (performance measures). This should be considered when selecting the set-up of a usability testing procedure.
This article examines the utility of two commonly used approaches in the evaluation of interactiv... more This article examines the utility of two commonly used approaches in the evaluation of interactive consumer products: lab-based testing and single task scenarios. These are compared to two more complex and resource-demanding approaches (field-based testing and dual task scenarios) with regard to the test results they produce. An experiment with N ¼ 80 users was carried out, employing a 2 (laboratory vs. field) by 2 (single task vs. dual task scenario) by 2 (on-product information: present vs. absent) between-subjects design. On-product information (advising users to save water and electricity during kettle usage) represented the intervention, of which the effects on user behaviour were compared under the different experimental conditions. The main finding was that the impact of on-product information on user behaviour was strongest in the lab-based testing environment using a single task scenario (i.e., most economical testing condition), compared to the three other experimental conditions. The work found similar effects for self-report measures. The findings of the study point to the risk that the effects of system redesign on user behaviour may be overestimated if low-fidelity testing approaches are employed. The relevance of these findings for other application areas is also discussed (e.g., design of warnings).
This article examined the effects of product aesthetics on several outcome variables in usability... more This article examined the effects of product aesthetics on several outcome variables in usability tests. Employing a computer simulation of a mobile phone, 60 adolescents (14-17 yrs) were asked to complete a number of typical tasks of mobile phone users. Two functionally identical mobile phones were manipulated with regard to their visual appearance (highly appealing vs not appealing) to determine the influence of appearance on perceived usability, performance measures and perceived attractiveness. The results showed that participants using the highly appealing phone rated their appliance as being more usable than participants operating the unappealing model. Furthermore, the visual appearance of the phone had a positive effect on performance, leading to reduced task completion times for the attractive model. The study discusses the implications for the use of adolescents in ergonomic research.
European Journal of Neuroscience, 2014
Microsaccades are involuntary, small-magnitude saccadic eye movements that occur during attempted... more Microsaccades are involuntary, small-magnitude saccadic eye movements that occur during attempted visual fixation. Recent research has found that attention can modulate microsaccade dynamics, but few studies have addressed the effects of task difficulty on microsaccade parameters, and those have obtained contradictory results. Further, no study to date has investigated the influence of task difficulty on microsaccade production during the performance of non-visual tasks. Thus, the effects of task difficulty on microsaccades, isolated from sensory modality, remain unclear. Here we investigated the effects of task difficulty on microsaccades during the performance of a non-visual, mental arithmetic task with two levels of complexity. We found that microsaccade rates decreased and microsaccade magnitudes increased with increased task difficulty. We propose that changes in microsaccade rates and magnitudes with task difficulty are mediated by the effects of varying attentional inputs on the rostral superior colliculus activity map.
An empirical study examined the impact of prototype fidelity on user behaviour, subjective user e... more An empirical study examined the impact of prototype fidelity on user behaviour, subjective user evaluation and emotion. The independent factors of prototype fidelity (paper prototype, computer prototype, fully operational appliance) and aesthetics of design (high vs. moderate) were varied in a betweensubjects design. The 60 participants of the experiment were asked to complete two typical tasks of mobile phone usage: sending a text message and suppressing a phone number. Both performance data and a number of subjective measures were recorded. The results suggested that task completion time may be overestimated when a computer prototype is being used. Furthermore, users appeared to compensate for deficiencies in aesthetic design by overrating the aesthetic qualities of reduced fidelity prototypes. Finally, user emotions were more positively affected by the operation of the more attractive mobile phone than by the less appealing one.