Gregor Strle | Research Centre of the Slovenian Academy of Sciences and Arts (original) (raw)

Papers by Gregor Strle

Research paper thumbnail of Emotion Elicitation in a Socially Intelligent Service: The Typing Tutor

Computers

This paper presents an experimental study on modeling machine emotion elicitation in a socially i... more This paper presents an experimental study on modeling machine emotion elicitation in a socially intelligent service, the typing tutor. The aim of the study is to evaluate the extent to which the machine emotion elicitation can influence the affective state (valence and arousal) of the learner during a tutoring session. The tutor provides continuous real-time emotion elicitation via graphically rendered emoticons, as an emotional feedback to learner's performance. Good performance is rewarded by the positive emoticon, based on the notion of positive reinforcement. Facial emotion recognition software is used to analyze the affective state of the learner for later evaluation. Experimental results show the correlation between the positive emoticon and the learner's affective state is significant for all 13 (100%) test participants on the arousal dimension and for 9 (69%) test participants on both affective dimensions. The results also confirm our hypothesis and show that the machine emotion elicitation is significant for 11 (85%) of 13 test participants. We conclude that the machine emotion elicitation with simple graphical emoticons has a promising potential for the future development of the tutor.

Research paper thumbnail of Towards Automatic Real-Time Estimation of Observed Learner’s Attention Using Psychophysiological and Affective Signals: The Touch-Typing Study Case

IEEE Access

This article presents an experimental study on the real-time estimation of observed learners' att... more This article presents an experimental study on the real-time estimation of observed learners' attention given the task of touch-typing. The aim is to examine whether the observed attention estimates gathered from human raters can be computationally modeled in real time, based on the learner's psychophysiological and affective signals. A key observation from this study is that the observed attention varies continuously and throughout the task. The findings show that a relatively high sampling interval is required for the modeling of observed learners' attention, which is impossible to achieve with traditional assessment methods (e.g., between-session self-reports). The results show that multiple linear regression models were relatively successful at discriminating low and high levels of the observed attention. In the best case, the within-learner model performed with the goodness-of-fit adjusted R 2 adj = 0.888 and RMSE = 0.103 (range of the attention scores 1-5). However, the multiple linear model underperformed in the estimation of the observed attention between learners, indicating that the differences among the learners are often significant and cannot be overcome by a general linear model of attention. The between-learner model achieved an adjusted R 2 adj = 0.227 and RMSE = 0.708), explaining only 22.7% of the variability. The influence of individual psychophysiological and affective signals (eye gaze, pupil dilation, and valence and arousal) on the estimation of the observed attention was also examined. The results show that both affective dimensions (valence and arousal), as well as the EyePos2D offset (the distance of an eye from the average position in the x-y plane parallel to the screen), and the EyePos-Z (the distance of an eye from the screen) significantly and most frequently influence the performance of the withinlearner model.

Research paper thumbnail of The Moodo dataset: Integrating user context with emotional and color perception of music for affective music information retrieval

Journal of New Music Research

Research paper thumbnail of Towards a Personalised and Context-Dependent User Experience in Multimedia and Information Systems

International Series on Computer Entertainment and Media Technology, 2016

Research paper thumbnail of Introducing a dataset of emotional and color responses to music

Research paper thumbnail of Computational folkloristics: A semantic analysis and visualization of topic distribution of song types | Računalniška folkloristika. Semantična analiza in vizualizacija tematske porazdelitve pesemskih tipov

Research paper thumbnail of New approaches: Uncovering semantic structures in ethnological materials | Novi pristopi. Odkrivanje semantičnih struktur v etnoloških vsebinah

Research paper thumbnail of New dataset of emotional and color responses to music

Research paper thumbnail of Introducing a dataset of emotional and color responses to music

Research paper thumbnail of The MoodStripe-An evaluation of a novel visual interface as an alternative for online response gatherin

We present an innovative dynamic visual interface, the Mood-Stripe, which provides a continuous-s... more We present an innovative dynamic visual interface, the Mood-Stripe, which provides a continuous-scale, multi-parameter drag-and-drop alternative to the standard n-degree (Likert) scale widgets, commonly used in online evaluation processes. We elaborate on the motivation for the development of the new user input interfaces, and present the results of cross evaluation of the GMail product by using the SUS questionnaire with the standard and the proposed MoodStripe interfaces. The overall goal is to design a more intuitive interface, by reducing the noise and task load inherent in traditional interfaces for standardized user-feedback gathering tests. The results show the MoodStripe interface outperforms the standard scale approach both in terms of intuitiveness and functionality. Additionally, the cross-evaluation of the both approaches shows comparable SUS scores.

Research paper thumbnail of Gathering a dataset of multi-modal mood-dependent perceptual responses to music

Research paper thumbnail of Capturing the mood: Evaluation of the moodstripe and moodgraph interfaces

2014 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), 2014

Research paper thumbnail of Ethnomuse: Multimedia Digital Archive of Slovenian Folk Song, Music, and Dance Collections

Research paper thumbnail of Ethnomuse: Archiving folk music and dance culture

The paper presents the development of EthnoMuse: multimedia digital library of Slovenian folk mus... more The paper presents the development of EthnoMuse: multimedia digital library of Slovenian folk music and dance culture. The main scope of the project concerns the digitization of production and post-production processes that relate to collecting, documenting and archiving of folk heritage and development of multimedia applications for various content types (folk song, music and dance) and formats (image, audio, video,

Research paper thumbnail of The EthnoMuse digital library: conceptual representation and annotation of ethnomusicological materials

International Journal on Digital Libraries, 2012

... M. Marolt (B) Faculty of Computer and Information Science, University of Ljubljana, Trzaska 2... more ... M. Marolt (B) Faculty of Computer and Information Science, University of Ljubljana, Trzaska 25, 1000 Ljubljana, Slovenia e-mail: matija.marolt@fri ... towards covering wide end-user populations and large quantities of different types of content (a good example with strong focus on ...

Research paper thumbnail of Conceptualizing the Ethnomuse: Application of CIDOC CRM and FRBR

The paper refers to the national funded project Ethnomuse: digital storage of Slovene folk music ... more The paper refers to the national funded project Ethnomuse: digital storage of Slovene folk music and dance culture . The main scope of the project concerns the development of advanced multimedia applications for various content (folk song, music and dance) and format (image, audio, video, notation, MIDI etc.), and digitisation of the production and postproduction processes that relate to collecting, documenting and archiving of Slovene folk songs, music and dance. The objective of this paper is to discuss the latter, with focus on conceptual design of the flexible data model.

Research paper thumbnail of Emotion Elicitation in a Socially Intelligent Service: The Typing Tutor

Computers

This paper presents an experimental study on modeling machine emotion elicitation in a socially i... more This paper presents an experimental study on modeling machine emotion elicitation in a socially intelligent service, the typing tutor. The aim of the study is to evaluate the extent to which the machine emotion elicitation can influence the affective state (valence and arousal) of the learner during a tutoring session. The tutor provides continuous real-time emotion elicitation via graphically rendered emoticons, as an emotional feedback to learner's performance. Good performance is rewarded by the positive emoticon, based on the notion of positive reinforcement. Facial emotion recognition software is used to analyze the affective state of the learner for later evaluation. Experimental results show the correlation between the positive emoticon and the learner's affective state is significant for all 13 (100%) test participants on the arousal dimension and for 9 (69%) test participants on both affective dimensions. The results also confirm our hypothesis and show that the machine emotion elicitation is significant for 11 (85%) of 13 test participants. We conclude that the machine emotion elicitation with simple graphical emoticons has a promising potential for the future development of the tutor.

Research paper thumbnail of Towards Automatic Real-Time Estimation of Observed Learner’s Attention Using Psychophysiological and Affective Signals: The Touch-Typing Study Case

IEEE Access

This article presents an experimental study on the real-time estimation of observed learners' att... more This article presents an experimental study on the real-time estimation of observed learners' attention given the task of touch-typing. The aim is to examine whether the observed attention estimates gathered from human raters can be computationally modeled in real time, based on the learner's psychophysiological and affective signals. A key observation from this study is that the observed attention varies continuously and throughout the task. The findings show that a relatively high sampling interval is required for the modeling of observed learners' attention, which is impossible to achieve with traditional assessment methods (e.g., between-session self-reports). The results show that multiple linear regression models were relatively successful at discriminating low and high levels of the observed attention. In the best case, the within-learner model performed with the goodness-of-fit adjusted R 2 adj = 0.888 and RMSE = 0.103 (range of the attention scores 1-5). However, the multiple linear model underperformed in the estimation of the observed attention between learners, indicating that the differences among the learners are often significant and cannot be overcome by a general linear model of attention. The between-learner model achieved an adjusted R 2 adj = 0.227 and RMSE = 0.708), explaining only 22.7% of the variability. The influence of individual psychophysiological and affective signals (eye gaze, pupil dilation, and valence and arousal) on the estimation of the observed attention was also examined. The results show that both affective dimensions (valence and arousal), as well as the EyePos2D offset (the distance of an eye from the average position in the x-y plane parallel to the screen), and the EyePos-Z (the distance of an eye from the screen) significantly and most frequently influence the performance of the withinlearner model.

Research paper thumbnail of The Moodo dataset: Integrating user context with emotional and color perception of music for affective music information retrieval

Journal of New Music Research

Research paper thumbnail of Towards a Personalised and Context-Dependent User Experience in Multimedia and Information Systems

International Series on Computer Entertainment and Media Technology, 2016

Research paper thumbnail of Introducing a dataset of emotional and color responses to music

Research paper thumbnail of Computational folkloristics: A semantic analysis and visualization of topic distribution of song types | Računalniška folkloristika. Semantična analiza in vizualizacija tematske porazdelitve pesemskih tipov

Research paper thumbnail of New approaches: Uncovering semantic structures in ethnological materials | Novi pristopi. Odkrivanje semantičnih struktur v etnoloških vsebinah

Research paper thumbnail of New dataset of emotional and color responses to music

Research paper thumbnail of Introducing a dataset of emotional and color responses to music

Research paper thumbnail of The MoodStripe-An evaluation of a novel visual interface as an alternative for online response gatherin

We present an innovative dynamic visual interface, the Mood-Stripe, which provides a continuous-s... more We present an innovative dynamic visual interface, the Mood-Stripe, which provides a continuous-scale, multi-parameter drag-and-drop alternative to the standard n-degree (Likert) scale widgets, commonly used in online evaluation processes. We elaborate on the motivation for the development of the new user input interfaces, and present the results of cross evaluation of the GMail product by using the SUS questionnaire with the standard and the proposed MoodStripe interfaces. The overall goal is to design a more intuitive interface, by reducing the noise and task load inherent in traditional interfaces for standardized user-feedback gathering tests. The results show the MoodStripe interface outperforms the standard scale approach both in terms of intuitiveness and functionality. Additionally, the cross-evaluation of the both approaches shows comparable SUS scores.

Research paper thumbnail of Gathering a dataset of multi-modal mood-dependent perceptual responses to music

Research paper thumbnail of Capturing the mood: Evaluation of the moodstripe and moodgraph interfaces

2014 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), 2014

Research paper thumbnail of Ethnomuse: Multimedia Digital Archive of Slovenian Folk Song, Music, and Dance Collections

Research paper thumbnail of Ethnomuse: Archiving folk music and dance culture

The paper presents the development of EthnoMuse: multimedia digital library of Slovenian folk mus... more The paper presents the development of EthnoMuse: multimedia digital library of Slovenian folk music and dance culture. The main scope of the project concerns the digitization of production and post-production processes that relate to collecting, documenting and archiving of folk heritage and development of multimedia applications for various content types (folk song, music and dance) and formats (image, audio, video,

Research paper thumbnail of The EthnoMuse digital library: conceptual representation and annotation of ethnomusicological materials

International Journal on Digital Libraries, 2012

... M. Marolt (B) Faculty of Computer and Information Science, University of Ljubljana, Trzaska 2... more ... M. Marolt (B) Faculty of Computer and Information Science, University of Ljubljana, Trzaska 25, 1000 Ljubljana, Slovenia e-mail: matija.marolt@fri ... towards covering wide end-user populations and large quantities of different types of content (a good example with strong focus on ...

Research paper thumbnail of Conceptualizing the Ethnomuse: Application of CIDOC CRM and FRBR

The paper refers to the national funded project Ethnomuse: digital storage of Slovene folk music ... more The paper refers to the national funded project Ethnomuse: digital storage of Slovene folk music and dance culture . The main scope of the project concerns the development of advanced multimedia applications for various content (folk song, music and dance) and format (image, audio, video, notation, MIDI etc.), and digitisation of the production and postproduction processes that relate to collecting, documenting and archiving of Slovene folk songs, music and dance. The objective of this paper is to discuss the latter, with focus on conceptual design of the flexible data model.