francesco cutugno | Università degli Studi di Napoli "Federico II" (original) (raw)
Papers by francesco cutugno
Lecture Notes in Information Systems and Organisation, 2014
Pupil dilation is known to reflect the emotional arousal. Pleasure, effort and fear are examples ... more Pupil dilation is known to reflect the emotional arousal. Pleasure, effort and fear are examples of stimuli inducing the nervous system to cause dilation mydriasis. The work proposes a tool to automatically quantify the mydriasis in order to evaluate mental effort in HCI. The system uses a feature-based approach and monitors the pupil behavior during a given task. As mydriasis is entailed by various reasons, our system distinguishes the cause-effect relationships by synchronizing monitoring and test, dividing the monitoring in fixed intervals and retrieving a survey of the mydriatic events for each determined period of time. We present a case of study analyzing users resolving arithmetical tasks, viewing pictures and using a mobile application. In each scenario, tests intend to impose gradually increasing reactions to the users. The paper will present different techniques for pupil dilation measurements and related results of mental effort evaluation.
2014 Ninth International Conference on P2P, Parallel, Grid, Cloud and Internet Computing, 2014
Spatial dimensionality is one of the main features of the physical environment in which humans li... more Spatial dimensionality is one of the main features of the physical environment in which humans live. When we think about 3D, we usually refer to 3D video, even if it is not the only existing channel of natural interaction. In this paper, we present an interaction system based on spatialized sounds. We developed an innovative cloud application in the cultural heritage context, a personal guide, in 3D sound, attracting the tourists' attention toward monuments or buildings, offering sounds capes of augmented reality. The designed system interacts with smart headphones that remotely takes the orientation of the listener's head and properly generates an audio output, which also takes into account the listener's position and orientation in the environment. Thus, an innovative headphones using an inertial measurement unit for determining the orientation of a user's head have been designed and developed in open-ear mode, in order to locate the user in the real context.
The ability of representing spatio-temporal features plays a crucial role in Cultural Heritage fi... more The ability of representing spatio-temporal features plays a crucial role in Cultural Heritage field. Management of thematic contexts, of time domain with qualitative and imprecise references, hierarchical structure of time and spatial and temporal multiple granularities are very important feature in this environment for users aiming at discovering knowledge process. In this paper a framework to manage spatiotemporal cultural heritage data exploration and visualization is presented. This framework allows to develop web applications able to handle spatio-temporal objects, regardless of the specific descriptive nature of data, and relies on a flexible architecture for web applications that shows low coupling among tiers and uses standard files and protocols, like WFS, GML, KML, becoming independent from storage and visualization tools. The Visualization layer is designed to offer spatio-temporal views with a high level of personalization: users can choose different kinds of views and visual metaphors to compare objects on the geobrowser by activating or disabling layers of their interest. A new method aiming at visualizing and exploring the hierarchical and stratified time domain is presented. The user interface is open and extendable to new methods of visualization. Finally, a specific web application, based on the described framework, is shown.
3rd European Conference on Speech Communication and Technology (Eurospeech 1993)
For the present experiment we singled out from the corpus a few sequences which were present both... more For the present experiment we singled out from the corpus a few sequences which were present both in the connected text and in the word lists, and chose as a sample a male speaker who is coded as Ъ4' in the original database. The sequences chosen are: [-alia], [-ast-], [-ozo], ...
4th European Conference on Speech Communication and Technology (Eurospeech 1995), 1995
ISCA Archive http://www.isca-speech.org/archive 4th European Conference on Speech Communication a... more ISCA Archive http://www.isca-speech.org/archive 4th European Conference on Speech Communication and Technology EUROSPEECH '95 Madrid, Spain, September 18-21,1995 ON PHONETIC BOUNDARIES ACROSS CATEGORIES FOR SYNTHETIC AND NATURAL VOCALIC SPEECH ...
Interspeech 2013, 2013
Effective human-robot communication is one of the main concerns in modern robotics. Involved syst... more Effective human-robot communication is one of the main concerns in modern robotics. Involved systems should be very robust, allowing little chance for misunderstanding users commands. The main purpose of this work is to develop a general framework for multimodal human-robot communication, which allows users to interact with robots using speech and gestures, integrated into unique commands. The produced architecture relies on the definition of different modules separately analysing the low level inputs and presenting a further fusion module able to extract semantics from these multiple channels. In this paper, we introduce our general approach and provide a case study where gesture and speech modalities are combined.
EVALITA Evaluation of NLP and Speech Tools for Italian, 2018
English. The SUGAR task is intended to develop a baseline to train a voicecontrolled robotic agen... more English. The SUGAR task is intended to develop a baseline to train a voicecontrolled robotic agent to act as a cooking assistant. The starting point will be therefore to provide authentic spoken data collected in a simulated natural context from which semantic predicates will be extracted to classify the actions to perform. Three different approaches were used by the two SUGAR participants to solve the task. The enlightening results show the different elements of criticality underlying the task itself.
Multimedia Tools and Applications, 2019
With the recent availability of industry-grade, high-performing engines for video games productio... more With the recent availability of industry-grade, high-performing engines for video games production, researchers in different fields have been exploiting the advanced technologies offered by these artefacts to improve the quality of the interactive experiences they design. While these engines provide excellent and easy-to-use tools to design interfaces and complex rule-based systems to control the experience, there are some aspects of Human-Computer Interaction (HCI) research they do not support in the same way because of their original mission and related design patterns pointing at a different primary target audience. In particular, the more research in HCI evolves towards natural, socially engaging approaches, the more there is the need to rapidly design and deploy software architectures to support these new paradigms. Topics such as knowledge representation, probabilistic reasoning and voice synthesis demand space as possible instruments within this new ideal design environment. In this work, we propose a framework, named FANTASIA, designed to integrate a set of chosen modules (a graph database, a dialogue manager, a game engine and a voice synthesis engine) and support rapid design and implementation of interactive applications for HCI studies. We will present a number of different case studies to exemplify how the proposed tools can be deployed to develop very different kinds of interactive applications and we will discuss ongoing and future work to further extend the framework we propose.
Proceedings of the Second Italian Conference on Computational Linguistics CLiC-it 2015
English. The CompWHoB (Computational White House press Briefings) Corpus, currently being develop... more English. The CompWHoB (Computational White House press Briefings) Corpus, currently being developed at the University of Naples Federico II, is a corpus of spoken American English focusing on political and media communication. It represents a large collection of the White House Press Briefings, namely, the daily meetings held by the White House Press Secretary and the news media. At the time of writing, the corpus amounts to more than 20 million words, covers a period of time of twenty-one years spanning from 1993 to 2014 and it is planned to be extended to the end of the second term of President Barack Obama. The aim of the present article is to describe the composition of the corpus and the techniques used to extract, process and annotate it. Moreover, attention is paid to the use of the Temporal Random Indexing (TRI) on the corpus as a tool for linguistic analysis.
Lecture Notes in Computer Science, 2016
This paper presents a CAVE-like architecture to support the interaction for small groups of peopl... more This paper presents a CAVE-like architecture to support the interaction for small groups of people with a leader in a multi-projection environment in the unusual condition where a vertical depth camera records people and their movements. In this framework, modelling people as gaussians, we localise and track people when they step into a defined area. We compared our approach with a typical local minimum one and our algorithm results to be faster and more accurate. Detected leaders manage the interaction with hands. We developed a trained gesture recognition model and a rule-based one and the former approach reports better outcomes. While the proposed virtual environment is mainly intended as a multi-projection system, the presented architecture allows to dynamically change the area such as to integrate further input and output devices. It can be extended up to provide support in collaborative tasks for remotely connected groups acting in the same virtual room. The whole system has been adopted in Cultural Heritage scenarios to provide an immersive experience for art, historical contents or virtual environments. Interviews with people participating to the experimentation phase of the OrCHeSTRA project show that the system was well-received by the general public and that future extensions towards collaborative environments are encouraged by the end-users.
Prosodic prominence is an umbrella term encompassing various related but conceptually and functio... more Prosodic prominence is an umbrella term encompassing various related but conceptually and functionally different phenomena such as phonological stress, paralinguistic emphasis, lexical, syntactic, semantic or pragmatic salience, to mention a few. Due to the high interest prominence has received from various disciplines, it has been studied from multiple perspectives (functional, physical, cogni-tive). It also has been operationalised and annotated across different descriptive levels (syllable, word), based on different scales (categorical, multi-level, continuous), and measured across a large variety of signal domains (acoustic, articulatory, gestural). The present paper offers an overview of the various perspectives involved and defines a preliminary roadmap for a better and more unified understanding of this multi-faceted phenomenon.
2014 Tenth International Conference on Signal-Image Technology and Internet-Based Systems, 2014
ABSTRACT
Acta otorhinolaryngologica Italica : organo ufficiale della Società italiana di otorinolaringologia e chirurgia cervico-facciale
Bisyllabic words are the most frequently used italian speech material in evaluating intelligibili... more Bisyllabic words are the most frequently used italian speech material in evaluating intelligibility function. The italian words presently used are those proposed by Bocca and Pellegrini in 1950. The Lists of these words do, however, present some problems with regard to phonemic balance and word familiarity. In speech audiometry testing, Lists are considered interchangeable if each individual List has the same phonemic balance. So as to avoid incorrect identification due to incomprehension of infrequently used words, we chose 200 of the most familiar bisyllabic words from the most recent, widely used occurrence vocabulary of the Italian Language. Secondly, we proceeded with phonemic balance of the speech material. The selected words were divided into ten lists of 20 words each, arranged in order to obtain the best phonemic balance within each individual List and among different Lists. The differences between the new and old speech material are presented and discussed.
Proceedings of the 2014 IEEE 15th International Conference on Information Reuse and Integration (IEEE IRI 2014), 2014
One of the most important qualities of the physical environment in which humans live is spatial d... more One of the most important qualities of the physical environment in which humans live is spatial dimensionality. Talking about 3D, we usually think of 3D video, even if it is not the only existing channel of natural interaction. In this paper, we present an interaction system based on spatialized sounds. We developed an application in the cultural heritage context; a personal guide, in 3D sound, attracting the tourists attention toward monuments or buildings, offering soundscapes of augmented reality. The designed system interacts with smart headphones, remotely takes the orientation of the listener's head and properly generates an audio output, which also takes into account the listener's position and orientation in the environment. Thus, an innovative headphones using an inertial measurement unit for determination of the orientation of a users head have been designed and developed in open-ear mode, in order to locate the user in the real context.
L'A. discute certains problemes concernant la relation entre la phonetique et la phonologie. ... more L'A. discute certains problemes concernant la relation entre la phonetique et la phonologie. Il examine en particulier certaines questions concernant la reduction phonique dans le discours spontane italien. L'experience simule les mecanismes de la deterioration de maniere a les observer etape par etape et d'analyser leur effet sur la perception du discours
Proceedings of the 15th ACM on International conference on multimodal interaction - ICMI '13, 2013
Evaluating human machine interaction in the case of multimodal systems is often a difficult task ... more Evaluating human machine interaction in the case of multimodal systems is often a difficult task involving the monitoring of multiple sources, data fusion and results interpretation. While subtasks are highly dependent on the specific goal of the application and on the available interaction modalities, it is possible to formalize this workflow into a standard process and to consider a generic measure to estimate the ease of use of a specific application. In this work, we present CoWME, a modular software architecture describing multimodal human machine interaction evaluation, from data collection to final evaluation, in a formal way, in terms of cognitive workload. Communication protocols between modules are described in XML while data fusion is delegated to a configurable rule engine. An interface module is introduced between the monitoring modules and the rule engine to collect and summarize data streams for cognitive workload evaluation. We present a deployment example showing how this architecture is deployed by monitoring an interactive session with an Android application taking into account stressed speech detection, mydriasis and touch analysis.
Lecture Notes in Information Systems and Organisation, 2014
Pupil dilation is known to reflect the emotional arousal. Pleasure, effort and fear are examples ... more Pupil dilation is known to reflect the emotional arousal. Pleasure, effort and fear are examples of stimuli inducing the nervous system to cause dilation mydriasis. The work proposes a tool to automatically quantify the mydriasis in order to evaluate mental effort in HCI. The system uses a feature-based approach and monitors the pupil behavior during a given task. As mydriasis is entailed by various reasons, our system distinguishes the cause-effect relationships by synchronizing monitoring and test, dividing the monitoring in fixed intervals and retrieving a survey of the mydriatic events for each determined period of time. We present a case of study analyzing users resolving arithmetical tasks, viewing pictures and using a mobile application. In each scenario, tests intend to impose gradually increasing reactions to the users. The paper will present different techniques for pupil dilation measurements and related results of mental effort evaluation.
2014 Ninth International Conference on P2P, Parallel, Grid, Cloud and Internet Computing, 2014
Spatial dimensionality is one of the main features of the physical environment in which humans li... more Spatial dimensionality is one of the main features of the physical environment in which humans live. When we think about 3D, we usually refer to 3D video, even if it is not the only existing channel of natural interaction. In this paper, we present an interaction system based on spatialized sounds. We developed an innovative cloud application in the cultural heritage context, a personal guide, in 3D sound, attracting the tourists' attention toward monuments or buildings, offering sounds capes of augmented reality. The designed system interacts with smart headphones that remotely takes the orientation of the listener's head and properly generates an audio output, which also takes into account the listener's position and orientation in the environment. Thus, an innovative headphones using an inertial measurement unit for determining the orientation of a user's head have been designed and developed in open-ear mode, in order to locate the user in the real context.
The ability of representing spatio-temporal features plays a crucial role in Cultural Heritage fi... more The ability of representing spatio-temporal features plays a crucial role in Cultural Heritage field. Management of thematic contexts, of time domain with qualitative and imprecise references, hierarchical structure of time and spatial and temporal multiple granularities are very important feature in this environment for users aiming at discovering knowledge process. In this paper a framework to manage spatiotemporal cultural heritage data exploration and visualization is presented. This framework allows to develop web applications able to handle spatio-temporal objects, regardless of the specific descriptive nature of data, and relies on a flexible architecture for web applications that shows low coupling among tiers and uses standard files and protocols, like WFS, GML, KML, becoming independent from storage and visualization tools. The Visualization layer is designed to offer spatio-temporal views with a high level of personalization: users can choose different kinds of views and visual metaphors to compare objects on the geobrowser by activating or disabling layers of their interest. A new method aiming at visualizing and exploring the hierarchical and stratified time domain is presented. The user interface is open and extendable to new methods of visualization. Finally, a specific web application, based on the described framework, is shown.
3rd European Conference on Speech Communication and Technology (Eurospeech 1993)
For the present experiment we singled out from the corpus a few sequences which were present both... more For the present experiment we singled out from the corpus a few sequences which were present both in the connected text and in the word lists, and chose as a sample a male speaker who is coded as Ъ4' in the original database. The sequences chosen are: [-alia], [-ast-], [-ozo], ...
4th European Conference on Speech Communication and Technology (Eurospeech 1995), 1995
ISCA Archive http://www.isca-speech.org/archive 4th European Conference on Speech Communication a... more ISCA Archive http://www.isca-speech.org/archive 4th European Conference on Speech Communication and Technology EUROSPEECH '95 Madrid, Spain, September 18-21,1995 ON PHONETIC BOUNDARIES ACROSS CATEGORIES FOR SYNTHETIC AND NATURAL VOCALIC SPEECH ...
Interspeech 2013, 2013
Effective human-robot communication is one of the main concerns in modern robotics. Involved syst... more Effective human-robot communication is one of the main concerns in modern robotics. Involved systems should be very robust, allowing little chance for misunderstanding users commands. The main purpose of this work is to develop a general framework for multimodal human-robot communication, which allows users to interact with robots using speech and gestures, integrated into unique commands. The produced architecture relies on the definition of different modules separately analysing the low level inputs and presenting a further fusion module able to extract semantics from these multiple channels. In this paper, we introduce our general approach and provide a case study where gesture and speech modalities are combined.
EVALITA Evaluation of NLP and Speech Tools for Italian, 2018
English. The SUGAR task is intended to develop a baseline to train a voicecontrolled robotic agen... more English. The SUGAR task is intended to develop a baseline to train a voicecontrolled robotic agent to act as a cooking assistant. The starting point will be therefore to provide authentic spoken data collected in a simulated natural context from which semantic predicates will be extracted to classify the actions to perform. Three different approaches were used by the two SUGAR participants to solve the task. The enlightening results show the different elements of criticality underlying the task itself.
Multimedia Tools and Applications, 2019
With the recent availability of industry-grade, high-performing engines for video games productio... more With the recent availability of industry-grade, high-performing engines for video games production, researchers in different fields have been exploiting the advanced technologies offered by these artefacts to improve the quality of the interactive experiences they design. While these engines provide excellent and easy-to-use tools to design interfaces and complex rule-based systems to control the experience, there are some aspects of Human-Computer Interaction (HCI) research they do not support in the same way because of their original mission and related design patterns pointing at a different primary target audience. In particular, the more research in HCI evolves towards natural, socially engaging approaches, the more there is the need to rapidly design and deploy software architectures to support these new paradigms. Topics such as knowledge representation, probabilistic reasoning and voice synthesis demand space as possible instruments within this new ideal design environment. In this work, we propose a framework, named FANTASIA, designed to integrate a set of chosen modules (a graph database, a dialogue manager, a game engine and a voice synthesis engine) and support rapid design and implementation of interactive applications for HCI studies. We will present a number of different case studies to exemplify how the proposed tools can be deployed to develop very different kinds of interactive applications and we will discuss ongoing and future work to further extend the framework we propose.
Proceedings of the Second Italian Conference on Computational Linguistics CLiC-it 2015
English. The CompWHoB (Computational White House press Briefings) Corpus, currently being develop... more English. The CompWHoB (Computational White House press Briefings) Corpus, currently being developed at the University of Naples Federico II, is a corpus of spoken American English focusing on political and media communication. It represents a large collection of the White House Press Briefings, namely, the daily meetings held by the White House Press Secretary and the news media. At the time of writing, the corpus amounts to more than 20 million words, covers a period of time of twenty-one years spanning from 1993 to 2014 and it is planned to be extended to the end of the second term of President Barack Obama. The aim of the present article is to describe the composition of the corpus and the techniques used to extract, process and annotate it. Moreover, attention is paid to the use of the Temporal Random Indexing (TRI) on the corpus as a tool for linguistic analysis.
Lecture Notes in Computer Science, 2016
This paper presents a CAVE-like architecture to support the interaction for small groups of peopl... more This paper presents a CAVE-like architecture to support the interaction for small groups of people with a leader in a multi-projection environment in the unusual condition where a vertical depth camera records people and their movements. In this framework, modelling people as gaussians, we localise and track people when they step into a defined area. We compared our approach with a typical local minimum one and our algorithm results to be faster and more accurate. Detected leaders manage the interaction with hands. We developed a trained gesture recognition model and a rule-based one and the former approach reports better outcomes. While the proposed virtual environment is mainly intended as a multi-projection system, the presented architecture allows to dynamically change the area such as to integrate further input and output devices. It can be extended up to provide support in collaborative tasks for remotely connected groups acting in the same virtual room. The whole system has been adopted in Cultural Heritage scenarios to provide an immersive experience for art, historical contents or virtual environments. Interviews with people participating to the experimentation phase of the OrCHeSTRA project show that the system was well-received by the general public and that future extensions towards collaborative environments are encouraged by the end-users.
Prosodic prominence is an umbrella term encompassing various related but conceptually and functio... more Prosodic prominence is an umbrella term encompassing various related but conceptually and functionally different phenomena such as phonological stress, paralinguistic emphasis, lexical, syntactic, semantic or pragmatic salience, to mention a few. Due to the high interest prominence has received from various disciplines, it has been studied from multiple perspectives (functional, physical, cogni-tive). It also has been operationalised and annotated across different descriptive levels (syllable, word), based on different scales (categorical, multi-level, continuous), and measured across a large variety of signal domains (acoustic, articulatory, gestural). The present paper offers an overview of the various perspectives involved and defines a preliminary roadmap for a better and more unified understanding of this multi-faceted phenomenon.
2014 Tenth International Conference on Signal-Image Technology and Internet-Based Systems, 2014
ABSTRACT
Acta otorhinolaryngologica Italica : organo ufficiale della Società italiana di otorinolaringologia e chirurgia cervico-facciale
Bisyllabic words are the most frequently used italian speech material in evaluating intelligibili... more Bisyllabic words are the most frequently used italian speech material in evaluating intelligibility function. The italian words presently used are those proposed by Bocca and Pellegrini in 1950. The Lists of these words do, however, present some problems with regard to phonemic balance and word familiarity. In speech audiometry testing, Lists are considered interchangeable if each individual List has the same phonemic balance. So as to avoid incorrect identification due to incomprehension of infrequently used words, we chose 200 of the most familiar bisyllabic words from the most recent, widely used occurrence vocabulary of the Italian Language. Secondly, we proceeded with phonemic balance of the speech material. The selected words were divided into ten lists of 20 words each, arranged in order to obtain the best phonemic balance within each individual List and among different Lists. The differences between the new and old speech material are presented and discussed.
Proceedings of the 2014 IEEE 15th International Conference on Information Reuse and Integration (IEEE IRI 2014), 2014
One of the most important qualities of the physical environment in which humans live is spatial d... more One of the most important qualities of the physical environment in which humans live is spatial dimensionality. Talking about 3D, we usually think of 3D video, even if it is not the only existing channel of natural interaction. In this paper, we present an interaction system based on spatialized sounds. We developed an application in the cultural heritage context; a personal guide, in 3D sound, attracting the tourists attention toward monuments or buildings, offering soundscapes of augmented reality. The designed system interacts with smart headphones, remotely takes the orientation of the listener's head and properly generates an audio output, which also takes into account the listener's position and orientation in the environment. Thus, an innovative headphones using an inertial measurement unit for determination of the orientation of a users head have been designed and developed in open-ear mode, in order to locate the user in the real context.
L'A. discute certains problemes concernant la relation entre la phonetique et la phonologie. ... more L'A. discute certains problemes concernant la relation entre la phonetique et la phonologie. Il examine en particulier certaines questions concernant la reduction phonique dans le discours spontane italien. L'experience simule les mecanismes de la deterioration de maniere a les observer etape par etape et d'analyser leur effet sur la perception du discours
Proceedings of the 15th ACM on International conference on multimodal interaction - ICMI '13, 2013
Evaluating human machine interaction in the case of multimodal systems is often a difficult task ... more Evaluating human machine interaction in the case of multimodal systems is often a difficult task involving the monitoring of multiple sources, data fusion and results interpretation. While subtasks are highly dependent on the specific goal of the application and on the available interaction modalities, it is possible to formalize this workflow into a standard process and to consider a generic measure to estimate the ease of use of a specific application. In this work, we present CoWME, a modular software architecture describing multimodal human machine interaction evaluation, from data collection to final evaluation, in a formal way, in terms of cognitive workload. Communication protocols between modules are described in XML while data fusion is delegated to a configurable rule engine. An interface module is introduced between the monitoring modules and the rule engine to collect and summarize data streams for cognitive workload evaluation. We present a deployment example showing how this architecture is deployed by monitoring an interactive session with an Android application taking into account stressed speech detection, mydriasis and touch analysis.