Kristiina Jokinen | University of Helsinki (original) (raw)

Papers by Kristiina Jokinen

Sensors

Since life expectancy has increased significantly over the past century, society is being forced ... more Since life expectancy has increased significantly over the past century, society is being forced to discover innovative ways to support active aging and elderly care. The e-VITA project, which receives funding from both the European Union and Japan, is built on a cutting edge method of virtual coaching that focuses on the key areas of active and healthy aging. The requirements for the virtual coach were ascertained through a process of participatory design in workshops, focus groups, and living laboratories in Germany, France, Italy, and Japan. Several use cases were then chosen for development utilising the open-source Rasa framework. The system uses common representations such as Knowledge Bases and Knowledge Graphs to enable the integration of context, subject expertise, and multimodal data, and is available in English, German, French, Italian, and Japanese.

Increased use of digital devices and data repositories has enabled a digital revolution in data c... more Increased use of digital devices and data repositories has enabled a digital revolution in data collection and language research, and has also led to important activities supporting speech and language technology research for less-resourced languages. This paper describes the DigiSami project and its research results, focussing on spoken corpus collection and speech technology for the Fenno-Ugric language North Sami. The paper also discusses multifaceted questions on ethics and privacy related to data collection for less-resourced languages and indigenous communities.

This paper discusses how to extend cognitive models with an explicit interaction model. The work ... more This paper discusses how to extend cognitive models with an explicit interaction model. The work is based on the Standard Model of Cognitive Architecture which is extended by an explicit model for (spoken) interactions following the Constructive Dialogue Modelling (CDM) approach. The goal is to study how to integrate a cognitively appropriate framework into an architecture which allows smooth communication in human-robot interactions, and the starting point is to model construction of shared understanding of the dialogue context and the partner’s intentions. Implementation of conversational interaction is considered important in the context of social robotics which aim to understand and respond to the user’s needs and affective state. The paper describes integration of the architectures but not experimental work towards this goal.

Lecture Notes in Electrical Engineering, 2019

Future and Emerging Trends in Language Technology. Machine Learning and Big Data, 2017

This paper describes work on dialogue data collection and dialogue system design for personal ass... more This paper describes work on dialogue data collection and dialogue system design for personal assistant humanoid robots undertaken at eNTERFACE 2016. The emphasis has been on the system's speech capabilities and dialogue modeling of what we call LifeLine Dialogues, i.e. dialogues that help people tell stories about their lives. The main goal behind this type of application is to help elderly people exercise their speech and memory capabilities. The system further aims at acquiring a good level of knowledge about the person's interests and thus is expected to feature open-domain conversations, presenting useful and interesting information to the user. The novel contributions of this work are: (1) a flexible spoken dialogue system that extends the Ravenclawtype agent-based dialogue management model with topic management and multi-modal capabilities, especially with face recognition technologies, (2) a collection of WOZ-data related to initial encounters and presentation of information to the user, and (3) the establishment of a closer conversational relationship with the user by utilizing additional data (e.g. context, dialogue history, emotions, user goals, etc.).

Lecture Notes in Computer Science

The paper discusses usability and communicative capability of mobile multimodal systems. It repor... more The paper discusses usability and communicative capability of mobile multimodal systems. It reports on the evaluation of one particular interactive multimodal route navigation system and discusses the challenges encountered in this task. The main questions concerned the user's preference of one input mode over the other (speech vs. tactile/graphics input), usefulness of the system in completing the task (route navigation), and user satisfaction (willingness to use the system in the future). The user's expectations and real experience of the system were analysed by comparing the users' assessments before and after the system use. Conclusions concerning system design are drawn and discussed from the perspective of the system's communicative capability, based on the view of the computer as an interactive agent.

Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 2015

At SIGDIAL-2013 our talking robot demonstrated Wikipedia-based spoken information access in Engli... more At SIGDIAL-2013 our talking robot demonstrated Wikipedia-based spoken information access in English. Our new demo shows a robot speaking different languages, getting content from different language Wikipedias, and switching languages to meet the linguistic capabilities of different dialogue partners.

Proceedings of the Eleventh European Workshop on Natural Language Generation - ENLG '07, 2007

The paper discusses quality of service evaluation which emphasises the user's experience in the e... more The paper discusses quality of service evaluation which emphasises the user's experience in the evaluation of system functionality and efficiency. For NLG systems, an important quality feature is communicatively adequate language generation, which affects the users' perception of the system and consequently, evaluation results. The paper drafts an evaluation task that aims at measuring quality of service, taking the system's communicative competence into account.

Coverbal Synchrony in Human-Machine Interaction, 2013

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2010

Frontiers in Artificial Intelligence and Applications, 2012

This paper discusses multimodal feedback signalling in Finnish first encounter conversations, esp... more This paper discusses multimodal feedback signalling in Finnish first encounter conversations, especially the use of head nodding to signal shared understanding of the presented information. The goal of the paper is to study the correlation between gestures and speech, and to build a model to describe to which extent head movements correlate with verbal feedback. We distinguish single and repeated nodding, as well as up-nods and down-nods, and hypothesise that downnods are used as backchannels while up-nods signal unexpected information.

, except for brief excerpts in connection with reviews or scholarly analysis. Use in connection w... more , except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights.

International Journal of Human-Computer Studies, 2000

This paper describes some of the basic cooperative mechanisms of dialogue. Ideal cooperation is s... more This paper describes some of the basic cooperative mechanisms of dialogue. Ideal cooperation is seen as consisting of four features (cognitive consideration, joint purpose, ethical consideration and trust), which can also to some extent be seen as requirements building on each other. Weaker concepts such as &&coordination'' and &&collaboration'' have only some of these features or have them to lesser degrees. We point out the central role of ethics and trust in cooperation, and contrast the result with popular AI accounts of collaboration. Dialogue is also seen as associated with social activities, in which certain obligations and rights are connected with particular roles. Dialogue is seen to progress through the written, vocal or gestural contributions made by participants. Each of the contributions has associated with it both expressive and evocative functions, as well as speci"c obligations for participants. These functions are dependent on the surface form of a contribution, the activity and the local context, for their interpretation. We illustrate the perspective by analysing dialogue extracts from three di!erent activity types (a travel dialogue, a quarrel and a dialogue with a computer system). Finally, we consider what kind of information is shared in dialogue, and the ways in which dialogue participants manifest this sharing to each other through linguistic and other communicative behaviour. The paper concludes with a comparison to other accounts of dialogue and prospects for integration of these ideas within dialogue systems.

… Systems: Interaction, Adaptation and Styles of …, 2003

This paper describes a distributed dialogue management scheme for speechbased information seeking... more This paper describes a distributed dialogue management scheme for speechbased information seeking dialogue. The dialogue management is distributed to several components, supported by a general blackboard-type architecture for speech systems. By breaking down the dialogue management, we can achieve more general solutions and support sophisticated decision making algorithms.

Proceedings of Workshop on Effective Multimodal …, 2006

In this paper we present the MUMS Multimodal Route Navigation System which combines speech, pen, ... more In this paper we present the MUMS Multimodal Route Navigation System which combines speech, pen, and graphics into a PDA-based multimodal system. We focus especially on the three-level modality fusion component which we believe provides an accurate and ...

Designations used by companies to distinguish their products are often claimed as trademarks. All... more Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners. The publisher is not associated with any product or vendor mentioned in this book. This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold on the understanding that the publisher is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a competent professional should be sought. Library of Congress Cataloging-in-Publication Data Jokinen, Kristiina. Constructive dialogue modelling: speech interaction and rational agents/Kristiina Jokinen. p. cm. Includes bibliographical references and index.