Toward an understanding of the history and impact of user studies in music information retrieval (original) (raw)
Related papers
The Impact (or Non-impact) of User Studies in Music Information Retrieval
2012
ABSTRACT Most Music Information Retrieval (MIR) researchers will agree that understanding users' needs and behaviors is critical for developing a good MIR system. The number of user studies in the MIR domain has been gradually increasing since the early 2000s reflecting the need for empirical studies of users.
User studies in the Music Information Retrieval Literature
This paper presents an overview of user studies in the Music Information Retrieval (MIR) literature. A focus on the user has repeatedly been identified as a key requirement for future MIR research; yet empirical user studies have been relatively sparse in the literature, the overwhelming research attention in MIR remaining systems-focused. We present research topics, methodologies, and design implications covered in the user studies conducted thus far.
The neglected user in music information retrieval research
Journal of Intelligent Information Systems, 2013
Personalization and context-awareness are highly important topics in research on Intelligent Information Systems. In the fields of Music Information Retrieval (MIR) and Music Recommendation in particular, user-centric algorithms should ideally provide music that perfectly fits each individual listener in each imaginable situation and for each of her information or entertainment needs. Even though preliminary steps towards such systems have recently been presented at the "International Society for Music Information Retrieval Conference" (ISMIR) and at similar venues, this vision is still far away from becoming a reality. In this article, we investigate and discuss literature on the topic of user-centric music retrieval and reflect on why the breakthrough in this field has not been achieved yet. Given the different expertises of the authors, we shed light on why this topic is a particularly challenging one, taking computer science and psychology points of view. Whereas the computer science aspect centers on the problems of user modeling, machine learning, J Intell Inf Syst and evaluation, the psychological discussion is mainly concerned with proper experimental design and interpretation of the results of an experiment. We further present our ideas on aspects crucial to consider when elaborating user-aware music retrieval systems.
A User-Oriented Approach to Music Information Retrieval
Dagstuhl Seminar Proceedings, 2006
Abstract. Search and retrieval of specific musical content such as emotive or sonic features has become an important aspect of Music Information Retrieval system development, but only little research is user-oriented. We summarize results of an elaborate user-study that explores ...
How potential users of music search and retrieval systems describe the semantic quality of music
2008
A large-scale study was set up aiming at the clarification of the influence of demographic and musical background on the semantic description of music. Our model for rating high-level music qualities distinguishes between affective/ emotive, structural and kinaesthetic descriptors. The focus was on the understanding of the most important attributes of music in view of the development of efficient search and retrieval systems. We emphasized who the users of such systems are and how they describe their favorite music. Particular interest went to inter-subjective similarities among listeners. The results from our study suggest that gender, age, musical expertise, active musicianship, broadness of taste and familiarity with the music have an influence on the semantic description of music.
Personalized and user-aware systems for retrieving multimedia items are becoming increasingly important as the amount of available multimedia data has been spiraling. A personalized system is one that incorporates information about the user into its data processing part (e.g., a particular user taste for a movie genre). A context-aware system, in contrast, takes into account dynamic aspects of the user context when processing the data (e.g., location and time where/when a user issues a query). Today's user-adaptive systems often incorporate both aspects.
Survey of music information needs, uses, and seeking behaviours: preliminary findings
ISMIR Proceedings, 2004
User studies focusing upon real-life music information needs, uses and seeking behaviours are still very scarce in the music information retrieval (MIR) and music digital library (MDL) fields. We are conducting a multigroup survey in an attempt to acquire information that can help eradicate false assumptions in designing MIR systems. Our goal is to provide an empirical basis for MIR/MDL system development. In this paper, we present our preliminary findings and analyses based on the 427 user responses we have received to date. Two major themes have been uncovered thus far that could have a significant influence the future development of successful MIR/MDL systems. First, people display "public information-seeking" behaviours by making use of collective knowledge and/or opinions of others about music such as reviews, ratings, recommendations, etc. in their music information-seeking. Second, respondents expressed needs for contextual metadata in addition to traditional bibliographic metadata.
Conversations With Expert Users In Music Retrieval And Research Challenges For Creative Mir
2016
Sample retrieval remains a central problem in the creative process of making electronic dance music. This paper describes the findings from a series of interview sessions involving users working creatively with electronic music. We conducted in-depth interviews with expert users on location at the Red Bull Music Academies in 2014 and 2015. When asked about their wishes and expectations for future technological developments in interfaces, most participants mentioned very practical requirements of storing and retrieving files. A central aspect of the desired systems is the need to provide increased flow and unbroken periods of concentration and creativity. From the interviews, it becomes clear that for Creative MIR, and in particular, for music interfaces for creative expression, traditional requirements and paradigms for music and audio retrieval differ to those from consumer-centered MIR tasks such as playlist generation and recommendation and that new paradigms need to be considered. Despite all technical aspects being controllable by the experts themselves, searching for sounds to use in composition remains a largely semantic process. From the outcomes of the interviews, we outline a series of possible conclusions and areas and pose two research challenges for future developments of sample retrieval interfaces in the creative domain. 1. MOTIVATION AND CONTEXT Considerable effort has been put into analysing user behaviour in the context of music retrieval in the past two decades [35]. This includes studies on music information seeking behaviour [14,17], organisation strategies [15], usage of commercial listening services [36], the needs or motivations of particular users, such as kids [28], adolescents [34], or musicologists [29], and behaviour analysis for specific tasks, e.g., playlist and mix generation [13], or in specific settings, e.g., riding together in a car [16] or in music lessons in secondary schools [49].
Users' relevance criteria in music retrieval in everyday life: An exploratory study
Proceedings of the 11th International Society for Music Information Retrieval Conference, 2010
The paper presents the findings of a qualitative study on the way young adults make relevance inferences about music items when searching for music for recreational purposes. Data were collected through in-depth interviews and analyzed following the constant comparative method. Content analysis revealed that participants used four types of clues to make relevance inferences: bibliographic metadata (e.g., names of contributors, labels), relational metadata (e.g., genres, similar artists), associative metadata (e.g., cover arts), and recommendations/reviews. Relevance judgments were also found to be influenced by the external context (i.e., the functions music plays in one’s life) and the internal context (i.e., individual tastes and beliefs, state of mind).
Journal of the Association for Information Science and Technology, 2015
Music Information Retrieval (MIR) evaluation has traditionally focused upon system-centered approaches where components of MIR systems are evaluated against predefined datasets and golden answers (i.e., ground truth). There are two major limitations of such system-centered evaluation approaches: 1) the evaluation focuses on subtasks in music information retrieval but not entire systems; and 2) users and their interactions with MIR systems are largely excluded. This paper describes the first implementation of a holistic user experience evaluation in MIR, the MIREX Grand Challenge, where complete MIR systems are evaluated with user experience being the single overarching goal. It is the first time complete MIR systems have been evaluated with end-users in a realistic scenario. We present the design of the evaluation task, the evaluation criteria and a novel evaluation interface and data collection platform. This is followed by an analysis of the results, reflection of the experience and lessons learned, and plans for future directions.