Jin Ha Lee | University of Washington (original) (raw)

Papers by Jin Ha Lee

Digital Humanities 2008, 2008

Proceedings of the iConference 2014, 2014

Prior research suggests that listeners from different cultural backgrounds appreciate music diffe... more Prior research suggests that listeners from different cultural backgrounds appreciate music differently. Although music mood/emotion is an important part of music seeking and appreciation, few cross-cultural music information retrieval (MIR) studies focus on music mood. Moreover, existing studies on crosscultural music perception often only compare listeners from two cultures, in most cases, Western vs. Nonwestern cultures. In order to fill these gaps, this study compares music mood perceptions of listeners from three distinct cultures: American, Korean, and Chinese. Our findings reveal that the perceptions of the three cultural groups are generally different, but in many aspects, Korean listeners are situated in between listeners from the two other cultures. This paper describes the comparison of the three cultural groups from the perspectives of mood perceptions, musical (stimuli) characteristics, and listeners' (subjects) characteristics. The findings of this study have implications for the design of cross-cultural and global MIR systems.

Proceedings of the iConference 2014, 2014

Genre is an important feature for organizing and accessing video games. However, current descript... more Genre is an important feature for organizing and accessing video games. However, current descriptors of video game genres are unstandardized, undefined, and embedded with multiple information dimensions. This paper describes the development of a more complex and sophisticated scheme consisting of 12 facets and 358 foci for describing and representing video game genre information. Using facet analysis, the authors analyzed existing genre labels from scholarly, commercial, and popular sources, and then synthesized them into discrete categories of indexing terms. This new, more robust scheme provides a framework for improved intellectual access to video games along multiple dimensions.

Proceedings of the iConference 2012, 2012

Digital games are saturated with information. Massively Multiplayer Online games (MMOs) require p... more Digital games are saturated with information. Massively Multiplayer Online games (MMOs) require players to collect, organize, manage, and interpret vast volumes and varieties of information in a distributed, networked environment. Yet, they often provide players insufficient tools to effectively accomplish these information tasks. In response, some members of the player community build modifications (mods) and addons to the game software. Mods and addons usefully and creatively address some problems of utilizing information in digital environments; by analyzing them, we can gain insights into possibilities for organizing information in digital environments.

Proceedings of the 12th Annual ACM/IEEE-CS Joint Conference on Digital Libraries page 129-138, 2012

Mood is an important access point in music digital libraries and online music repositories, but g... more Mood is an important access point in music digital libraries and online music repositories, but generating ground truth for evaluating various music mood classification algorithms is a challenging problem. This is because collecting enough human judgments is time-consuming and costly due to the subjectivity of music mood. In this study, we explore the viability of crowdsourcing music mood classification judgments using Amazon Mechanical Turk (MTurk). Specifically, we compare the mood classification judgments collected for the annual Music Information Retrieval Evaluation eXchange (MIREX) with judgments collected using MTurk. Our data show that the overall distribution of mood clusters and agreement rates from MIREX and MTurk were comparable. However, Turkers tended to agree less with the pre-labeled mood clusters than MIREX evaluators. The system evaluation results generated using both sets of data were mostly the same except for detecting one statistically significant pair using Friedman's test. We conclude that MTurk can potentially serve as a viable alternative for ground truth collection, with some reservation with regards to particular mood clusters.

Lecture Notes in Computer Science Volume 7489 page 280-285, 2012

As interest in video games increases, so does the need for intelligent access to them. However, t... more As interest in video games increases, so does the need for intelligent access to them. However, traditional organization systems and standards fall short. Through domain analysis and cataloging real-world examples while attempting to develop a formal metadata schema for video games, we encountered challenges in description. Inconsistent, vague, and subjective sources of information for genre, release date, feature, region, language, developer and publisher information confirm the imporatnce of developing a standardized description model for video games.

The 13th Annual ACM/IEEE-CS Joint Conference on Digital Libraries, 2013

Video games and interactive media are increasingly becoming important part of our culture and eve... more Video games and interactive media are increasingly becoming important part of our culture and everyday life, and subsequently, of archival and digital library collections. However, existing organizational systems often use vague or inconsistent terms to describe video games or attempt to use schemas designed for textual bibliographic resources. Our research aims to create a standardized metadata schema and encoding scheme that provides an intelligent and comprehensive way to represent video games. We conducted interviews with 24 gamers, focusing on their video game-related information needs and seeking behaviors. We also performed a domain analysis of current organizational systems used in catalog records and popular game websites, evaluating metadata elements used to describe games. With these results in mind, we created a list of elements which form a metadata schema for describing video games, with both a core set of 16 elements and an extended set of 46 elements providing more flexibility in expressing the nature of a game.

Journal of Intelligent Information System, 2013

Most Music Information Retrieval (MIR) researchers will agree that understanding users' needs and... more Most Music Information Retrieval (MIR) researchers will agree that understanding users' needs and behaviors is critical for developing a good MIR system. The number of user studies in the MIR domain has been gradually increasing since the early 2000s, reflecting this growing appreciation of the need for empirical studies of users. However, despite the growing number of user studies and the wide recognition of their importance, it is unclear how great their impact has been in the field: on how systems are developed, how evaluation tasks are created, and how MIR system developers in particular understand critical concepts such as music similarity or music mood. In this paper, we present our analysis on the growth, publication and citation patterns, topics, and design of 198 user studies. This is followed by a discussion of a number of issues/challenges in conducting MIR user studies and distributing the research results.

The 13th Annual ACM/IEEE-CS Joint Conference on Digital Libraries, 2013

Despite the increase in interest in video games across commercial and academic areas, organizatio... more Despite the increase in interest in video games across commercial and academic areas, organizational systems for classifying them remain inadequate, particularly in describing the visual styles of video games. Because video games are by and large a visual medium, the ability to describe their visual "look" coherently and consistently greatly contributes to their discovery through classification. A set of controlled terms would be instrumental in complementing game recommendation engines and search applications in digital libraries to meet users' content-related information needs. In our study we examine the academic and user-generated content about video games' visual styles in order to extract potentially useful controlled vocabulary terms. These terms are then organized into facets and arranged into a classified schedule. In this poster, we discuss the challenges in our controlled vocabulary term definitions and their application.

International Journal on Digital Libraries, 2013

As interest in video games increases, so does the need for intelligent access to them. However, t... more As interest in video games increases, so does the need for intelligent access to them. However, traditional organizational systems and standards fall short. To fil this gap, we are collaborating with the Seattle Interactive Media Museum to develop a formal metadata schema for video games. In the paper, we describe how the schema was established from a user-centered design approach and introduce the core elements from our schema. We also discuss the challenges we encountered as we were conducting a domain analysis and cataloging real-world examples of video games. Inconsistent, vague, and subjective sources of information for title, genre, release date, feature, region, language, developer and publisher information confir the importance of developing a standardized description model for video games.

ABSTRACT Most Music Information Retrieval (MIR) researchers will agree that understanding users' ... more ABSTRACT Most Music Information Retrieval (MIR) researchers will agree that understanding users' needs and behaviors is critical for developing a good MIR system. The number of user studies in the MIR domain has been gradually increasing since the early 2000s reflecting the need for empirical studies of users.

ABSTRACT Music mood has been recognized as an important access point for music and many online mu... more ABSTRACT Music mood has been recognized as an important access point for music and many online music services support browsing by mood. However, how people judge music mood has not been well studied in the Music Information Retrieval (MIR) domain. In particular, people's cultural background is often assumed to be an important factor in music mood perception, but this assumption has not been verified by empirical studies.

ABSTRACT User studies in the music information retrieval and music digital library fields have be... more ABSTRACT User studies in the music information retrieval and music digital library fields have been gradually increasing in recent years, but large-scale studies that can help detect common user behaviors are still lacking. We have conducted a large-scale user survey in which we asked numerous questions related to users' music needs, uses, seeking, and management behaviors.

ABSTRACT This paper is a preliminary report of findings in an ongoing study of the role of music ... more ABSTRACT This paper is a preliminary report of findings in an ongoing study of the role of music in the lives of homeless young people which is taking place in Vancouver, British Columbia and Seattle, WA. One hundred homeless young people in Vancouver took part in online surveys, 20 of these young people participated in interviews and 64 completed design activities. Surveys included demographic and music questions. Interviews consisted of questions about music listening and preferences.

Abstract This poster describes a pilot study investigating what, if any, associative thesaural re... more Abstract This poster describes a pilot study investigating what, if any, associative thesaural relationships are identifiable and distinguishable to information seekers. Using associative relationships from the Library of Congress Subject Headings (LCSH), we asked potential library users via Amazon Mechanical Turk to identify six narrower distinctions of established associative relationship types.

Proceedings of the iConference 2012, 2012

Mood has recently received increasing attention as an interesting approach for organizing and acc... more Mood has recently received increasing attention as an interesting approach for organizing and accessing music. Our understanding of how users determine and describe music mood, however, is not fully developed. In this exploratory study, we investigate the concept of music mood from the end-user's perspective. In particular, we want to see how users describe music mood in their own terms as they react to different musical features. We investigate this by asking users to provide mood tags for various cover versions of the same song. The findings suggest that users rely on a small limited set of mood terms, although they do use a wide variety of terms. Typically, certain moods seem to carry over multiple cover versions despite differences in musical features. Along with lyrics, tempo, instrumentation, and delivery, factors like sources of mood, genre, musical expectancy, cultural context, etc. also seem to affect how people feel about music.

Proceedings of the ACM SIGIR 2005 Workshop on Information Retrieval in Context (IRIX) page 33-36, 2005

In this paper, we propose a framework to exploit user-generated context metadata from Questions a... more In this paper, we propose a framework to exploit user-generated context metadata from Questions and Answers (Q&A) websites where users provide natural language descriptions of information objects they aim to identify. Users' descriptions from Q&A websites are good sources of obtaining metadata regarding the real-life search context of real users in their information retrieval (IR) processes. We describe our suggested framework and how it may help address some known difficulties in exploiting usergenerated metadata and metadata schemes for Information Retrieval in Context (IRiX).

Proceedings of the …, Jan 1, 2005

In this poster we present preliminary findings of an exploratory study of natural language music ... more In this poster we present preliminary findings of an exploratory study of natural language music information queries posted to the Google Answers web site. We discuss the proportion of queries answered as a function of time and attempt to identify factors which affect the probability of a query being answered.

Digital Humanities 2008, 2008

Proceedings of the iConference 2014, 2014

Prior research suggests that listeners from different cultural backgrounds appreciate music diffe... more Prior research suggests that listeners from different cultural backgrounds appreciate music differently. Although music mood/emotion is an important part of music seeking and appreciation, few cross-cultural music information retrieval (MIR) studies focus on music mood. Moreover, existing studies on crosscultural music perception often only compare listeners from two cultures, in most cases, Western vs. Nonwestern cultures. In order to fill these gaps, this study compares music mood perceptions of listeners from three distinct cultures: American, Korean, and Chinese. Our findings reveal that the perceptions of the three cultural groups are generally different, but in many aspects, Korean listeners are situated in between listeners from the two other cultures. This paper describes the comparison of the three cultural groups from the perspectives of mood perceptions, musical (stimuli) characteristics, and listeners' (subjects) characteristics. The findings of this study have implications for the design of cross-cultural and global MIR systems.

Proceedings of the iConference 2014, 2014

Genre is an important feature for organizing and accessing video games. However, current descript... more Genre is an important feature for organizing and accessing video games. However, current descriptors of video game genres are unstandardized, undefined, and embedded with multiple information dimensions. This paper describes the development of a more complex and sophisticated scheme consisting of 12 facets and 358 foci for describing and representing video game genre information. Using facet analysis, the authors analyzed existing genre labels from scholarly, commercial, and popular sources, and then synthesized them into discrete categories of indexing terms. This new, more robust scheme provides a framework for improved intellectual access to video games along multiple dimensions.

Proceedings of the iConference 2012, 2012

Digital games are saturated with information. Massively Multiplayer Online games (MMOs) require p... more Digital games are saturated with information. Massively Multiplayer Online games (MMOs) require players to collect, organize, manage, and interpret vast volumes and varieties of information in a distributed, networked environment. Yet, they often provide players insufficient tools to effectively accomplish these information tasks. In response, some members of the player community build modifications (mods) and addons to the game software. Mods and addons usefully and creatively address some problems of utilizing information in digital environments; by analyzing them, we can gain insights into possibilities for organizing information in digital environments.

Proceedings of the 12th Annual ACM/IEEE-CS Joint Conference on Digital Libraries page 129-138, 2012

Mood is an important access point in music digital libraries and online music repositories, but g... more Mood is an important access point in music digital libraries and online music repositories, but generating ground truth for evaluating various music mood classification algorithms is a challenging problem. This is because collecting enough human judgments is time-consuming and costly due to the subjectivity of music mood. In this study, we explore the viability of crowdsourcing music mood classification judgments using Amazon Mechanical Turk (MTurk). Specifically, we compare the mood classification judgments collected for the annual Music Information Retrieval Evaluation eXchange (MIREX) with judgments collected using MTurk. Our data show that the overall distribution of mood clusters and agreement rates from MIREX and MTurk were comparable. However, Turkers tended to agree less with the pre-labeled mood clusters than MIREX evaluators. The system evaluation results generated using both sets of data were mostly the same except for detecting one statistically significant pair using Friedman's test. We conclude that MTurk can potentially serve as a viable alternative for ground truth collection, with some reservation with regards to particular mood clusters.

Lecture Notes in Computer Science Volume 7489 page 280-285, 2012

As interest in video games increases, so does the need for intelligent access to them. However, t... more As interest in video games increases, so does the need for intelligent access to them. However, traditional organization systems and standards fall short. Through domain analysis and cataloging real-world examples while attempting to develop a formal metadata schema for video games, we encountered challenges in description. Inconsistent, vague, and subjective sources of information for genre, release date, feature, region, language, developer and publisher information confirm the imporatnce of developing a standardized description model for video games.

The 13th Annual ACM/IEEE-CS Joint Conference on Digital Libraries, 2013

Video games and interactive media are increasingly becoming important part of our culture and eve... more Video games and interactive media are increasingly becoming important part of our culture and everyday life, and subsequently, of archival and digital library collections. However, existing organizational systems often use vague or inconsistent terms to describe video games or attempt to use schemas designed for textual bibliographic resources. Our research aims to create a standardized metadata schema and encoding scheme that provides an intelligent and comprehensive way to represent video games. We conducted interviews with 24 gamers, focusing on their video game-related information needs and seeking behaviors. We also performed a domain analysis of current organizational systems used in catalog records and popular game websites, evaluating metadata elements used to describe games. With these results in mind, we created a list of elements which form a metadata schema for describing video games, with both a core set of 16 elements and an extended set of 46 elements providing more flexibility in expressing the nature of a game.

Journal of Intelligent Information System, 2013

Most Music Information Retrieval (MIR) researchers will agree that understanding users' needs and... more Most Music Information Retrieval (MIR) researchers will agree that understanding users' needs and behaviors is critical for developing a good MIR system. The number of user studies in the MIR domain has been gradually increasing since the early 2000s, reflecting this growing appreciation of the need for empirical studies of users. However, despite the growing number of user studies and the wide recognition of their importance, it is unclear how great their impact has been in the field: on how systems are developed, how evaluation tasks are created, and how MIR system developers in particular understand critical concepts such as music similarity or music mood. In this paper, we present our analysis on the growth, publication and citation patterns, topics, and design of 198 user studies. This is followed by a discussion of a number of issues/challenges in conducting MIR user studies and distributing the research results.

The 13th Annual ACM/IEEE-CS Joint Conference on Digital Libraries, 2013

Despite the increase in interest in video games across commercial and academic areas, organizatio... more Despite the increase in interest in video games across commercial and academic areas, organizational systems for classifying them remain inadequate, particularly in describing the visual styles of video games. Because video games are by and large a visual medium, the ability to describe their visual "look" coherently and consistently greatly contributes to their discovery through classification. A set of controlled terms would be instrumental in complementing game recommendation engines and search applications in digital libraries to meet users' content-related information needs. In our study we examine the academic and user-generated content about video games' visual styles in order to extract potentially useful controlled vocabulary terms. These terms are then organized into facets and arranged into a classified schedule. In this poster, we discuss the challenges in our controlled vocabulary term definitions and their application.

International Journal on Digital Libraries, 2013

As interest in video games increases, so does the need for intelligent access to them. However, t... more As interest in video games increases, so does the need for intelligent access to them. However, traditional organizational systems and standards fall short. To fil this gap, we are collaborating with the Seattle Interactive Media Museum to develop a formal metadata schema for video games. In the paper, we describe how the schema was established from a user-centered design approach and introduce the core elements from our schema. We also discuss the challenges we encountered as we were conducting a domain analysis and cataloging real-world examples of video games. Inconsistent, vague, and subjective sources of information for title, genre, release date, feature, region, language, developer and publisher information confir the importance of developing a standardized description model for video games.

ABSTRACT Most Music Information Retrieval (MIR) researchers will agree that understanding users' ... more ABSTRACT Most Music Information Retrieval (MIR) researchers will agree that understanding users' needs and behaviors is critical for developing a good MIR system. The number of user studies in the MIR domain has been gradually increasing since the early 2000s reflecting the need for empirical studies of users.

ABSTRACT Music mood has been recognized as an important access point for music and many online mu... more ABSTRACT Music mood has been recognized as an important access point for music and many online music services support browsing by mood. However, how people judge music mood has not been well studied in the Music Information Retrieval (MIR) domain. In particular, people's cultural background is often assumed to be an important factor in music mood perception, but this assumption has not been verified by empirical studies.

ABSTRACT User studies in the music information retrieval and music digital library fields have be... more ABSTRACT User studies in the music information retrieval and music digital library fields have been gradually increasing in recent years, but large-scale studies that can help detect common user behaviors are still lacking. We have conducted a large-scale user survey in which we asked numerous questions related to users' music needs, uses, seeking, and management behaviors.

ABSTRACT This paper is a preliminary report of findings in an ongoing study of the role of music ... more ABSTRACT This paper is a preliminary report of findings in an ongoing study of the role of music in the lives of homeless young people which is taking place in Vancouver, British Columbia and Seattle, WA. One hundred homeless young people in Vancouver took part in online surveys, 20 of these young people participated in interviews and 64 completed design activities. Surveys included demographic and music questions. Interviews consisted of questions about music listening and preferences.

Abstract This poster describes a pilot study investigating what, if any, associative thesaural re... more Abstract This poster describes a pilot study investigating what, if any, associative thesaural relationships are identifiable and distinguishable to information seekers. Using associative relationships from the Library of Congress Subject Headings (LCSH), we asked potential library users via Amazon Mechanical Turk to identify six narrower distinctions of established associative relationship types.

Proceedings of the iConference 2012, 2012

Mood has recently received increasing attention as an interesting approach for organizing and acc... more Mood has recently received increasing attention as an interesting approach for organizing and accessing music. Our understanding of how users determine and describe music mood, however, is not fully developed. In this exploratory study, we investigate the concept of music mood from the end-user's perspective. In particular, we want to see how users describe music mood in their own terms as they react to different musical features. We investigate this by asking users to provide mood tags for various cover versions of the same song. The findings suggest that users rely on a small limited set of mood terms, although they do use a wide variety of terms. Typically, certain moods seem to carry over multiple cover versions despite differences in musical features. Along with lyrics, tempo, instrumentation, and delivery, factors like sources of mood, genre, musical expectancy, cultural context, etc. also seem to affect how people feel about music.

Proceedings of the ACM SIGIR 2005 Workshop on Information Retrieval in Context (IRIX) page 33-36, 2005

In this paper, we propose a framework to exploit user-generated context metadata from Questions a... more In this paper, we propose a framework to exploit user-generated context metadata from Questions and Answers (Q&A) websites where users provide natural language descriptions of information objects they aim to identify. Users' descriptions from Q&A websites are good sources of obtaining metadata regarding the real-life search context of real users in their information retrieval (IR) processes. We describe our suggested framework and how it may help address some known difficulties in exploiting usergenerated metadata and metadata schemes for Information Retrieval in Context (IRiX).

Proceedings of the …, Jan 1, 2005

In this poster we present preliminary findings of an exploratory study of natural language music ... more In this poster we present preliminary findings of an exploratory study of natural language music information queries posted to the Google Answers web site. We discuss the proportion of queries answered as a function of time and attempt to identify factors which affect the probability of a query being answered.