Kathleen R. Agres | National University of Singapore (original) (raw)

Papers by Kathleen R. Agres

Statistical learning (SL) is the ability to generate predictions based on probabilistic dependenc... more Statistical learning (SL) is the ability to generate predictions based on probabilistic dependencies in the environment, an ability that is present throughout life. The effect of aging on SL is still unclear. Here, we explore statistical learning in healthy adults (40 younger and 40 older). The novel paradigm tracks learning trajectories and shows age-related differences in overall performance, yet similarities in learning rates. Bayesian models reveal further differences between younger and older adults in dealing with uncertainty in this probabilistic SL task. We test computational models of 3 different learning strategies: (a) Win-Stay, Lose-Shift, (b) Delta Rule Learning, (c) Information Weights to explore whether they capture age-related differences in performance and learning in the present task. A likely candidate mechanism emerges in the form of age-dependent differences in information weights, in which young adults more readily change their behavior, but also show disproportionally strong reactions toward erroneous predictions. With lower but more balanced information weights, older adults show slower behavioral adaptation but eventually arrive at more stable and accurate representations of the underlying transitional probability matrix.

Frontiers in Psychology, 2021

In recent years, the field of music therapy (MT) has increasingly embraced the use of technology ... more In recent years, the field of music therapy (MT) has increasingly embraced the use of technology for conducting therapy sessions and enhancing patient outcomes. Amidst a worldwide pandemic, we sought to examine whether this is now true to an even greater extent, as many music therapists have had to approach and conduct their work differently. The purpose of this survey study is to observe trends in how music therapists from different regions around the world have had to alter their practice, especially in relation to their use of technology during the COVID-19 pandemic, because of limited options to conduct in-person therapy due to social distancing measures. Further, the findings aim to clarify music therapists’ perspectives on the benefits and limitations of technology in MT, as well as online MT. In addition, this survey investigated what changes have been necessary to administer MT during COVID-19, in terms of virtual therapy and online tools, and how the changes made now may affect MT in the future. We also explored music therapists’ views on whether special technology-focused training might be helpful to support the practice of MT in the future. This is the first survey, to our knowledge, to break down opinions of and trends in technology use based on geographical region (North America, Europe, and Asia), and several noteworthy differences were apparent across regions. We hope our findings provide useful information, guidance, and a global reference point for music therapists on effectively continuing the practice of MT during times of crisis, and can encourage reflection and improvement in administering MT.

The fields of music, health, and technology have seen significant interactions in recent years in... more The fields of music, health, and technology have seen significant interactions in recent years in developing music technology for health care and well-being. In an effort to strengthen the collaboration between the involved disciplines, the workshop "Music, Computing, and Health" was held to discuss best practices and state-of-the-art at the intersection of these areas with researchers from music psychology and neuroscience, music therapy, music information retrieval, music technology, medical technology (medtech), and robotics. Following the discussions at the workshop, this article provides an overview of the different methods of the involved disciplines and their potential contributions to developing music technology for health and well-being. Furthermore, the article summarizes the state of the art in music technology that can be applied in various health scenarios and provides a perspective on challenges and opportunities for developing music technology that (1) supports person-centered care and evidence-based treatments, and (2) contributes to developing standardized, large-scale research on music-based interventions in an interdisciplinary manner. The article provides a resource for those seeking to engage in interdisciplinary research using music-based computational methods to develop technology for health care, and aims to inspire future research directions by evaluating the state of the art with respect to the challenges facing each field.

Psychology of Music, 2019

Research into vision has highlighted the importance of gist representations in change detection a... more Research into vision has highlighted the importance of gist representations in change detection and memory. This article puts forth the hypothesis that schematic processing and gist provide an account for change detection in music as well, where a musical gist is an abstracted memory representation for schematically consistent tones. The present experiments illuminate the content of gist memory representations by testing when listeners can detect single-tone changes in pairs of melodies. In Experiment 1, musicians and non-musicians listened to melodies varying in tonal structure. Less structure resulted in compromised change detection in both groups. Most often, musicians displayed more accurate change detection than non-musicians, but, surprisingly, when schematic processing could not contribute to memory encoding, musicians performed worse than their untrained counterparts. Experiment 2 utilized a full-factorial design to examine tonality, interval of pitch change, metrical position, and rhythm. Tonality had a particularly large effect on performance, with non-scale tones generally aiding change detection. Listeners were unlikely, however, to detect schematically inconsistent tones when only brief melodic context was available. The results uphold the hypothesis that memory for melodies relies on schematic processing, with change detection dependent upon whether the change alters the schematic gist of the melody.

PLOS ONE, 2019

Emotions play a critical role in rational and intelligent behavior; a better fundamental knowledg... more Emotions play a critical role in rational and intelligent behavior; a better fundamental knowledge of them is indispensable for understanding higher order brain function. We propose a non-invasive brain-computer interface (BCI) system to feedback a person's affective state such that a closed-loop interaction between the participant's brain responses and the musical stimuli is established. We realized this concept technically in a functional prototype of an algorithm that generates continuous and controllable patterns of synthesized affective music in real-time, which is embedded within a BCI architecture. We evaluated our concept in two separate studies. In the first study, we tested the efficacy of our music algorithm by measuring subjective affective responses from 11 participants. In a second pilot study, the algorithm was embedded in a real-time BCI architecture to investigate affective closed-loop interactions in 5 participants. Preliminary results suggested that participants were able to intentionally modulate the musical feedback by self-inducing emotions (e.g., by recalling memories), suggesting that the system was able not only to capture the listener's current affective state in real-time, but also potentially provide a tool for listeners to mediate their own emotions by interacting with music. The proposed concept offers a tool to study emotions in the loop, promising to cast a complementary light on emotion-related brain research, particularly in terms of clarifying the interactive, spatio-temporal dynamics underlying affec-tive processing in the brain.

Neural Computing and Applications, 2019

We explore the potential of a popular distributional semantics vector space model, word2vec, for ... more We explore the potential of a popular distributional semantics vector space model, word2vec, for capturing meaningful relationships in ecological (complex poly-phonic) music. More precisely, the skip-gram version of word2vec is used to model slices of music from a large corpus spanning eight musical genres. In this newly learned vector space, a metric based on cosine distance is able to distinguish between functional chord relationships, as well as harmonic associations in the music. Evidence, based on cosine distance between chord-pair vectors, suggests that an implicit circle-of-fifths exists in the vector space. In addition, a comparison between pieces in different keys reveals that key relationships are represented in word2vec space. These results suggest that the newly learned embedded vector representation does in fact capture tonal and harmonic characteristics of music, without receiving explicit information about the musical content of the constituent slices. In order to investigate whether proximity in the discovered space of embeddings is indicative of 'semantically-related' slices, we explore a music generation task, by automatically replacing existing slices from a given piece of music with new slices. We propose an algorithm to find substitute slices based on spatial proximity and the pitch class distribution inferred in the chosen subspace. The results indicate that the size of the subspace used has a significant effect on whether slices belonging to the same key are

Cognitive Science, 2018

A basic function of cognition is to detect regularities in sensory input to facilitate the predic... more A basic function of cognition is to detect regularities in sensory input to facilitate the prediction and recognition of future events. It has been proposed that these implicit expectations arise from an internal predictive coding model, based on knowledge acquired through processes such as statistical learning, but it is unclear how different types of statistical information affect listeners’ memory for auditory stimuli. We used a combination of behavioral and computational methods to investigate memory for non-linguistic auditory sequences. Participants repeatedly heard tone sequences varying systematically in their information-theoretic properties. Expectedness ratings of tones were collected during three listening sessions, and a recognition memory test was given after each session. Information-theoretic measures of sequential predictability significantly influenced listeners’ expectedness ratings, and variations in these properties had a significant impact on memory performance. Predictable sequences yielded increasingly better memory performance with increasing exposure. Computational simulations using a probabilistic model of auditory expectation suggest that listeners dynamically formed a new, and increasingly accurate, implicit cognitive model of the information-theoretic structure of the sequences throughout the experimental session.

ACM Computers in Entertainment: Sound and Music Computing, 2016

The field of computational creativity, including musical metacreation, strives to develop artific... more The field of computational creativity, including musical metacreation, strives to develop artificial systems that are capable of demonstrating creative behavior or producing creative artefacts. But the claim of creativity is often assessed, subjectively only on the part of the researcher and not objectively at all. This article provides theoretical motivation for more systematic evaluation of musical metacreation and computationally creative systems and presents an overview of current methods used to assess human and machine creativity that may be adapted for this purpose. In order to highlight the need for a varied set of evaluation tools, a distinction is drawn among three types of creative systems: those that are purely generative, those that contain internal or external feedback, and those that are capable of reflection and self-reflection. To address the evaluation of each of these aspects, concrete examples of methods and techniques are suggested to help researchers (1) evaluate their systems' creative process and generated artefacts, and test their impact on the perceptual, cognitive, and affective states of the audience, and (2) build mechanisms for reflection into the creative system, including models of human perception and cognition, to endow creative systems with internal evaluative mechanisms to drive self-reflective processes. The first type of evaluation can be considered external to the creative system and may be employed by the researcher to both better understand the efficacy of their system and its impact and to incorporate feedback into the system. Here we take the stance that understanding human creativity can lend insight to computational approaches, and knowledge of how humans perceive creative systems and their output can be incorporated into artificial agents as feedback to provide a sense of how a creation will impact the audience. The second type centers around internal evaluation, in which the system is able to reason about its own behavior and generated output. We argue that creative behavior cannot occur without feedback and reflection by the creative/metacreative system itself. More rigorous empirical testing will allow computational and metacreative systems to become more creative by definition and can be used to demonstrate the impact and novelty of particular approaches. ACM Reference Format: Kat Agres, Jamie Forth, and Geraint A. Wiggins. 2016. Evaluation of musical creativity and musical metacre-ation systems.

Proceedings of the IEEE International Conference on Orange Technologies, 2017

Traditional physical therapy methods require significant time from trained medical staff, which i... more Traditional physical therapy methods require significant time from trained medical staff, which is costly for clinics and hospitals, and often leave patients bored and unmotivated to complete their exercises. We offer a prototype for a motion-detection and music game to inspire greater engagement and adherence from patients undergoing physical therapy exercises for rehabilitation or strengthening. The game is customizable based on the patient's needs, dynamically reacts to the patient's performance in real-time, and may be used with or without the guidance of a medical professional.

19th International Society for Music Information Retrieval Conference, 2018

Millions of individuals around the world suffer from motor impairment or disability, yet effectiv... more Millions of individuals around the world suffer from motor impairment or disability, yet effective, engaging, and cost-effective therapeutic solutions are still lacking. In this work, we propose a game for hand rehabilitation that leverages the therapeutic aspects of music for motor rehabilitation , incorporates the power of gamification to improve adherence to medical treatment, and uses the versatility of devices such as the Leap Motion Controller to track users' movements. The main characteristics of the game as well as future research directions are outlined.

Journal of Artificial General Intelligence, 2015

We investigate the relationship between lexical spaces and contextually-defined conceptual spaces... more We investigate the relationship between lexical spaces and contextually-defined conceptual spaces, offering applications to creative concept discovery. We define a computational method for discovering members of concepts based on semantic spaces: starting with a standard distributional model derived from corpus co-occurrence statistics, we dynamically select characteristic dimensions associated with seed terms, and thus a subspace of terms defining the related concept. This approach performs as well as, and in some cases better than, leading distributional semantic models on a WordNet-based concept discovery task, while also providing a model of concepts as convex regions within a space with interpretable dimensions. In particular, it performs well on more specific, contextualized concepts; to investigate this we therefore move beyond WordNet to a set of human empirical studies, in which we compare output against human responses on a membership task for novel concepts. Finally, a separate panel of judges rate both model output and human responses, showing similar ratings in many cases, and some commonalities and divergences which reveal interesting issues for computational concept discovery.

Proceedings of the Cognitive Science Society, 2015

The ability to extract meaningful relationships from sequences is crucial to many aspects of perc... more The ability to extract meaningful relationships from sequences is crucial to many aspects of perception and cognition, such as speech and music. This paper explores how leading computational techniques may be used to model how humans learn abstract musical relationships, namely, tonality and octave equivalence. Rather than hard-coding musical rules, this model uses an unsupervised learning approach to glean tonal relationships from a musical corpus. We develop and test a novel input representation technique, using a perceptually-inspired harmonics-based representation, to boot-strap the model's learning of tonal structure. The results are compared with behavioral data from listeners' performance on a standard music perception task: the model effectively encodes tonal relationships from musical data, simulating expert performance on the listening task. Lastly, the results are contrasted with previous findings from a computational model that uses a more simple symbolic input representation of pitch.

18th International Society for Music Information Retrieval Conference, 2017

Tonal structure is in part conveyed by statistical regularities between musical events, and resea... more Tonal structure is in part conveyed by statistical regularities between musical events, and research has shown that computational models reflect tonal structure in music by capturing these regularities in schematic constructs like pitch histograms. Of the few studies that model the acquisition of perceptual learning from musical data, most have employed self-organizing models that learn a topology of static descriptions of musical contexts. Also, the stimuli used to train these models are often symbolic rather than acoustically faithful representations of musical material. In this work we investigate whether sequential predictive models of musical memory (specifically, recurrent neural networks), trained on audio from commercial CD recordings , induce tonal knowledge in a similar manner to listeners (as shown in behavioral studies in music perception). Our experiments indicate that various types of recurrent neural networks produce musical expectations that clearly convey tonal structure. Furthermore, the results imply that although implicit knowledge of tonal structure is a necessary condition for accurate musical expectation, the most accurate predictive models also use other cues beyond the tonal structure of the musical context.

Proceedings of the Cognitive Science Society, 2009

Evidence suggests that sparse coding allows for a more efficient and effective way to distill str... more Evidence suggests that sparse coding allows for a more efficient and effective way to distill structural information about the environment. Our simple recurrent network has demonstrated the same to be true of learning musical structure. Two experiments are presented that examine the learning trajectory of a simple recurrent network exposed to musical input. Both experiments compare the network's internal representations to behavioral data: Listeners rate the network's own novel musical output from different points along the learning trajectory. The first study focused on learning the tonal relationships inherent in five simple melodies. The developmental trajectory of the network was studied by examining sparseness of the hidden layer activations and the sophistication of the network's compositions. The second study used more complex musical input and focused on both tonal and rhythmic relationships in music. We found that increasing sparseness of the hidden layer activations strongly correlated with the increasing sophistication of the network's output. Interestingly, sparseness was not programmed into the network; this property simply arose from learning the musical input. We argue that sparseness underlies the network's success: It is the mechanism through which musical characteristics are learned and distilled, and facilitates the network's ability to produce more complex and stylistic novel compositions over time.

Frontiers in Psychology: Auditory Cognitive Neuroscience, 2016

We present a novel hypothetical account of entrainment in music and language, in context of the I... more We present a novel hypothetical account of entrainment in music and language, in context of the Information Dynamics of Thinking model, IDyOT. The extended model affords an alternative view of entrainment, and its companion term, pulse, from earlier accounts. The model is based on hierarchical, statistical prediction, modeling expectations of both what an event will be and when it will happen. As such, it constitutes a kind of predictive coding, with a particular novel hypothetical implementation. Here, we focus on the model's mechanism for predicting when a perceptual event will happen, given an existing sequence of past events, which may be musical or linguistic. We propose a range of tests to validate or falsify the model, at various different levels of abstraction, and argue that computational modeling in general, and this model in particular, can offer a means of providing limited but useful evidence for evolutionary hypotheses.

Proceedings of the 5th International Workshop on Computational Creativity, Concept Invention, and General Intelligence, 2016

In this paper, we present a novel application of a computational model of word meaning to capture... more In this paper, we present a novel application of a computational model of word meaning to capture human judgments of the linguistic properties of metaphoricity, familiarity, and meaningfulness. We present data gathered from human subjects regarding their ratings of these properties over a set of word pairs specifically designed to exhibit varying degrees of metaphoricity. We then investigate whether these properties can be measured in terms of geometric features of a model of distributional lexical semantics. We compare the performance of two models, our own Concept Discovery Model which dynamically constructs context-sensitive subspaces, and a state-of-the-art static distributional semantic model, and find that our dynamic model performs significantly better in its measurement of metaphoricity.

BEHAVIORAL AND BRAIN SCIENCES, 2008

Response to 'Emotional responses to music: The need to consider underlying mechanisms' by Juslin ... more Response to 'Emotional responses to music: The need to consider underlying mechanisms' by Juslin and Vastfjall.
Abstract: When examining how emotions are evoked through music,
the role of musical expectancy is often surprisingly under-credited. This
mechanism, however, is most strongly tied to the actual structure of the
music, and thus is important when considering how music elicits
emotions. We briefly summarize Leonard Meyer’s theoretical
framework on musical expectancy and emotion and cite relevant
research in the area.

In this paper, we present a novel application of a computational model of word meaning to capture... more In this paper, we present a novel application of a computational model of word meaning to capture human judgments of the linguistic properties of metaphoricity, familiarity, and meaningfulness. We present data gathered from human subjects regarding their ratings of these properties over a set of word pairs specifically designed to exhibit varying degrees of metaphoricity. We then investigate whether these properties can be measured in terms of geometric features of a model of distributional lexical semantics. We compare the performance of two models, our own Concept Discovery Model which dynamically constructs context-sensitive subspaces, and a state-of-the-art static distributional semantic model, and find that our dynamic model performs significantly better in its measurement of metaphoricity.

Frontiers in Psychology: Cognitive Science, 2017

An empirical investigation of how local harmonic structures (e.g., chord progressions) contribute... more An empirical investigation of how local harmonic structures (e.g., chord progressions) contribute to the experience and enjoyment of uplifting trance (UT) music is presented. The connection between rhythmic and percussive elements and resulting trance-like states has been highlighted by musicologists, but no research, to our knowledge, has explored whether repeated harmonic elements influence affective responses in listeners of trance music. Two alternative hypotheses are discussed, the first highlighting the direct relationship between repetition/complexity and enjoyment, and the second based on the theoretical inverted-U relationship described by the Wundt curve. We investigate the connection between harmonic structure and subjective enjoyment through interdisciplinary behavioral and computational methods: First we discuss an experiment in which listeners provided enjoyment ratings for computer-generated UT anthems with varying levels of harmonic repetition and complexity. The anthems were generated using a statistical model trained on a corpus of 100 uplifting trance anthems created for this purpose, and harmonic structure was constrained by imposing particular repetition structures (semiotic patterns defining the order of chords in the sequence) on a professional UT music production template. Second, the relationship between harmonic structure and enjoyment is further explored using two computational approaches, one based on average Information Content, and another that measures average tonal tension between chords. The results of the listening experiment indicate that harmonic repetition does in fact contribute to the enjoyment of uplifting trance music. More compelling evidence was found for the second hypothesis discussed above, however some maximally repetitive structures were also preferred. Both computational models provide evidence for a Wundt-type relationship between complexity and enjoyment. By systematically manipulating the structure of chord progressions, we have discovered specific harmonic contexts in which repetitive or complex structure contribute to the enjoyment of uplifting trance music.

The field of computational creativity, including musical metacreation, strives to develop artific... more The field of computational creativity, including musical metacreation, strives to develop artificial systems that are capable of demonstrating creative behavior or producing creative artefacts. But the claim of creativity
is often assessed, subjectively only on the part of the researcher and not objectively at all. This article provides theoretical motivation for more systematic evaluation of musical metacreation and computationally creative systems and presents an overview of current methods used to assess human and machine creativity that may be adapted for this purpose. In order to highlight the need for a varied set of evaluation tools, a
distinction is drawn among three types of creative systems: those that are purely generative, those that contain internal or external feedback, and those that are capable of reflection and self-reflection. To address
the evaluation of each of these aspects, concrete examples of methods and techniques are suggested to help researchers (1) evaluate their systems’ creative process and generated artefacts, and test their impact
on the perceptual, cognitive, and affective states of the audience, and (2) build mechanisms for reflection into the creative system, including models of human perception and cognition, to endow creative systems
with internal evaluative mechanisms to drive self-reflective processes. The first type of evaluation can be considered external to the creative system and may be employed by the researcher to both better understand
the efficacy of their system and its impact and to incorporate feedback into the system. Here we take the stance that understanding human creativity can lend insight to computational approaches, and knowledge
of how humans perceive creative systems and their output can be incorporated into artificial agents as feedback to provide a sense of how a creation will impact the audience. The second type centers around
internal evaluation, in which the system is able to reason about its own behavior and generated output. We argue that creative behavior cannot occur without feedback and reflection by the creative/metacreative
system itself. More rigorous empirical testing will allow computational and metacreative systems to become more creative by definition and can be used to demonstrate the impact and novelty of particular approaches.

Statistical learning (SL) is the ability to generate predictions based on probabilistic dependenc... more Statistical learning (SL) is the ability to generate predictions based on probabilistic dependencies in the environment, an ability that is present throughout life. The effect of aging on SL is still unclear. Here, we explore statistical learning in healthy adults (40 younger and 40 older). The novel paradigm tracks learning trajectories and shows age-related differences in overall performance, yet similarities in learning rates. Bayesian models reveal further differences between younger and older adults in dealing with uncertainty in this probabilistic SL task. We test computational models of 3 different learning strategies: (a) Win-Stay, Lose-Shift, (b) Delta Rule Learning, (c) Information Weights to explore whether they capture age-related differences in performance and learning in the present task. A likely candidate mechanism emerges in the form of age-dependent differences in information weights, in which young adults more readily change their behavior, but also show disproportionally strong reactions toward erroneous predictions. With lower but more balanced information weights, older adults show slower behavioral adaptation but eventually arrive at more stable and accurate representations of the underlying transitional probability matrix.

Frontiers in Psychology, 2021

In recent years, the field of music therapy (MT) has increasingly embraced the use of technology ... more In recent years, the field of music therapy (MT) has increasingly embraced the use of technology for conducting therapy sessions and enhancing patient outcomes. Amidst a worldwide pandemic, we sought to examine whether this is now true to an even greater extent, as many music therapists have had to approach and conduct their work differently. The purpose of this survey study is to observe trends in how music therapists from different regions around the world have had to alter their practice, especially in relation to their use of technology during the COVID-19 pandemic, because of limited options to conduct in-person therapy due to social distancing measures. Further, the findings aim to clarify music therapists’ perspectives on the benefits and limitations of technology in MT, as well as online MT. In addition, this survey investigated what changes have been necessary to administer MT during COVID-19, in terms of virtual therapy and online tools, and how the changes made now may affect MT in the future. We also explored music therapists’ views on whether special technology-focused training might be helpful to support the practice of MT in the future. This is the first survey, to our knowledge, to break down opinions of and trends in technology use based on geographical region (North America, Europe, and Asia), and several noteworthy differences were apparent across regions. We hope our findings provide useful information, guidance, and a global reference point for music therapists on effectively continuing the practice of MT during times of crisis, and can encourage reflection and improvement in administering MT.

The fields of music, health, and technology have seen significant interactions in recent years in... more The fields of music, health, and technology have seen significant interactions in recent years in developing music technology for health care and well-being. In an effort to strengthen the collaboration between the involved disciplines, the workshop "Music, Computing, and Health" was held to discuss best practices and state-of-the-art at the intersection of these areas with researchers from music psychology and neuroscience, music therapy, music information retrieval, music technology, medical technology (medtech), and robotics. Following the discussions at the workshop, this article provides an overview of the different methods of the involved disciplines and their potential contributions to developing music technology for health and well-being. Furthermore, the article summarizes the state of the art in music technology that can be applied in various health scenarios and provides a perspective on challenges and opportunities for developing music technology that (1) supports person-centered care and evidence-based treatments, and (2) contributes to developing standardized, large-scale research on music-based interventions in an interdisciplinary manner. The article provides a resource for those seeking to engage in interdisciplinary research using music-based computational methods to develop technology for health care, and aims to inspire future research directions by evaluating the state of the art with respect to the challenges facing each field.

Psychology of Music, 2019

Research into vision has highlighted the importance of gist representations in change detection a... more Research into vision has highlighted the importance of gist representations in change detection and memory. This article puts forth the hypothesis that schematic processing and gist provide an account for change detection in music as well, where a musical gist is an abstracted memory representation for schematically consistent tones. The present experiments illuminate the content of gist memory representations by testing when listeners can detect single-tone changes in pairs of melodies. In Experiment 1, musicians and non-musicians listened to melodies varying in tonal structure. Less structure resulted in compromised change detection in both groups. Most often, musicians displayed more accurate change detection than non-musicians, but, surprisingly, when schematic processing could not contribute to memory encoding, musicians performed worse than their untrained counterparts. Experiment 2 utilized a full-factorial design to examine tonality, interval of pitch change, metrical position, and rhythm. Tonality had a particularly large effect on performance, with non-scale tones generally aiding change detection. Listeners were unlikely, however, to detect schematically inconsistent tones when only brief melodic context was available. The results uphold the hypothesis that memory for melodies relies on schematic processing, with change detection dependent upon whether the change alters the schematic gist of the melody.

PLOS ONE, 2019

Emotions play a critical role in rational and intelligent behavior; a better fundamental knowledg... more Emotions play a critical role in rational and intelligent behavior; a better fundamental knowledge of them is indispensable for understanding higher order brain function. We propose a non-invasive brain-computer interface (BCI) system to feedback a person's affective state such that a closed-loop interaction between the participant's brain responses and the musical stimuli is established. We realized this concept technically in a functional prototype of an algorithm that generates continuous and controllable patterns of synthesized affective music in real-time, which is embedded within a BCI architecture. We evaluated our concept in two separate studies. In the first study, we tested the efficacy of our music algorithm by measuring subjective affective responses from 11 participants. In a second pilot study, the algorithm was embedded in a real-time BCI architecture to investigate affective closed-loop interactions in 5 participants. Preliminary results suggested that participants were able to intentionally modulate the musical feedback by self-inducing emotions (e.g., by recalling memories), suggesting that the system was able not only to capture the listener's current affective state in real-time, but also potentially provide a tool for listeners to mediate their own emotions by interacting with music. The proposed concept offers a tool to study emotions in the loop, promising to cast a complementary light on emotion-related brain research, particularly in terms of clarifying the interactive, spatio-temporal dynamics underlying affec-tive processing in the brain.

Neural Computing and Applications, 2019

We explore the potential of a popular distributional semantics vector space model, word2vec, for ... more We explore the potential of a popular distributional semantics vector space model, word2vec, for capturing meaningful relationships in ecological (complex poly-phonic) music. More precisely, the skip-gram version of word2vec is used to model slices of music from a large corpus spanning eight musical genres. In this newly learned vector space, a metric based on cosine distance is able to distinguish between functional chord relationships, as well as harmonic associations in the music. Evidence, based on cosine distance between chord-pair vectors, suggests that an implicit circle-of-fifths exists in the vector space. In addition, a comparison between pieces in different keys reveals that key relationships are represented in word2vec space. These results suggest that the newly learned embedded vector representation does in fact capture tonal and harmonic characteristics of music, without receiving explicit information about the musical content of the constituent slices. In order to investigate whether proximity in the discovered space of embeddings is indicative of 'semantically-related' slices, we explore a music generation task, by automatically replacing existing slices from a given piece of music with new slices. We propose an algorithm to find substitute slices based on spatial proximity and the pitch class distribution inferred in the chosen subspace. The results indicate that the size of the subspace used has a significant effect on whether slices belonging to the same key are

Cognitive Science, 2018

A basic function of cognition is to detect regularities in sensory input to facilitate the predic... more A basic function of cognition is to detect regularities in sensory input to facilitate the prediction and recognition of future events. It has been proposed that these implicit expectations arise from an internal predictive coding model, based on knowledge acquired through processes such as statistical learning, but it is unclear how different types of statistical information affect listeners’ memory for auditory stimuli. We used a combination of behavioral and computational methods to investigate memory for non-linguistic auditory sequences. Participants repeatedly heard tone sequences varying systematically in their information-theoretic properties. Expectedness ratings of tones were collected during three listening sessions, and a recognition memory test was given after each session. Information-theoretic measures of sequential predictability significantly influenced listeners’ expectedness ratings, and variations in these properties had a significant impact on memory performance. Predictable sequences yielded increasingly better memory performance with increasing exposure. Computational simulations using a probabilistic model of auditory expectation suggest that listeners dynamically formed a new, and increasingly accurate, implicit cognitive model of the information-theoretic structure of the sequences throughout the experimental session.

ACM Computers in Entertainment: Sound and Music Computing, 2016

The field of computational creativity, including musical metacreation, strives to develop artific... more The field of computational creativity, including musical metacreation, strives to develop artificial systems that are capable of demonstrating creative behavior or producing creative artefacts. But the claim of creativity is often assessed, subjectively only on the part of the researcher and not objectively at all. This article provides theoretical motivation for more systematic evaluation of musical metacreation and computationally creative systems and presents an overview of current methods used to assess human and machine creativity that may be adapted for this purpose. In order to highlight the need for a varied set of evaluation tools, a distinction is drawn among three types of creative systems: those that are purely generative, those that contain internal or external feedback, and those that are capable of reflection and self-reflection. To address the evaluation of each of these aspects, concrete examples of methods and techniques are suggested to help researchers (1) evaluate their systems' creative process and generated artefacts, and test their impact on the perceptual, cognitive, and affective states of the audience, and (2) build mechanisms for reflection into the creative system, including models of human perception and cognition, to endow creative systems with internal evaluative mechanisms to drive self-reflective processes. The first type of evaluation can be considered external to the creative system and may be employed by the researcher to both better understand the efficacy of their system and its impact and to incorporate feedback into the system. Here we take the stance that understanding human creativity can lend insight to computational approaches, and knowledge of how humans perceive creative systems and their output can be incorporated into artificial agents as feedback to provide a sense of how a creation will impact the audience. The second type centers around internal evaluation, in which the system is able to reason about its own behavior and generated output. We argue that creative behavior cannot occur without feedback and reflection by the creative/metacreative system itself. More rigorous empirical testing will allow computational and metacreative systems to become more creative by definition and can be used to demonstrate the impact and novelty of particular approaches. ACM Reference Format: Kat Agres, Jamie Forth, and Geraint A. Wiggins. 2016. Evaluation of musical creativity and musical metacre-ation systems.

Proceedings of the IEEE International Conference on Orange Technologies, 2017

Traditional physical therapy methods require significant time from trained medical staff, which i... more Traditional physical therapy methods require significant time from trained medical staff, which is costly for clinics and hospitals, and often leave patients bored and unmotivated to complete their exercises. We offer a prototype for a motion-detection and music game to inspire greater engagement and adherence from patients undergoing physical therapy exercises for rehabilitation or strengthening. The game is customizable based on the patient's needs, dynamically reacts to the patient's performance in real-time, and may be used with or without the guidance of a medical professional.

19th International Society for Music Information Retrieval Conference, 2018

Millions of individuals around the world suffer from motor impairment or disability, yet effectiv... more Millions of individuals around the world suffer from motor impairment or disability, yet effective, engaging, and cost-effective therapeutic solutions are still lacking. In this work, we propose a game for hand rehabilitation that leverages the therapeutic aspects of music for motor rehabilitation , incorporates the power of gamification to improve adherence to medical treatment, and uses the versatility of devices such as the Leap Motion Controller to track users' movements. The main characteristics of the game as well as future research directions are outlined.

Journal of Artificial General Intelligence, 2015

We investigate the relationship between lexical spaces and contextually-defined conceptual spaces... more We investigate the relationship between lexical spaces and contextually-defined conceptual spaces, offering applications to creative concept discovery. We define a computational method for discovering members of concepts based on semantic spaces: starting with a standard distributional model derived from corpus co-occurrence statistics, we dynamically select characteristic dimensions associated with seed terms, and thus a subspace of terms defining the related concept. This approach performs as well as, and in some cases better than, leading distributional semantic models on a WordNet-based concept discovery task, while also providing a model of concepts as convex regions within a space with interpretable dimensions. In particular, it performs well on more specific, contextualized concepts; to investigate this we therefore move beyond WordNet to a set of human empirical studies, in which we compare output against human responses on a membership task for novel concepts. Finally, a separate panel of judges rate both model output and human responses, showing similar ratings in many cases, and some commonalities and divergences which reveal interesting issues for computational concept discovery.

Proceedings of the Cognitive Science Society, 2015

The ability to extract meaningful relationships from sequences is crucial to many aspects of perc... more The ability to extract meaningful relationships from sequences is crucial to many aspects of perception and cognition, such as speech and music. This paper explores how leading computational techniques may be used to model how humans learn abstract musical relationships, namely, tonality and octave equivalence. Rather than hard-coding musical rules, this model uses an unsupervised learning approach to glean tonal relationships from a musical corpus. We develop and test a novel input representation technique, using a perceptually-inspired harmonics-based representation, to boot-strap the model's learning of tonal structure. The results are compared with behavioral data from listeners' performance on a standard music perception task: the model effectively encodes tonal relationships from musical data, simulating expert performance on the listening task. Lastly, the results are contrasted with previous findings from a computational model that uses a more simple symbolic input representation of pitch.

18th International Society for Music Information Retrieval Conference, 2017

Tonal structure is in part conveyed by statistical regularities between musical events, and resea... more Tonal structure is in part conveyed by statistical regularities between musical events, and research has shown that computational models reflect tonal structure in music by capturing these regularities in schematic constructs like pitch histograms. Of the few studies that model the acquisition of perceptual learning from musical data, most have employed self-organizing models that learn a topology of static descriptions of musical contexts. Also, the stimuli used to train these models are often symbolic rather than acoustically faithful representations of musical material. In this work we investigate whether sequential predictive models of musical memory (specifically, recurrent neural networks), trained on audio from commercial CD recordings , induce tonal knowledge in a similar manner to listeners (as shown in behavioral studies in music perception). Our experiments indicate that various types of recurrent neural networks produce musical expectations that clearly convey tonal structure. Furthermore, the results imply that although implicit knowledge of tonal structure is a necessary condition for accurate musical expectation, the most accurate predictive models also use other cues beyond the tonal structure of the musical context.

Proceedings of the Cognitive Science Society, 2009

Evidence suggests that sparse coding allows for a more efficient and effective way to distill str... more Evidence suggests that sparse coding allows for a more efficient and effective way to distill structural information about the environment. Our simple recurrent network has demonstrated the same to be true of learning musical structure. Two experiments are presented that examine the learning trajectory of a simple recurrent network exposed to musical input. Both experiments compare the network's internal representations to behavioral data: Listeners rate the network's own novel musical output from different points along the learning trajectory. The first study focused on learning the tonal relationships inherent in five simple melodies. The developmental trajectory of the network was studied by examining sparseness of the hidden layer activations and the sophistication of the network's compositions. The second study used more complex musical input and focused on both tonal and rhythmic relationships in music. We found that increasing sparseness of the hidden layer activations strongly correlated with the increasing sophistication of the network's output. Interestingly, sparseness was not programmed into the network; this property simply arose from learning the musical input. We argue that sparseness underlies the network's success: It is the mechanism through which musical characteristics are learned and distilled, and facilitates the network's ability to produce more complex and stylistic novel compositions over time.

Frontiers in Psychology: Auditory Cognitive Neuroscience, 2016

We present a novel hypothetical account of entrainment in music and language, in context of the I... more We present a novel hypothetical account of entrainment in music and language, in context of the Information Dynamics of Thinking model, IDyOT. The extended model affords an alternative view of entrainment, and its companion term, pulse, from earlier accounts. The model is based on hierarchical, statistical prediction, modeling expectations of both what an event will be and when it will happen. As such, it constitutes a kind of predictive coding, with a particular novel hypothetical implementation. Here, we focus on the model's mechanism for predicting when a perceptual event will happen, given an existing sequence of past events, which may be musical or linguistic. We propose a range of tests to validate or falsify the model, at various different levels of abstraction, and argue that computational modeling in general, and this model in particular, can offer a means of providing limited but useful evidence for evolutionary hypotheses.

Proceedings of the 5th International Workshop on Computational Creativity, Concept Invention, and General Intelligence, 2016

In this paper, we present a novel application of a computational model of word meaning to capture... more In this paper, we present a novel application of a computational model of word meaning to capture human judgments of the linguistic properties of metaphoricity, familiarity, and meaningfulness. We present data gathered from human subjects regarding their ratings of these properties over a set of word pairs specifically designed to exhibit varying degrees of metaphoricity. We then investigate whether these properties can be measured in terms of geometric features of a model of distributional lexical semantics. We compare the performance of two models, our own Concept Discovery Model which dynamically constructs context-sensitive subspaces, and a state-of-the-art static distributional semantic model, and find that our dynamic model performs significantly better in its measurement of metaphoricity.

BEHAVIORAL AND BRAIN SCIENCES, 2008

Response to 'Emotional responses to music: The need to consider underlying mechanisms' by Juslin ... more Response to 'Emotional responses to music: The need to consider underlying mechanisms' by Juslin and Vastfjall.
Abstract: When examining how emotions are evoked through music,
the role of musical expectancy is often surprisingly under-credited. This
mechanism, however, is most strongly tied to the actual structure of the
music, and thus is important when considering how music elicits
emotions. We briefly summarize Leonard Meyer’s theoretical
framework on musical expectancy and emotion and cite relevant
research in the area.

In this paper, we present a novel application of a computational model of word meaning to capture... more In this paper, we present a novel application of a computational model of word meaning to capture human judgments of the linguistic properties of metaphoricity, familiarity, and meaningfulness. We present data gathered from human subjects regarding their ratings of these properties over a set of word pairs specifically designed to exhibit varying degrees of metaphoricity. We then investigate whether these properties can be measured in terms of geometric features of a model of distributional lexical semantics. We compare the performance of two models, our own Concept Discovery Model which dynamically constructs context-sensitive subspaces, and a state-of-the-art static distributional semantic model, and find that our dynamic model performs significantly better in its measurement of metaphoricity.

Frontiers in Psychology: Cognitive Science, 2017

An empirical investigation of how local harmonic structures (e.g., chord progressions) contribute... more An empirical investigation of how local harmonic structures (e.g., chord progressions) contribute to the experience and enjoyment of uplifting trance (UT) music is presented. The connection between rhythmic and percussive elements and resulting trance-like states has been highlighted by musicologists, but no research, to our knowledge, has explored whether repeated harmonic elements influence affective responses in listeners of trance music. Two alternative hypotheses are discussed, the first highlighting the direct relationship between repetition/complexity and enjoyment, and the second based on the theoretical inverted-U relationship described by the Wundt curve. We investigate the connection between harmonic structure and subjective enjoyment through interdisciplinary behavioral and computational methods: First we discuss an experiment in which listeners provided enjoyment ratings for computer-generated UT anthems with varying levels of harmonic repetition and complexity. The anthems were generated using a statistical model trained on a corpus of 100 uplifting trance anthems created for this purpose, and harmonic structure was constrained by imposing particular repetition structures (semiotic patterns defining the order of chords in the sequence) on a professional UT music production template. Second, the relationship between harmonic structure and enjoyment is further explored using two computational approaches, one based on average Information Content, and another that measures average tonal tension between chords. The results of the listening experiment indicate that harmonic repetition does in fact contribute to the enjoyment of uplifting trance music. More compelling evidence was found for the second hypothesis discussed above, however some maximally repetitive structures were also preferred. Both computational models provide evidence for a Wundt-type relationship between complexity and enjoyment. By systematically manipulating the structure of chord progressions, we have discovered specific harmonic contexts in which repetitive or complex structure contribute to the enjoyment of uplifting trance music.

The field of computational creativity, including musical metacreation, strives to develop artific... more The field of computational creativity, including musical metacreation, strives to develop artificial systems that are capable of demonstrating creative behavior or producing creative artefacts. But the claim of creativity
is often assessed, subjectively only on the part of the researcher and not objectively at all. This article provides theoretical motivation for more systematic evaluation of musical metacreation and computationally creative systems and presents an overview of current methods used to assess human and machine creativity that may be adapted for this purpose. In order to highlight the need for a varied set of evaluation tools, a
distinction is drawn among three types of creative systems: those that are purely generative, those that contain internal or external feedback, and those that are capable of reflection and self-reflection. To address
the evaluation of each of these aspects, concrete examples of methods and techniques are suggested to help researchers (1) evaluate their systems’ creative process and generated artefacts, and test their impact
on the perceptual, cognitive, and affective states of the audience, and (2) build mechanisms for reflection into the creative system, including models of human perception and cognition, to endow creative systems
with internal evaluative mechanisms to drive self-reflective processes. The first type of evaluation can be considered external to the creative system and may be employed by the researcher to both better understand
the efficacy of their system and its impact and to incorporate feedback into the system. Here we take the stance that understanding human creativity can lend insight to computational approaches, and knowledge
of how humans perceive creative systems and their output can be incorporated into artificial agents as feedback to provide a sense of how a creation will impact the audience. The second type centers around
internal evaluation, in which the system is able to reason about its own behavior and generated output. We argue that creative behavior cannot occur without feedback and reflection by the creative/metacreative
system itself. More rigorous empirical testing will allow computational and metacreative systems to become more creative by definition and can be used to demonstrate the impact and novelty of particular approaches.