Score following: An artificially intelligent musical accompanist (original) (raw)

Artificially intelligent accompaniment using Hidden Markov Models to model musical structure

MUSICAL STRUCTURE, 2008

For performing musicians, musical accompanists may not always be available during practice, or an available accompanist may not have the technical ability necessary. As a solution to this problem, many musicians practise with pre-recorded accompaniment. Such an accompaniment is fixed and does not interact with the musician’s playing: the musician must adapt their performance to match the recording. To synchronise accompaniment with the soloist, it is preferable that an accompanist should be able to follow the musician through the score as they play, rather than the other way around. During performance, musicians may deviate from what is written in the score (either intentionally, by adding their own musical interpretation, or accidentally, by making performance errors). The accompanist should adjust their playing to follow the soloist. This work investigates how an artificial musician can follow a human musician through the performance of a piece (perform score following) using a Hidden Markov Model of the piece’s musical structure. The computer musician is designed to interact with the human musician and provide accompaniment as a human accompanist would: musically and in real time. Having successfully implemented this representation, the performances of the resulting artificial accompanists has been evaluated both qualitatively, by human testers and quantitatively, by objective criteria based on that used at the Music Infomation Retrieval and EXchange Conference in 2006. The artificial accompanists can, in general, accompany human performers with a reasonable degree of accuracy. Testing has also raised an interesting reflection on the nature of co-operation between soloist and accompanist, and more generally on the role of the computer musician in ensemble performance.

Computer Accompaniment and Music Understanding

Music Understanding is the recognition or identification of structure and pattern in musical information. Five music understanding projects are discussed. In the first, Computer Accompaniment of Melodic Instruments, the goal is for a computer system to listen to and play along with a live performer, using a predetermined score. In the second project, the technique is extended to handle polyphonic keyboard input so that a computer can accompany a pianist. In the third project, accompaniment is extended to a situation, a blues improvisation, where the score is not known in advance, so an understanding of deeper musical structures is required. The fourth project, Beat Tracking, attempts to identify musical beats in a live performance without a score. The tempo and a transcription of the performance are by-products of Beat Tracking. The final project is the Piano Tutor, a computer-based intelligent teaching system.

Investigating the role of score following in automatic musical accompaniment

Journal of New Music Research, 2009

When suitable accompanists are not available to a soloist musician, an alternative possibility is to use computer-generated accompaniment. A computer accompanist should interact with the soloist and adapt to the soloist’s playing as a human accompanist would, both reacting to expressive nuances of tempo and to unintentional errors such as wrong or mistimed notes. Over the past 25 years, accompaniment systems have been developed, all of which employ some form of score following: the process of following a musician’s progress through the score of a piece during performance. This work considers the role of score following in automatic accompaniment. In this investigation we developed a computer accompanist that employs score following. Our computer musician uses Hidden Markov Models to model the score by metrical structure and to provide accompaniment to a soloist playing monophonic music in real time, as the soloist is playing. Working with MIDI input/output, it tracks tempo ﬂuctuations, anticipates the soloist’s next note and supports some amount of unintentional deviation from the score. Qualitative evaluation, by human testers, and quantitative evaluation, using measurable criteria taken from MIREX, reported that the system performs adequately. We then used interviews with eight human accompanists to consider how well a score following system models the accompaniment process. This evaluation raises questions about the musical interaction between soloist and accompanist that have received relatively little attention. The information we gathered from interviews suggests the importance of other aspects of accompaniment, such as the sharing of shape of the performance between musicians, rather than treating the accompanist as purely subservient. We discuss the implications of these issues for the design of automated accompanists.

Computer Accompaniment and Music Understanding 1

2009

Automatic music accompaniment allowing errors and arbitrary repeats and jumps

2014

Automatic music accompaniment is particularly useful for exercises, rehearsals and personal enjoyment of ensemble music and one-hand piano performances. As musicians may make errors and want to correct them, or they may want to skip hard parts in the score, the system should allow errors as well as arbitrary repeats and skips. Detecting such repeats/skips, however, involves a large complexity of search for a player's score position in the entire score for every input event. Several efficient algorithms have been developed to cope with this problem under practical assumptions used in an online automatic accompaniment system named "Eurydice". In Eurydice for MIDI instruments, music performance is modeled by a hidden Markov model and maximum probability estimation is applied to the polyphonic MIDI input to yield an accompanying MIDI output (e.g., orchestra sound). Another version of Eurydice accepts monaural audio signal input and accompanies to it. Other issues such as treating ornaments, tempo estimation, and accompaniment algorithms are also discussed.

Automated Accompaniment of Musical Ensembles

This paper describes a computer accompaniment system capable of providing musical accompaniment for an ensemble of performers. The system tracks the performance of each musician in the ensemble to determine current score location and tempo of the ensemble. "Missing parts" in the composition (i.e., the accompaniment) are synthesized and synchronized to the ensemble. The paper presents an overview of the component problems of automated musical accompaniment and discusses solutions and their implementation. The system has been tested with solo performers as well as ensembles having as many as three performers. ducing an accompaniment in synchrony with the detected performance. A solution for each subproblem and a method for its implementation is also provided.

A Reference Architecture and Score Representation for Popular Music Human-Computer Music Performance Systems

2011

Popular music (characterized by improvised instrumental parts, beat and measure-level organization, and steady tempo) poses challenges for human-computer music performance (HCMP). Pieces of music are typically rearrangeable on-the-fly and involve a high degree of variation from ensemble to ensemble, and even between rehearsal and performance. Computer systems aiming to participate in such ensembles must therefore cope with a dynamic high-level structure in addition to the more traditional problems of beat-tracking, score-following, and machine improvisation. There are many approaches to integrating the components required to implement dynamic human-computer music performance systems. This paper presents a reference architecture designed to allow the typical sub-components (e.g. beat-tracking, tempo prediction, improvisation) to be integrated in a consistent way, allowing them to be combined and/or compared systematically. In addition, the paper presents a dynamic score representatio...

Implicit Learning of Musical Performance Parameters

This paper describes our attempt to make the Hidden Markov Model (HMM) score following system developed at IRCAM sensible to past experiences in order to adapt itself to a certain style of performance of musicians on a particular piece. We focus mostly on the aspects of the implemented machine learning technic pertaining to the style of performance of the score follower. To this end, a new observation modeling based on Gaussian Mixture Models is developed which is trainable using a novel learning algorithm we would call automatic discriminative training. The novelty of this system lies in the fact that this method, unlike classical methods for HMM training, is not concerned with modeling the music signal but with correctly choosing the sequence of music events that was performed.

Computers Composing Music: An Artistic Utilization of Hidden Markov Models for Music Composition

Journal of Undergraduate Research, 2005

Natural systems are the source of inspiration for the human tendency to pursue creative endeavors. Music composition is a language for human expression and can therefore be utilized in conveying the expressive capabilities of other systems. Using a Hidden Markov Model (HMM) learning system, a computer can be taught to create music that is coherent and aesthetically sufficient given the correct tools. The tools selected for this project include: twenty-two years of sun spot data as the natural system from which to creatively draw; a compositional framework for structure, pitch, dynamics, and rhythm to facilitate a human understanding of the systems expressiveness; the jMusic1, open source, music composition software; and an HMM learning system2 with implementations of the Forward-Backward, Viterbi, and Baum-Welch algorithms. In composing a final piece of music the attempt was made to impose as few creative restrictions on the system as possible. Through these tools every aspect of the compositions generation can be repeated. In this way the robust analytical capabilities of the system are displayed via the piece and its generative procedures, thereby displaying an artificial intelligences potential for music composition and perhaps larger creative projects.

Score Following: State of the Art and New Developments

2003

Score following is the synchronisation of a computer with a performer playing a known musical score. It now has a history of about twenty years as a research and musical topic, and is an ongoing project at Ircam. We present an overview of existing and historical score following systems, followed by fundamental definitions and terminology, and considerations about score formats, evaluation of score followers, and training. The score follower that we developed at Ircam is based on a Hidden Markov Model and on the modeling of the expected signal received from the performer. The model has been implemented in an audio and a Midi version, and is now being used in production. We report here our first experiences and our first steps towards a complete evaluation of system performances. Finally, we indicate directions how score following can go beyond the artistic applications known today.

Score following: An artificially intelligent musical accompanist (original) (raw)

Related papers