The Recognition of Melodies by Humans and by Machine (original) (raw)

Development of a methodology and theoretical framework for melodic discrimination

1987

The psychology of music has the potential to make great contributions to knowledge of cognition in general because musical stimuli come from a natural and highly structured domain, (as do the linguistic stimuli normally used in cognitive research), and yet they lack the referential meaning of linguistic stimuli. This lack of reference by musical stimuli to entities outside the musical domain should make it easier to elucidate the cognitive mechanisms by which structured and patterned stimuli are manipulated than is the case with linguistic stimuli. Yet there is not a single result from the psychology of music that is generally regarded as illuminating basic cognitive mechanisms. The ability to discriminate two melodies as differing or identical is arguably the most basic musical task. I have concentrated on that task in this research. Modern research on melody discrimination dates from 1960 and yet the literature reporting that research is quite small with only a limited number of investigators. The effects and theoretical models reported have been specific to the musical domain with almost no appeal to any general cognitive mechanisms. I believe that because of this the field is seen as peripheral to the advance of cognition, very few researchers are attracted to it, and hence the rate of progress is low. In order to unlock the potential of the psychology of music this area of research must be tied back to general cognition. In my review of previous research I have identified the lack of a sophisticated theoretical framework and adequate experimental and data analytic methodologies as the key factors preventing rapid progress in this field. Therefore, the aim of my research has been threefold: to develop an adequate theoretical framework that will serve as a bridge from musical phenomena to concepts from general cognition; to develop sophisticated methods of data analysis that allow much more fine-grained analyses of data than are currently possible in order to investigate the proposed framework; and to conduct empirical investigations in order to explore the consequences of the framework, test the new data analyses, and to discover new, more subtle effects than those previously reported, to serve as the starting points for further investigation. The framework that I have proposed is based on the distinction between attributes describing a melody and the units in which those attributes are organised. This distinction is commonly acknowledged in models of cognitive representations and yet it has not been made at all in previous melody discrimination research. That research appears to have identified the attribute with the units, as exemplified by the assertion that "well-known melodies must be stored as sequences of pitch intervals between successive notes" (Dowling, 1978, p. 346). My framework makes it clear that the issue of what information is stored (attributes) can be investigated independently of the issue of how that information is organised in the representation (units). Another point which is made clear by my framework is the need for some mechanism to ensure that corresponding units are compared in the process of comparing the representations of two melodies. In my fourth experiment I discovered a pair of melodies in which non-corresponding units were compared. This problem of matching corresponding units for comparison is likely to exist in all cognitive domains. Thus this is an area in which the psychology of music could elucidate a general cognitive mechanism. In my review of the methodology used in melody discrimination research I found that there were characteristics of the melodic domain that made traditional means of analysis particularly inappropriate. There are marked differences in the psychological characteristics of individual melodies and there are also idiosyncratic interactions of subjects with melodies. Another previously unrecognised problem is that it is literally impossible to generate musical stimuli in such a way as to avoid confounding. I refer to this characteristic as obligatory confounding. These characteristics reduce the utility of traditional analyses which aggregate data over melodies and analyse them in terms of fixed orthogonal factors. However, the major problem with previous analyses is that they are carried out at the level of groups of melody pairs rather than at the single pair level. That is, measures of discriminability could not be derived for single pairs of melodies. Thus the discriminabilities could only be analysed and explained in terms of the summary properties of groups of melody pairs rather than the properties of individual melodies involved in individual judgments. I have developed an extension to signal detection analysis that allows the discriminability of individual melody pairs to be calculated. These discriminabilities can then be analysed and interpreted in terms of the characteristics of the particular melodies involved. Thus, I have developed a technique to allow melody discrimination phenomena to be examined at a far finer level of resolution than was previously possible. Using this technique I have demonstrated several previously unknown effects and the number of effects found appears to have been limited primarily by the quantity of data collected. In my first experiment I was attempting to find clues to the existence of previously unreported effects for further investigation in the subsequent experiments. The effect which I found and chose to pursue was the increased saliency of changes occurring at a contour reversal. This experiment also demonstrated effects related to the overall contour shape, a particular pitch transition, the pitch range, and cadence. The second and third experiments further investigated the contour reversal effect. This effect was convincingly demonstrated in these experiments while I developed my methods of data analysis. The third experiment also demonstrated effects associated with the repetition of a note across a contour reversal and with rhythmic stress. In my fourth experiment I used the methodology that I had developed to attack directly the question of the relative importance of interval-based and note-based attributes. This experiment demonstrated that simple measures of the number of notes and interval changed are inadequate to account for melody discrimination performance. I found effects related to repetition across a contour reversal, the tonic, the major triad, the unusuality of particular pitch transitions, and the process of matching of corresponding units. These effects were more naturally described in terms of note-based attributes than interval-based attributes. Thus, the research reported here has accomplished my original aims of developing a framework, developing a methodology, and demonstrating novel effects with that framework and methodology. However, the most important aspect of this work is that it provides a significant break from previous work and acts as a starting point from which to develop the area of melody discrimination research in the manner which it deserves.

A figural approach to the role of melodic contour in melody recognition

Attention, Perception, & Psychophysics, 1984

These experiments looked at specific components of melodic contour, reversals in pitch, and nonreversals, using a short-term recognition memory paradigm. Listeners found it easier to discriminate between same and different melody pairs if changes occurred at contour reversals. This result was independent of rate of presentation, suggesting that the salience of upper and lower reversals is not attributable to a form of perceptual streaming. These results suggest an analogy between auditory and visual contours. If melodic contour is represented as a series of pitches extending over time, then reversals can be interpreted as "comers" and nonreversals &s "slopes." The results are consistent with a global strategy of perceptual analysis whereby comers are more salient because they "define the figure." An alternative explanation is that the melodies are processed in a way that is similar to the processing of the intonation pattern of speech. These contour feature effects were drastically reduced when the second melody was transposed and/or following a familiarization procedure. This last result is attributed to a more detailed perceptual analysis being performed under these conditions, a process that is less influenced by contour. Therefore,while specificcontour features, reversals in pitch, playa part in the recognition of untransposed novel melodies, these features are not prevalent in the recognition of transposed and more familiar sequences.

Steps and leaps in human memory for melodies: The ef- fect of pitch interval magnitude in a melodic contour dis- crimination task

2006

Melodic contour, the shape of a melody without reference to the individual notes, is important in the cognition of short, unfamiliar melodies. Melodic contour has generally been conceived as specifically the direction of pitch intervals (i.e., up or down). It is argued that the magnitude of the pitch intervals (i.e., step or leap) is also encoded as part of relative pitch memory for melodies. It was hypothesised that participants are sensitive to changes in pitch interval magnitude (PIM) in a discrimination task using transposed melodies. It was also hypothesized that PIM is encoded in relative pitch memory for melodies as part of melodic contour. The results indicate that participants are sensitive to changes in pitch interval magnitude, but does not support the hypothesis that PIM is encoded as part of melodic contour. Instead, the results of the experiment suggest that the effect of on contour discrimination appears to arise from melodic expectations..

Melodic contour similarity using folk melodies

2008

Music Perception vo lu m e 28, issue 2, pp. 169-193, issn 0730-7829, electronic issn 1533-8312 © 2010 by t h e regents o f t h e university o f california. all rights reserved. please d i r e c t all requests f o r permission to p h oto c o p y o r r e p ro d u c e a rt i c l e c o n t e n t t h ro u g h t h e university o f california press's rights a n d permissions w e b s i t e , h t t p ://www.ucpressjournals.com/reprintinfo.asp.

Music Perception melodic contour similarity using folk melodies

Music Perception, 2010

Music Perception vo lu m e 28, issue 2, pp. 169-193, issn 0730-7829, electronic issn 1533-8312 © 2010 by t h e regents o f t h e university o f california. all rights reserved. please d i r e c t all requests f o r permission to p h oto c o p y o r r e p ro d u c e a rt i c l e c o n t e n t t h ro u g h t h e university o f california press's rights a n d permissions w e b s i t e , h t t p ://www.ucpressjournals.com/reprintinfo.asp.

Contour-Based Melody Representation: An Analytical Study

In this paper, we identify parameters crucial to the performance of a Query By Humming (QBH) system, and present an analytical approach to de- termining optimal values of such parameters. Ex- isting systems use heuristically chosen parameters - our analytical results are in accordance with such values. We present results of experimentation with simulated data, as well as an actual melody database of a QBH system.

The effect of implied harmony, contour and musical expertise on judgments of similarity of familiar melodies

Journal of New Music Research, 2006

According to Western music theory, familiar melodies containing alterations which shift them a long distance (harmonically) from the original should be considered perceptually dissimilar relative to the original. It is possible to obtain such a large shift with a small change in melodic pitch. However, this creates a paradox with pitch (contour) shifting, which is known to reduce similarity minimally if the distance from the original is small. We investigated the hypotheses that (1) manipulations to contour and implied harmony of an original melody reduce similarity scores and (2) novices are less sensitive to implied harmony changes than experts, but as sensitive as experts to contour manipulations. Twenty-eight novices and 44 expert musicians rated similarity of familiar nursery rhyme tunes compared with close and distant harmonic transformations plus close and distant pitch shifts. Results indicated that both groups use contour (pitch distance) to determine similarity, but that musically experienced listeners also use implied harmony to make further distinctions. It is argued that as listeners become more experienced, they rely on more sophisticated strategies for encoding and organizing melodies in memory, with deeper structural aspects of music being used when other strategies such as contour are uninformative or non-distinctive. These new findings, that contour and harmony affect judged similarity for simple, familiar melodies, have implications for theories of memory for music, and for the design of automated music retrieval and data mining systems.

Measuring melodic similarity: Human vs. algorithmic judgments

Proceedings of the Conference on …, 2004

Background in first subdiscipline (music psychology): Melodic similarity is a key concept in many of musicology's subdiciplines, and the variety of algorithms for measuring melodic similarity is abundant (for an overview see Vol. 11 of Computing in Musicology and Vol. 18, Nr. 3 of Music Perception). But though some melodic similarity measures have been subjected to psychological testing (e.g. Schmuckler 1999;, until today there has been no comparative study which takes into account a wide variety of algorithms, and that could inform about which of these measures are cognitively most adequate and which are not.

8 Cognitive Adequacy in the Measurement of Melodic Similarity: Algorithmic vs. Human Judgments

Melodic similarity is a central concept in many sub-disciplines of musicology, as well as for many computer based applications that deal with the classifications and retrieval of melodic material. This paper describes a research paradigm for finding an 'optimal' similarity measure out of a multitude of different approaches and algorithmic variants. The repertory used in this study are short melodies from popular (pop) songs and the empirical data for validation stem from two extensive listener experiments with expert listeners (musicology students). The different approaches to melodic similarity measurement are first discussed and mathematically systematized. Detailed description of the listener experiments are given and the results are discussed. Strengths and weaknesses of the several tested similarity measures are outlined and an 'optimal' similarity measure for this specific melodic repertory is proposed.