Quantitative, Notional, and Comprehensive Evaluations of Spontaneous Engaged Speech (original) (raw)
Related papers
A MIXED-METHODS STUDY EXPLORING PERCEPTIONS OF SPEECH FLUENCY
ARTICLES Enhancing English language learners' speech fluency is often a key learning outcome in communicative language classrooms. Notably, how fluent a learner's speech is has been shown to affect how comprehensible it is (Derwing, Rossiter, Munro, & Thomson, 2004). For this reason, it is not surprising that fluency has long been an integral component of both high-stakes and low-stakes oral proficiency assessment rubrics (Fulcher, 2003). Decisions that are made based on the results of these assessments may have real-world implications on test-takers' lives. Thus, it is important to understand which features of speech influence how fluency is perceived in order to enhance the validity of fluency assessments. In this study, although the participants reported that a wide range of temporal, non-temporal, and even non-linguistic features of speech influenced how they perceive fluency, it would seem as though, the speed of speech and the percentage of time speaking most strongly influenced how they assessed it. But what exactly is fluency? According to Lennon (1990), fluency is referred to in both broad and narrow terms. In the broadest sense of the term, fluency equates to overall language proficiency as in "I can speak three languages fluently!" However, in the realm of second language instruction and assessment, fluency is often defined much more narrowly as the overall speed and flow of speech. Yet, even within this narrow realm, definitions seem to vary widely, which can be problematic for assessors. Chambers (1997) highlights this problem by stating that "it cannot be assumed that we all share the same definition of fluency. Otherwise the validity of the judgements made by assessors is seriously in question. (p. 543)" Much research on fluency has involved investigating temporal variables of speech in terms of speed, pauses, and repairs. Since the 1970s, second language researchers have examined a wide variety of temporal measures including speech rate (number of syllables/duration, including silent pauses), articulation rate (number of syllables/duration, excluding silent pauses), mean length of runs (average number of syllables/utterance), the number, length, and location of silent and filled pauses, and the number and type of repairs. In the early 1990s, Lennon (1990), Riggenbach (1991), and Freed (1995) began correlating these temporal measures with overall impressions of fluency, as assessed on Likert scales. Their results revealed that, depending on the context, certain temporal measures of fluency seem to exert a degree of influence over raters' judgements of speech fluency. On the whole, these results indicate that increased speed and less hesitancy may lead to higher fluency
Fluency in Native and Nonnative English Speech
This book takes a new and holistic approach to fluency in English speech and differentiates between productive, perceptive, and nonverbal fluency. The in-depth corpus-based description of productive fluency points out major differences of how fluency is established in native and nonnative speech. It also reveals areas in which even highly advanced learners of English still deviate strongly from the native target norm and in which they have already approximated to it. Based on these findings, selected learners are subjected to native speakers' ratings of seven perceptive fluency variables in order to test which variables are most responsible for a perception of oral proficiency on the sides of the listeners. Finally, language-pedagogical implications derived from these findings for the improvement of fluency in learner language are presented. This book is conceptually and methodologically relevant for corpus-linguistics, learner corpus research and foreign language teaching and learning.
Aspects of Fluency Across Assessed Levels of Speaking Proficiency
The Modern Language Journal, 2020
Recent research in second language acquisition suggests that a number of speed, breakdown, repair and composite measures reliably assess fluency and predict proficiency. However, there is little research evidence to indicate which measures best characterize fluency at each assessed level of proficiency, and which can consistently distinguish one level from the next. This study investigated fluency in 32 speakers' performing four tasks of the British Council's Aptis Speaking test, which were awarded four different levels of proficiency (CEFR A2-C1). Using PRAAT, the performances were analysed for various aspects of utterance fluency across different levels of proficiency. The results suggest that speed and composite measures consistently distinguish fluency from the lowest to upper-intermediate levels (A2-B2), and many breakdown measures differentiate between the lowest level (A2) and the rest of the proficiency groups, with a few differentiating between lower (A2, B1) and higher levels (B2, C1). The varied use of repair measures at different levels suggest that a more complex process is at play. The findings imply that a detailed micro-analysis of fluency offers a more reliable understanding of the construct and its relationship with assessment of proficiency.
This chapter reviews Götz's book, Fluency in Native and Nonnative English Speech, which models speech fluency using quantitative corpus linguistic methodologies. Speech fluency has long been a key area of concern in English language teaching (ELT). For decades linguists and ELT experts have been searching for ways to enhance learners' second language (L2) speech fluency. Some key topics that have been explored so far include the characteristics and assessment of fluent speech, cognitive and psycholinguistic processes underlying fluent speech production and possible interventions that may improve learners' fluency (e.g. pre-task planning time, familiarity with communicative task-specific vocabulary items, short-term residence abroad). When it comes to an examination of factors that contribute to the perception of speech fluency, previous studies tend to address one or two factors at a time. Götz's Fluency in Native and Nonnative English Speech, however, demonstrates that it is possible to explore the effects of, and interactions between, multiple fluency-contributing factors in a single study using quantitative corpus linguistic methodologies. This is a new development for speech fluency research. The book presents a two-part empirical study in eight chapters. Chapter 1 introduces the rationale, aims and scope of the book and outlines a model of speech fluency which forms the backbone of the empirical study. The model consists of three components, each of which is reviewed separately in a following chapter. Chapter 2 begins with the first component, productive fluency, which comprises temporal variables (e.g. speech rate, mean length of runs, pauses), formulaic
2017
Second language oral fluency has long been considered as an important construct in communicative language ability (e.g. de Jong et al, 2012) and many speaking tests are designed to measure fluency aspect(s) of candidates’ language (e.g. IELTS, TOEFL iBT, PTE Academic). Current research in second language acquisition suggests that a number of measures of speed, breakdown and repair fluency can reliably assess fluency and predict proficiency. However, there is little research evidence to indicate which measures best characterise fluency at each level of proficiency, and which can consistently distinguish one proficiency level from the next. The study reported in this report is an attempt to help answer these questions. This study investigated fluency constructs across four different levels of proficiency (A2-C1) and four different semi-direct speaking test tasks performed by 32 candidates taking the Aptis Speaking test. Using PRAAT (Boersma & Weenik, 2013), we analysed 120 task perfor...
Automatic Assessment Of Second Language Learners' Fluency
This paper describes an experiment aimed at determining whether speech. automatic assessment of second language learners' fluency in This experiment produced interesting results in two respects, spontaneous speech is feasible and whether it differs from a. fluency assessment by expert raters and b. the relationship automatic fluency assessment in read speech. Spontaneous speech between expert fluency ratings and automatically obtained objective of 60 learners of Dutch was scored for fluency by five raters and fluency measures. With regard to a., the results showed that expert was analyzed by means of a continuous speech recognizer to ratings of fluency in read speech are reliable (Cronbachs' varies calculate seven quantitative measures of speech quality known to be between .90 and .96). With respect to b., very high correlations related to perceived fluency. The results show that automatic were found between the expert fluency ratings and the automatic assessment of second language learners' fluency in spontaneous measures of fluency: five automatic measures showed correlations speech is feasible, although not all variables suitable for measuring with the fluency scores whose magnitude varied between .77 and fluency in read speech are as effective in spontaneous speech. In .91. The highest correlations were found for rate of speech (between particular, measures that express the rate at which sounds are .86 and .91). Further analyses revealed that two factors are produced without taking pauses into account appear to be important for perceived fluency in read speech: the rate at which unsuitable for measuring fluency in spontaneous speech.
Automatic large-scale oral language proficiency assessment
2007
We describe first results obtained during the development of an automatic system for the assessment of spoken English proficiency of university students. The ultimate aim of this system is to allow fast, consistent and objective assessment of oral proficiency for the purpose of placing students in courses appropriate to their language skills. Rate of speech (ROS) was chosen as an indicator of fluency for a number of oral language exercises. In a test involving 106 student subjects, the assessments of 5 human raters are compared with evaluations based on automatically-derived ROS scores. It is found that, although the ROS is estimated accurately, the correlation between human assessments and the ROS scores varies between 0.5 and 0.6. However, the results also indicate that only two of the five human raters were consistent in their appraisals, and that there was only mild inter-rater agreement.
Assessing Oral Proficiency: A Comparison of Holistic and Objective Measures
The Modern Language Journal, 1995
This study examines the relationship between holistic judgments of oral proficiency and objective measures of syntactic maturity in the Oral Proficiency Interviews (OPIs) of 107 students of English as a foreign language (EFL) in Tianjin, China. The analysis compares a holistic rating of proficiency level (Intermediate, Advanced, and Superior) with objective measures (Mean T-Unit Length, Mean Error-Free T-Unit Length, and Percent of Error-Free T-Units) for three separate interview tasks (Describing/Narrating, Role Playing, and Asking Questions). Results of the repeated measures Multivariate Analyses of Variance (MAN-OVAs) indicate significant main effects for proficiency level and interview task.