Marianne Gullberg | Lund University (original) (raw)

Papers by Marianne Gullberg

Research paper thumbnail of Can sign-naïve adults learn about the phonological regularities of an unfamiliar sign language from minimal exposure

Language Development Research, 2024

Adults can extract phonological regularities from just several minutes' exposure to naturalistic ... more Adults can extract phonological regularities from just several minutes' exposure to naturalistic input of an unknown spoken language (Gullberg et al., 2010). We examined whether such implicit statistical learning mechanisms also operate in the sign language modality. The input materials consisted of a continuous sign stream in the form of a weather forecast in Svenskt Teckenspråk (STS). L1speakers of English with no prior knowledge of a sign language were assigned to two experimental groups who watched the forecast once (N=43) or twice (N=38), and a control group who did not watch it (N=40). Participants completed a 'surprise' lexical decision task designed to tap into their awareness of the phonological properties of the core STS lexicon. They viewed individual signs and indicated whether or not these could be real STS signs. The signs comprised four sets: STS signs that (1) were presented, and (2) were not presented, in the forecast; and signs that are not STS signs and (3) contain handshapes outside the STS handshape inventory, and (4) contain sets of phonological features that are dispreferred across sign languages. We found no evidence of any learning of STS phonological regularities. Considered in conjunction with two companion studies which did demonstrate some learning of sign forms and their meanings from these same input materials, our findings suggest limits to what can be learnt after just a few minutes of implicit and naturalistic exposure to language in an unfamiliar modality: information about specific lexical items is learnable, but information that requires generalisation across items may require greater amounts, or a different quality, of input.

Research paper thumbnail of Behavioral science labs: How to solve the multi-user problem

Behavior Research Methods, 2024

When lab resources are shared among multiple research projects, issues such as experimental integ... more When lab resources are shared among multiple research projects, issues such as experimental integrity, replicability, and data safety become important. Different research projects often need different software and settings that may well conflict with one another, and data collected for one project may not be safeguarded from exposure to researchers from other projects. In this paper we provide an infrastructure design and an open-source tool, labManager, that render multiuser lab facilities in the behavioral sciences accessible to research projects with widely varying needs. The solutions proposed ensure ease of management while simultaneously offering maximum flexibility by providing research projects with fully separated bare metal environments. This solution also ensures that collected data is kept separate, and compliant with relevant ethical standards and regulations such as General Data Protection Regulation (GDPR) legislation. Furthermore, we discuss preconditions for running shared lab facilities and provide practical advice.

Research paper thumbnail of Frontiers in Psychology: Visual Language

Frontiers in Psychology, 2019

Research paper thumbnail of Providing evidence for a well-worn stereotype: Italians and Swedes do gesture differently

Frontiers in Communication, 2024

Across cultures and languages spontaneous speech is often accompanied by gestures. It is a popula... more Across cultures and languages spontaneous speech is often accompanied by gestures. It is a popular belief that people in Italy gesture more than people in Northern Europe, such as in Sweden. Despite this general assumption few studies empirically investigate cultural differences in gesture frequency and gesture function under similar circumstances. This study compares the spoken and gestural behaviours of Italian and Swedish speakers, assumed to represent gesture-rich vs. gesture-sparse cultures. We examine the groups' gestural behaviour for frequency, and in terms of possible differences in rhetorical style probing the distribution of gestural functions (referential vs. pragmatic) across narrative levels (narrative, metanarrative, and paranarrative). The results show that (1) Italians overall do gesture more than Swedes; (2) Italians produce more pragmatic gestures than Swedes who produce more referential gestures; (3) both groups show sensitivity to narrative level: referential gestures mainly occur with narrative clauses, and pragmatic gestures with meta-and paranarrative clauses. However, the overall group preferences for different functions still lead to different styles. These findings indicate that the two groups differ in gesture rate and, more interestingly, in rhetorical styles, one focused on events and actions in speech and gesture (Swedish), the other alternating between events in speech and gesture, and the highlighting of the presentation of new pieces of information in gesture only (Italian). We propose that the findings suggest that the two groups conceptualise narrative production in different ways reflected in two different rhetorical styles revealed by gesture production more than by speech.

Research paper thumbnail of Early or synchronized gestures facilitate speech recall-a study based on motion capture data

Frontiers in Psychology, 2024

Introduction: Temporal coordination between speech and gestures has been thoroughly studied in na... more Introduction: Temporal coordination between speech and gestures has been thoroughly studied in natural production. In most cases gesture strokes precede or coincide with the stressed syllable in words that they are semantically associated with. Methods: To understand whether processing of speech and gestures is attuned to such temporal coordination, we investigated the effect of delaying, preposing or eliminating individual gestures on the memory for words in an experimental study in which 83 participants watched video sequences of naturalistic 3D-animated speakers generated based on motion capture data. A target word in the sequence appeared (a) with a gesture presented in its original position synchronized with speech, (b) temporally shifted 500 ms before or (c) after the original position, or (d) with the gesture eliminated. Participants were asked to retell the videos in a free recall task. The strength of recall was operationalized as the inclusion of the target word in the free recall. Results: Both eliminated and delayed gesture strokes resulted in reduced recall rates compared to synchronized strokes, whereas there was no difference between advanced (preposed) and synchronized strokes. An item-level analysis also showed that the greater the interval between the onsets of delayed strokes and stressed syllables in target words, the greater the negative effect was on recall. Discussion: These results indicate that speech-gesture synchrony affects memory for speech, and that temporal patterns that are common in production lead to the best recall. Importantly, the study also showcases a procedure for using motion capture-based 3D-animated speakers to create an experimental paradigm for the study of speech-gesture comprehension.

Research paper thumbnail of Which Aspects of Visual Motivation Aid the Implicit Learning of Signs at First Exposure?

Language Learning

We investigated whether sign-naïve learners can infer and learn the meaning of signs after minima... more We investigated whether sign-naïve learners can infer and learn the meaning of signs after minimal exposure to continuous, naturalistic input in the form of a weather forecast in Swedish Sign Language. Participants were L1-English adults. Two experimental groups watched the forecast once (n=40) or twice (n=42); a control group did not (n=42). Participants were then asked to assign meaning to 22 target signs. We explored predictors of meaning assignment with respect to item occurrence frequency and three facets of visual motivation: iconicity, transparency, and gesture similarity. Meaning assignment was enhanced by exposure and item frequency, thereby providing evidence for implicit language learning in a new modality, even under challenging naturalistic conditions. Accuracy was also contingent upon iconicity and transparency, but not upon gesture similarity. Meaning assignment at first exposure is thus visually motivated, although the overall low accuracy rates and further qualitative analyses suggest that visually motivated meaning assignment is not always successful.

Research paper thumbnail of Input in study abroad and views from acquisition: Focus on constructs, operationalization and measurement issues: Introduction to the special issue

Second Language Research

This article briefly discusses the notion of input in a study abroad perspective, situating it ag... more This article briefly discusses the notion of input in a study abroad perspective, situating it against how input is treated in second language acquisition (SLA) more broadly, with a focus on methodological issues, operationalizations, and measurements. It further introduces three studies that examine input as studied in ‘the real wild’, and two studies that instead focus on ‘the digital wild’.

Research paper thumbnail of Today read she the paper: An ERP study of the processing of word order in Swedish L2

There is ample evidence that word order is a problematic domain in L2 usage. In particular, produ... more There is ample evidence that word order is a problematic domain in L2 usage. In particular, production of the verb-second (V2) phenomenon, which requires the finite verb in main clauses to appear in second position, (e.g., Ganuza, 2008 for an overview). Surprisingly, we know very little about how these structures are processed and how production relates to comprehension. We therefore examined how advanced German (N=14) and English (N=14) adult learners, matched for proficiency and age of acquisition (AoA; German M=22, English M=23), process word order in Swedish compared to native speakers (N=20) depending on language background (L1 with [German] or without [English] V2), preposed adverb frequency (frequent idag ‘today’ vs. infrequent hemma ‘at home’, ex. 1), and the length of the preposed constituent (short vs. long prefield, ex. 2). (1) Idag/Hemma laste hon tidningen. vs. *Idag/Hemma hon laste tidningen. Today/At home read she paper.def vs. *Today/At home she read paper.def (2) Idag/Hemma hos Maria laste hon tidningen. vs. *Idag/Hemma hos Maria hon laste tidningen. Today/At home at Maria read she paper.def vs. *Today/At home at Maria she read paper.def. We examined responses to word order violations in an acceptability judgement task and an ERP experiment, and probed the production of word order in a sentence completion task. Preliminary results from the judgment task indicated that native speakers were faster and more accurate on judging sentences than both L2 groups who did not differ. Overall, the more frequent adverb, idag, also affected accuracy and reaction times positively, but there were no interactions with group. The outcome from the sentence completion task showed similar results: native speakers were more accurate than the L2 groups who did not differ, and an overall adverb frequency effect was found, but not difference across groups. In contrast, the ERP data showed different patterns. In native speakers V2 violations elicited a bimodal ERP response, an anterior negativity followed by a posterior P600. These effects were increased in amplitude and the anterior negativity was left lateralized (LAN) when the prefield was long. In the German group a bimodal response was detected only when V2 violations followed a frequent adverb in a long prefield. In other cases only a posterior P600 was evident. The English group, in contrast, showed an early anterior positivity, and a later lateral parietal negativity in the N400 time window that was followed by a posterior P600. These responses were affected only by prefield length and only in amplitude. Overall, the results indicated that advanced German and English learners, matched on proficiency and AoA, who performed similarly on behavioural measures of comprehension and production of word order, still differed in online processing. More specifically, language background mattered since the German learners whose L1 share V2 with target Swedish, overall showed similar ERP patterns to native speakers. In contrast, the English learners, whose L1 does not share V2, showed more variation in their ERP responses. We discuss the implications of these findings for theories of crosslinguistic influence and theories of nativelike syntactic processing. (Less)

Research paper thumbnail of Which Aspects of Visual Motivation Aid the Implicit Learning of Signs at First Exposure

Language Learning, 2023

We investigated whether sign-naïve learners can infer and learn the meaning of signs after minima... more We investigated whether sign-naïve learners can infer and learn the meaning of signs after minimal exposure to continuous, naturalistic input in the form of a weather forecast in Swedish Sign Language. Participants were L1-English adults. Two experimental groups watched the forecast once (n=40) or twice (n=42); a control group did not (n=42). Participants were then asked to assign meaning to 22 target signs. We explored predictors of meaning assignment with respect to item occurrence frequency and three facets of visual motivation: iconicity, transparency, and gesture similarity. Meaning assignment was enhanced by exposure and item frequency, thereby providing evidence for implicit language learning in a new modality, even under challenging naturalistic conditions. Accuracy was also contingent upon iconicity and transparency, but not upon gesture similarity. Meaning assignment at first exposure is thus visually motivated, although the overall low accuracy rates and further qualitative analyses suggest that visually motivated meaning assignment is not always successful.

Research paper thumbnail of The role of manual gestures in second language comprehension: a simultaneous interpreting experiment

Frontiers in Psychology, 2023

Manual gestures and speech form a single integrated system during native language comprehension. ... more Manual gestures and speech form a single integrated system during native language comprehension. However, it remains unclear whether this hold for second language (L2) comprehension, more specifically for simultaneous interpreting (SI), which involves comprehension in one language and simultaneous production in another. In a combined mismatch and priming paradigm, we presented Swedish speakers fluent in L2 English with multimodal stimuli in which speech was congruent or incongruent with a gesture. A picture prime was displayed before the stimuli. Participants had to decide whether the video was related to the prime, focusing either on the auditory or the visual information. Participants performed the task either during passive viewing or during SI into their L1 Swedish (order counterbalanced). Incongruent stimuli yielded longer reaction times than congruent stimuli, during both viewing and interpreting. Visual and audio targets were processed equally easily in both activities. However, in both activities incongruent speech was more disruptive for gesture processing than incongruent gesture was for speech processing. Thus, the data only partly supports the expected mutual and obligatory interaction of gesture and speech in L2 comprehension. Interestingly, there were no differences between activities suggesting that the language comprehension component in SI shares features with other (L2) comprehension tasks.

Research paper thumbnail of Structural priming of code-switches in non-shared-word-order utterances: The effect of lexical repetition

Bilingualism: Language and Cognition

Code-switching is generally dispreferred at points of non-shared word order across a bilingual&#3... more Code-switching is generally dispreferred at points of non-shared word order across a bilingual's two languages. In priming studies, this dispreference persists even following exposure to a code-switched non-shared-word-order utterance. The present study delves deeper into the scope of code-switching priming by investigating whether lexical repetition across target and prime, a factor known to boost structural priming, can increase code-switching at points of word order divergence. Afrikaans–English bilinguals (n=46) heard prime sentences in which word order, lexical repetition, and switch position were manipulated and subsequently produced code-switched picture descriptions. The results show that lexical repetition boosts the priming of code-switching in a non-shared word order. The findings demonstrate that code-switching in production is affected by a dynamic interplay between factors both language-internal (i.e., word order) and language-external (i.e., priming, and specifica...

Research paper thumbnail of When Attentional and Politeness Demands Clash: The Case of Mutual Gaze Avoidance and Chin Pointing in Quiahije Chatino

Journal of Nonverbal Behavior, 2023

Pointing with the chin is a practice attested worldwide: it is an effective and highly recognizab... more Pointing with the chin is a practice attested worldwide: it is an effective and highly recognizable device for reorienting the attention of the addressee. For the chin point to be observed, the addressee must attend carefully to the movements of the sender's head. This demand comes into conflict with the politeness norms of many cultures, since these often require conversationalists to avoid meeting the gaze of their interlocutor, and can require them to look away from their interlocutor's face and head. In this paper we explore how the chin point is successfully used in just such a culture, among the Chatino indigenous group of Oaxaca, Mexico. We analyze interactions between multiple dyads of Chatino speakers, examining how senders invite visual attention to the pointing gesture, and how addressees signal that attention, while both participants avoid stretches of mutual gaze. We find that in the Chatino context, the senior (or higher-status) party to the conversation is highly consistent in training their gaze away from their interlocutor. This allows their interlocutor to give visual attention to their face without the risk of meeting the gaze of a higher-status sender, and facilitates close attention to head movements including the chin point. Abstracts in Spanish and Quiahije Chatino are published as appendices. Se incluyen como apéndices resúmenes en español y en el chatino de San Juan Quiahije.

Research paper thumbnail of Reviewing the potential of the Experience Sampling Method (ESM) for capturing second language exposure and use

Second Language Research, 2021

Frequent language exposure and use are among the most important conditions for successful languag... more Frequent language exposure and use are among the most important conditions for successful language learning, whether in classrooms, during study abroad, or in other informal contexts. Research probing exposure and usage often relies on one-off self-report questionnaires in which participants estimate their typical level of language exposure over extended periods of time, often long after it occurred. This may negatively affect the validity of the resulting data. This article instead explores the potential of methods used in medical and psychological research, variably known as the Experience Sampling Method (ESM), Ecological Momentary Assessment (EMA), or diary methods. These methods are often combined with electronic and mobile survey applications to elicit self-report assessments at frequent, sometimes randomized intervals. We consider the possibilities of these methods for strengthening research into language exposure and use, second language acquisition more broadly, and study abroad research specifically. The methods have the potential to drastically reduce biases associated with summative recall. Additionally, they enable researchers to collect richer data about how individuals engage with language differently over time, and the contexts in which they do so, thus ultimately contributing to our understanding of individual differences in language acquisition.

Research paper thumbnail of Input in study abroad and views from acquisition: Focus on constructs, operationalization and measurement issues: Introduction to the special issue

Second Language Research, 2022

This article briefly discusses the notion of input in a study abroad perspective, situating it ag... more This article briefly discusses the notion of input in a study abroad perspective, situating it against how input is treated in second language acquisition (SLA) more broadly, with a focus on methodological issues, operationalizations, and measurements. It further introduces three studies that examine input as studied in 'the real wild', and two studies that instead focus on 'the digital wild'.

Research paper thumbnail of Semantically related gestures facilitate language comprehension during simultaneous interpreting

Bilingualism: Language and Cognition, 2022

Manual co-speech gestures can facilitate language comprehension, but do they influence language c... more Manual co-speech gestures can facilitate language comprehension, but do they influence language comprehension in simultaneous interpreters, and if so, is this influence modulated by simultaneous interpreting (SI) and/or by interpreting experience? In a picture-matching task, 24 professional interpreters and 24 professional translators were exposed to utterances accompanied by semantically matching representational gestures, semantically unrelated pragmatic gestures, or no gestures while viewing passively (interpreters and translators) or during SI (interpreters only). During passive viewing, both groups were faster with semantically related than with semantically unrelated gestures. During SI, interpreters showed the same result. The results suggest that language comprehension is sensitive to the semantic relationship between speech and gesture, and facilitated when speech and gestures are semantically linked. This sensitivity is not modulated by SI or interpreting experience. Thus, despite simultaneous interpreters' extreme language use, multimodal language processing facilitates comprehension in SI the same way as in all other language processing.

Research paper thumbnail of The Lang-Track-App: Open-Source Tools for Implementing the Experience Sampling Method in Second Language Acquisition Research

Language Learning, 2022

This paper introduces the Lang-Track-App, a smartphone application and backend system to aid seco... more This paper introduces the Lang-Track-App, a smartphone application and backend system to aid second language acquisition researchers in implementing the experience sampling method (ESM). Surveying research participants with the Lang-Track-App multiple times per day can yield exceptionally rich data that can be analyzed CRediT author statement-Henriette Arndt: conceptualization (supporting); methodology (equal); investigation (lead); formal analysis (lead); writing (lead). Jonas Granfeldt: conceptualization (lead); methodology (equal); investigation (supporting); project administration (lead); writing-reviewing & editing (supporting). Marianne Gullberg: conceptualization (lead); methodology (equal); project administration (supporting); writing-reviewing & editing (supporting). Stephan Björck: software (equal). Josef Granqvist: software (equal). A one-page Accessible Summary of this article in non-technical language is freely available in the Supporting Information online and at https://oasis-database.org We gratefully acknowledge Stephan Björck and Josef Granqvist's work on developing the Lang-Track-App software. The Lang-Track-App project was funded by the Marcus and Amalia Wallenberg Foundation (grant number MAW 2018.0025). We also gratefully acknowledge support from the Lund University Humanities Lab. We thank the reviewers and editor for their helpful comments on previous versions of this manuscript. In addition, we express our thanks for the fruitful and stimulating discussions that we had while working on the Study Abroad Research in European Perspectives (SAREP) COST Action 15130 led by

Research paper thumbnail of REaCHeS" Dataset from The Language Archive

Research paper thumbnail of Motion capture-based animated characters for the study of speech–gesture integration

Behavior Research Methods, 2019

Digitally animated characters are promising tools in research studying how we integrate informati... more Digitally animated characters are promising tools in research studying how we integrate information from speech and visual sources such as gestures because they allow specific gesture features to be manipulated in isolation. We present an approach combining motion capture and 3D-animated characters that allows us to manipulate natural individual gesture strokes for experimental purposes, for example to temporally shift and present gestures in ecologically valid sequences. We exemplify how such stimuli can be used in an experiment investigating implicit detection of speech–gesture (a) synchrony, and discuss the general applicability of the workflow for research in this domain.

Research paper thumbnail of Editorial: Visual Language

Frontiers in Psychology, 2019

Research paper thumbnail of Language background affects online word order processing in a second language but not offline

Bilingualism: Language and Cognition, 2018

This study examines possible crosslinguistic influence on basic word order processing in a second... more This study examines possible crosslinguistic influence on basic word order processing in a second language (L2). Targeting Swedish V2 word order we investigate adult German learners (+V2 in the L1) and English learners (-V2 in the L1) of Swedish who are matched for proficiency. We report results from two offline behavioural tasks (written production, metalinguistic judgements), and online processing as measured by event-related potentials (ERPs). All groups showed sensitivity to word order violations behaviourally and neurocognitively. Behaviourally, the learners differed from the native speakers only on judgements. Crucially, they did not differ from each other. Neurocognitively, all groups showed a similar increased centro-parietal P600 ERP-effect, but German learners (+V2) displayed more nativelike anterior ERP-effects than English learners (-V2). The results suggest crosslinguistic influence in that the presence of a similar word order in the L1 can facilitate online processing ...

Research paper thumbnail of Can sign-naïve adults learn about the phonological regularities of an unfamiliar sign language from minimal exposure

Language Development Research, 2024

Adults can extract phonological regularities from just several minutes' exposure to naturalistic ... more Adults can extract phonological regularities from just several minutes' exposure to naturalistic input of an unknown spoken language (Gullberg et al., 2010). We examined whether such implicit statistical learning mechanisms also operate in the sign language modality. The input materials consisted of a continuous sign stream in the form of a weather forecast in Svenskt Teckenspråk (STS). L1speakers of English with no prior knowledge of a sign language were assigned to two experimental groups who watched the forecast once (N=43) or twice (N=38), and a control group who did not watch it (N=40). Participants completed a 'surprise' lexical decision task designed to tap into their awareness of the phonological properties of the core STS lexicon. They viewed individual signs and indicated whether or not these could be real STS signs. The signs comprised four sets: STS signs that (1) were presented, and (2) were not presented, in the forecast; and signs that are not STS signs and (3) contain handshapes outside the STS handshape inventory, and (4) contain sets of phonological features that are dispreferred across sign languages. We found no evidence of any learning of STS phonological regularities. Considered in conjunction with two companion studies which did demonstrate some learning of sign forms and their meanings from these same input materials, our findings suggest limits to what can be learnt after just a few minutes of implicit and naturalistic exposure to language in an unfamiliar modality: information about specific lexical items is learnable, but information that requires generalisation across items may require greater amounts, or a different quality, of input.

Research paper thumbnail of Behavioral science labs: How to solve the multi-user problem

Behavior Research Methods, 2024

When lab resources are shared among multiple research projects, issues such as experimental integ... more When lab resources are shared among multiple research projects, issues such as experimental integrity, replicability, and data safety become important. Different research projects often need different software and settings that may well conflict with one another, and data collected for one project may not be safeguarded from exposure to researchers from other projects. In this paper we provide an infrastructure design and an open-source tool, labManager, that render multiuser lab facilities in the behavioral sciences accessible to research projects with widely varying needs. The solutions proposed ensure ease of management while simultaneously offering maximum flexibility by providing research projects with fully separated bare metal environments. This solution also ensures that collected data is kept separate, and compliant with relevant ethical standards and regulations such as General Data Protection Regulation (GDPR) legislation. Furthermore, we discuss preconditions for running shared lab facilities and provide practical advice.

Research paper thumbnail of Frontiers in Psychology: Visual Language

Frontiers in Psychology, 2019

Research paper thumbnail of Providing evidence for a well-worn stereotype: Italians and Swedes do gesture differently

Frontiers in Communication, 2024

Across cultures and languages spontaneous speech is often accompanied by gestures. It is a popula... more Across cultures and languages spontaneous speech is often accompanied by gestures. It is a popular belief that people in Italy gesture more than people in Northern Europe, such as in Sweden. Despite this general assumption few studies empirically investigate cultural differences in gesture frequency and gesture function under similar circumstances. This study compares the spoken and gestural behaviours of Italian and Swedish speakers, assumed to represent gesture-rich vs. gesture-sparse cultures. We examine the groups' gestural behaviour for frequency, and in terms of possible differences in rhetorical style probing the distribution of gestural functions (referential vs. pragmatic) across narrative levels (narrative, metanarrative, and paranarrative). The results show that (1) Italians overall do gesture more than Swedes; (2) Italians produce more pragmatic gestures than Swedes who produce more referential gestures; (3) both groups show sensitivity to narrative level: referential gestures mainly occur with narrative clauses, and pragmatic gestures with meta-and paranarrative clauses. However, the overall group preferences for different functions still lead to different styles. These findings indicate that the two groups differ in gesture rate and, more interestingly, in rhetorical styles, one focused on events and actions in speech and gesture (Swedish), the other alternating between events in speech and gesture, and the highlighting of the presentation of new pieces of information in gesture only (Italian). We propose that the findings suggest that the two groups conceptualise narrative production in different ways reflected in two different rhetorical styles revealed by gesture production more than by speech.

Research paper thumbnail of Early or synchronized gestures facilitate speech recall-a study based on motion capture data

Frontiers in Psychology, 2024

Introduction: Temporal coordination between speech and gestures has been thoroughly studied in na... more Introduction: Temporal coordination between speech and gestures has been thoroughly studied in natural production. In most cases gesture strokes precede or coincide with the stressed syllable in words that they are semantically associated with. Methods: To understand whether processing of speech and gestures is attuned to such temporal coordination, we investigated the effect of delaying, preposing or eliminating individual gestures on the memory for words in an experimental study in which 83 participants watched video sequences of naturalistic 3D-animated speakers generated based on motion capture data. A target word in the sequence appeared (a) with a gesture presented in its original position synchronized with speech, (b) temporally shifted 500 ms before or (c) after the original position, or (d) with the gesture eliminated. Participants were asked to retell the videos in a free recall task. The strength of recall was operationalized as the inclusion of the target word in the free recall. Results: Both eliminated and delayed gesture strokes resulted in reduced recall rates compared to synchronized strokes, whereas there was no difference between advanced (preposed) and synchronized strokes. An item-level analysis also showed that the greater the interval between the onsets of delayed strokes and stressed syllables in target words, the greater the negative effect was on recall. Discussion: These results indicate that speech-gesture synchrony affects memory for speech, and that temporal patterns that are common in production lead to the best recall. Importantly, the study also showcases a procedure for using motion capture-based 3D-animated speakers to create an experimental paradigm for the study of speech-gesture comprehension.

Research paper thumbnail of Which Aspects of Visual Motivation Aid the Implicit Learning of Signs at First Exposure?

Language Learning

We investigated whether sign-naïve learners can infer and learn the meaning of signs after minima... more We investigated whether sign-naïve learners can infer and learn the meaning of signs after minimal exposure to continuous, naturalistic input in the form of a weather forecast in Swedish Sign Language. Participants were L1-English adults. Two experimental groups watched the forecast once (n=40) or twice (n=42); a control group did not (n=42). Participants were then asked to assign meaning to 22 target signs. We explored predictors of meaning assignment with respect to item occurrence frequency and three facets of visual motivation: iconicity, transparency, and gesture similarity. Meaning assignment was enhanced by exposure and item frequency, thereby providing evidence for implicit language learning in a new modality, even under challenging naturalistic conditions. Accuracy was also contingent upon iconicity and transparency, but not upon gesture similarity. Meaning assignment at first exposure is thus visually motivated, although the overall low accuracy rates and further qualitative analyses suggest that visually motivated meaning assignment is not always successful.

Research paper thumbnail of Input in study abroad and views from acquisition: Focus on constructs, operationalization and measurement issues: Introduction to the special issue

Second Language Research

This article briefly discusses the notion of input in a study abroad perspective, situating it ag... more This article briefly discusses the notion of input in a study abroad perspective, situating it against how input is treated in second language acquisition (SLA) more broadly, with a focus on methodological issues, operationalizations, and measurements. It further introduces three studies that examine input as studied in ‘the real wild’, and two studies that instead focus on ‘the digital wild’.

Research paper thumbnail of Today read she the paper: An ERP study of the processing of word order in Swedish L2

There is ample evidence that word order is a problematic domain in L2 usage. In particular, produ... more There is ample evidence that word order is a problematic domain in L2 usage. In particular, production of the verb-second (V2) phenomenon, which requires the finite verb in main clauses to appear in second position, (e.g., Ganuza, 2008 for an overview). Surprisingly, we know very little about how these structures are processed and how production relates to comprehension. We therefore examined how advanced German (N=14) and English (N=14) adult learners, matched for proficiency and age of acquisition (AoA; German M=22, English M=23), process word order in Swedish compared to native speakers (N=20) depending on language background (L1 with [German] or without [English] V2), preposed adverb frequency (frequent idag ‘today’ vs. infrequent hemma ‘at home’, ex. 1), and the length of the preposed constituent (short vs. long prefield, ex. 2). (1) Idag/Hemma laste hon tidningen. vs. *Idag/Hemma hon laste tidningen. Today/At home read she paper.def vs. *Today/At home she read paper.def (2) Idag/Hemma hos Maria laste hon tidningen. vs. *Idag/Hemma hos Maria hon laste tidningen. Today/At home at Maria read she paper.def vs. *Today/At home at Maria she read paper.def. We examined responses to word order violations in an acceptability judgement task and an ERP experiment, and probed the production of word order in a sentence completion task. Preliminary results from the judgment task indicated that native speakers were faster and more accurate on judging sentences than both L2 groups who did not differ. Overall, the more frequent adverb, idag, also affected accuracy and reaction times positively, but there were no interactions with group. The outcome from the sentence completion task showed similar results: native speakers were more accurate than the L2 groups who did not differ, and an overall adverb frequency effect was found, but not difference across groups. In contrast, the ERP data showed different patterns. In native speakers V2 violations elicited a bimodal ERP response, an anterior negativity followed by a posterior P600. These effects were increased in amplitude and the anterior negativity was left lateralized (LAN) when the prefield was long. In the German group a bimodal response was detected only when V2 violations followed a frequent adverb in a long prefield. In other cases only a posterior P600 was evident. The English group, in contrast, showed an early anterior positivity, and a later lateral parietal negativity in the N400 time window that was followed by a posterior P600. These responses were affected only by prefield length and only in amplitude. Overall, the results indicated that advanced German and English learners, matched on proficiency and AoA, who performed similarly on behavioural measures of comprehension and production of word order, still differed in online processing. More specifically, language background mattered since the German learners whose L1 share V2 with target Swedish, overall showed similar ERP patterns to native speakers. In contrast, the English learners, whose L1 does not share V2, showed more variation in their ERP responses. We discuss the implications of these findings for theories of crosslinguistic influence and theories of nativelike syntactic processing. (Less)

Research paper thumbnail of Which Aspects of Visual Motivation Aid the Implicit Learning of Signs at First Exposure

Language Learning, 2023

We investigated whether sign-naïve learners can infer and learn the meaning of signs after minima... more We investigated whether sign-naïve learners can infer and learn the meaning of signs after minimal exposure to continuous, naturalistic input in the form of a weather forecast in Swedish Sign Language. Participants were L1-English adults. Two experimental groups watched the forecast once (n=40) or twice (n=42); a control group did not (n=42). Participants were then asked to assign meaning to 22 target signs. We explored predictors of meaning assignment with respect to item occurrence frequency and three facets of visual motivation: iconicity, transparency, and gesture similarity. Meaning assignment was enhanced by exposure and item frequency, thereby providing evidence for implicit language learning in a new modality, even under challenging naturalistic conditions. Accuracy was also contingent upon iconicity and transparency, but not upon gesture similarity. Meaning assignment at first exposure is thus visually motivated, although the overall low accuracy rates and further qualitative analyses suggest that visually motivated meaning assignment is not always successful.

Research paper thumbnail of The role of manual gestures in second language comprehension: a simultaneous interpreting experiment

Frontiers in Psychology, 2023

Manual gestures and speech form a single integrated system during native language comprehension. ... more Manual gestures and speech form a single integrated system during native language comprehension. However, it remains unclear whether this hold for second language (L2) comprehension, more specifically for simultaneous interpreting (SI), which involves comprehension in one language and simultaneous production in another. In a combined mismatch and priming paradigm, we presented Swedish speakers fluent in L2 English with multimodal stimuli in which speech was congruent or incongruent with a gesture. A picture prime was displayed before the stimuli. Participants had to decide whether the video was related to the prime, focusing either on the auditory or the visual information. Participants performed the task either during passive viewing or during SI into their L1 Swedish (order counterbalanced). Incongruent stimuli yielded longer reaction times than congruent stimuli, during both viewing and interpreting. Visual and audio targets were processed equally easily in both activities. However, in both activities incongruent speech was more disruptive for gesture processing than incongruent gesture was for speech processing. Thus, the data only partly supports the expected mutual and obligatory interaction of gesture and speech in L2 comprehension. Interestingly, there were no differences between activities suggesting that the language comprehension component in SI shares features with other (L2) comprehension tasks.

Research paper thumbnail of Structural priming of code-switches in non-shared-word-order utterances: The effect of lexical repetition

Bilingualism: Language and Cognition

Code-switching is generally dispreferred at points of non-shared word order across a bilingual&#3... more Code-switching is generally dispreferred at points of non-shared word order across a bilingual's two languages. In priming studies, this dispreference persists even following exposure to a code-switched non-shared-word-order utterance. The present study delves deeper into the scope of code-switching priming by investigating whether lexical repetition across target and prime, a factor known to boost structural priming, can increase code-switching at points of word order divergence. Afrikaans–English bilinguals (n=46) heard prime sentences in which word order, lexical repetition, and switch position were manipulated and subsequently produced code-switched picture descriptions. The results show that lexical repetition boosts the priming of code-switching in a non-shared word order. The findings demonstrate that code-switching in production is affected by a dynamic interplay between factors both language-internal (i.e., word order) and language-external (i.e., priming, and specifica...

Research paper thumbnail of When Attentional and Politeness Demands Clash: The Case of Mutual Gaze Avoidance and Chin Pointing in Quiahije Chatino

Journal of Nonverbal Behavior, 2023

Pointing with the chin is a practice attested worldwide: it is an effective and highly recognizab... more Pointing with the chin is a practice attested worldwide: it is an effective and highly recognizable device for reorienting the attention of the addressee. For the chin point to be observed, the addressee must attend carefully to the movements of the sender's head. This demand comes into conflict with the politeness norms of many cultures, since these often require conversationalists to avoid meeting the gaze of their interlocutor, and can require them to look away from their interlocutor's face and head. In this paper we explore how the chin point is successfully used in just such a culture, among the Chatino indigenous group of Oaxaca, Mexico. We analyze interactions between multiple dyads of Chatino speakers, examining how senders invite visual attention to the pointing gesture, and how addressees signal that attention, while both participants avoid stretches of mutual gaze. We find that in the Chatino context, the senior (or higher-status) party to the conversation is highly consistent in training their gaze away from their interlocutor. This allows their interlocutor to give visual attention to their face without the risk of meeting the gaze of a higher-status sender, and facilitates close attention to head movements including the chin point. Abstracts in Spanish and Quiahije Chatino are published as appendices. Se incluyen como apéndices resúmenes en español y en el chatino de San Juan Quiahije.

Research paper thumbnail of Reviewing the potential of the Experience Sampling Method (ESM) for capturing second language exposure and use

Second Language Research, 2021

Frequent language exposure and use are among the most important conditions for successful languag... more Frequent language exposure and use are among the most important conditions for successful language learning, whether in classrooms, during study abroad, or in other informal contexts. Research probing exposure and usage often relies on one-off self-report questionnaires in which participants estimate their typical level of language exposure over extended periods of time, often long after it occurred. This may negatively affect the validity of the resulting data. This article instead explores the potential of methods used in medical and psychological research, variably known as the Experience Sampling Method (ESM), Ecological Momentary Assessment (EMA), or diary methods. These methods are often combined with electronic and mobile survey applications to elicit self-report assessments at frequent, sometimes randomized intervals. We consider the possibilities of these methods for strengthening research into language exposure and use, second language acquisition more broadly, and study abroad research specifically. The methods have the potential to drastically reduce biases associated with summative recall. Additionally, they enable researchers to collect richer data about how individuals engage with language differently over time, and the contexts in which they do so, thus ultimately contributing to our understanding of individual differences in language acquisition.

Research paper thumbnail of Input in study abroad and views from acquisition: Focus on constructs, operationalization and measurement issues: Introduction to the special issue

Second Language Research, 2022

This article briefly discusses the notion of input in a study abroad perspective, situating it ag... more This article briefly discusses the notion of input in a study abroad perspective, situating it against how input is treated in second language acquisition (SLA) more broadly, with a focus on methodological issues, operationalizations, and measurements. It further introduces three studies that examine input as studied in 'the real wild', and two studies that instead focus on 'the digital wild'.

Research paper thumbnail of Semantically related gestures facilitate language comprehension during simultaneous interpreting

Bilingualism: Language and Cognition, 2022

Manual co-speech gestures can facilitate language comprehension, but do they influence language c... more Manual co-speech gestures can facilitate language comprehension, but do they influence language comprehension in simultaneous interpreters, and if so, is this influence modulated by simultaneous interpreting (SI) and/or by interpreting experience? In a picture-matching task, 24 professional interpreters and 24 professional translators were exposed to utterances accompanied by semantically matching representational gestures, semantically unrelated pragmatic gestures, or no gestures while viewing passively (interpreters and translators) or during SI (interpreters only). During passive viewing, both groups were faster with semantically related than with semantically unrelated gestures. During SI, interpreters showed the same result. The results suggest that language comprehension is sensitive to the semantic relationship between speech and gesture, and facilitated when speech and gestures are semantically linked. This sensitivity is not modulated by SI or interpreting experience. Thus, despite simultaneous interpreters' extreme language use, multimodal language processing facilitates comprehension in SI the same way as in all other language processing.

Research paper thumbnail of The Lang-Track-App: Open-Source Tools for Implementing the Experience Sampling Method in Second Language Acquisition Research

Language Learning, 2022

This paper introduces the Lang-Track-App, a smartphone application and backend system to aid seco... more This paper introduces the Lang-Track-App, a smartphone application and backend system to aid second language acquisition researchers in implementing the experience sampling method (ESM). Surveying research participants with the Lang-Track-App multiple times per day can yield exceptionally rich data that can be analyzed CRediT author statement-Henriette Arndt: conceptualization (supporting); methodology (equal); investigation (lead); formal analysis (lead); writing (lead). Jonas Granfeldt: conceptualization (lead); methodology (equal); investigation (supporting); project administration (lead); writing-reviewing & editing (supporting). Marianne Gullberg: conceptualization (lead); methodology (equal); project administration (supporting); writing-reviewing & editing (supporting). Stephan Björck: software (equal). Josef Granqvist: software (equal). A one-page Accessible Summary of this article in non-technical language is freely available in the Supporting Information online and at https://oasis-database.org We gratefully acknowledge Stephan Björck and Josef Granqvist's work on developing the Lang-Track-App software. The Lang-Track-App project was funded by the Marcus and Amalia Wallenberg Foundation (grant number MAW 2018.0025). We also gratefully acknowledge support from the Lund University Humanities Lab. We thank the reviewers and editor for their helpful comments on previous versions of this manuscript. In addition, we express our thanks for the fruitful and stimulating discussions that we had while working on the Study Abroad Research in European Perspectives (SAREP) COST Action 15130 led by

Research paper thumbnail of REaCHeS" Dataset from The Language Archive

Research paper thumbnail of Motion capture-based animated characters for the study of speech–gesture integration

Behavior Research Methods, 2019

Digitally animated characters are promising tools in research studying how we integrate informati... more Digitally animated characters are promising tools in research studying how we integrate information from speech and visual sources such as gestures because they allow specific gesture features to be manipulated in isolation. We present an approach combining motion capture and 3D-animated characters that allows us to manipulate natural individual gesture strokes for experimental purposes, for example to temporally shift and present gestures in ecologically valid sequences. We exemplify how such stimuli can be used in an experiment investigating implicit detection of speech–gesture (a) synchrony, and discuss the general applicability of the workflow for research in this domain.

Research paper thumbnail of Editorial: Visual Language

Frontiers in Psychology, 2019

Research paper thumbnail of Language background affects online word order processing in a second language but not offline

Bilingualism: Language and Cognition, 2018

This study examines possible crosslinguistic influence on basic word order processing in a second... more This study examines possible crosslinguistic influence on basic word order processing in a second language (L2). Targeting Swedish V2 word order we investigate adult German learners (+V2 in the L1) and English learners (-V2 in the L1) of Swedish who are matched for proficiency. We report results from two offline behavioural tasks (written production, metalinguistic judgements), and online processing as measured by event-related potentials (ERPs). All groups showed sensitivity to word order violations behaviourally and neurocognitively. Behaviourally, the learners differed from the native speakers only on judgements. Crucially, they did not differ from each other. Neurocognitively, all groups showed a similar increased centro-parietal P600 ERP-effect, but German learners (+V2) displayed more nativelike anterior ERP-effects than English learners (-V2). The results suggest crosslinguistic influence in that the presence of a similar word order in the L1 can facilitate online processing ...

Research paper thumbnail of Structural priming of code-switches in nonshared- word-order utterances: The effect of lexical repetition

Bilingualism: Language and Cognition, 2023

Code-switching is generally dispreferred at points of non-shared word order across a bilingual's ... more Code-switching is generally dispreferred at points of non-shared word order across a bilingual's two languages. In priming studies, this dispreference persists even following exposure to a codeswitched non-shared-word-order utterance. The present study delves deeper into the scope of code-switching priming by investigating whether lexical repetition across target and prime, a factor known to boost structural priming, can increase code-switching at points of word order divergence. Afrikaans-English bilinguals (n=46) heard prime sentences in which word order, lexical repetition, and switch position were manipulated and subsequently produced code-switched picture descriptions. The results show that lexical repetition boosts the priming of code-switching in a non-shared word order. The findings demonstrate that code-switching in production is affected by a dynamic interplay between factors both language-internal (i.e., word order) and language-external (i.e., priming, and specifically lexical repetition).

Research paper thumbnail of Gestures in Language Development

Benjamins Current Topics, 2010

The purpose of the current paper is to outline how gestures can contribute to the study of some k... more The purpose of the current paper is to outline how gestures can contribute to the study of some key issues in language development. Specifically, we (1) briefly summarise what is already known about gesture in the domains of first and second language development, and development or changes over the life span more generally; (2) highlight theoretical and empirical issues in these domains where gestures can contribute in important ways to further our understanding; and (3) summarise some common themes in all strands of research on language development that could be the target of concentrated research efforts.

Research paper thumbnail of Special issue of Studies in Second Language Acquisition: Gesture and SLA – Toward an Integrated Approach

Research paper thumbnail of Special issue on Gestures and second language acquisition

Research paper thumbnail of Special issue of Language Learning: Time to speak. Cognitive and neural prerequisites for time in language

Research paper thumbnail of Special issue Gestures in language development

Research paper thumbnail of Encyclopedia of Applied Linguistics: Cognitive second language acquisition

Research paper thumbnail of Visual language (A Frontiers in Psychology e-book)

Research paper thumbnail of The earliest stages of language learning

To understand the nature of language learning, the factors that influence it, and the mechanisms ... more To understand the nature of language learning, the factors that influence it, and the mechanisms that govern it, it is crucial to study the very earliest stages of language learning. This volume provides a state-of-the art overview of what we know about the cognitive and neurobiological aspects of the adult capacity for language learning. It brings together studies from several fields that examine learning from multiple perspectives using various methods. The papers examine learning after anything from a few minutes to months of language exposure; they target the learning of both artificial and natural languages, involve both explicit and implicit learning, and cover linguistic domains ranging from phonology and semantics to morphosyntax. The findings will inform and extend further studies of language learning in multiple disciplines.

Research paper thumbnail of The cognitive neuroscience of second language acquisition

The articles in this volume explore the cognitive neuroscience of second language acquisition fro... more The articles in this volume explore the cognitive neuroscience of second language acquisition from the perspectives of critical/sensitive periods, maturational effects, individual differences, neural regions involved, and processing characteristics. The research methodologies used include functional magnetic resonance imaging (fMRI), positron emission tomography (PET), and event related potentials (ERP). Questions addressed include: Which brain areas are reliably activated in second language processing? Are they the same or different from those activated in first language acquisition and use? What are the behavioral consequences of individual differences among brains? What are the consequences of anatomical and physiological differences, learner proficiency effects, critical/sensitive periods? What role does degeneracy, in which two different neural systems can produce the same behavioral output, play? What does it mean that learners' brains respond to linguistic distinctions that cannot be recognized or produced yet? The studies in this volume provide initial answers to all of these questions.

Research paper thumbnail of Gesture as a communication strategy in second language discourse. A study of learners of French and Swedish

Gestures are often regarded as the most typical compensatory device used by language learners in ... more Gestures are often regarded as the most typical compensatory device used by language learners in communicative trouble. Yet gestural solutions to communicative problems have rarely been studied within any theory of second language use. The work presented in this volume aims to account for second language learners’ strategic use of speech-associated gestures by combining a process-oriented framework for communication strategies with a cognitive theory of gesture.
Two empirical studies are presented. The production study investigates Swedish learners of French and French learners of Swedish and their use of strategic gestures. The results, which are based on analyses of both individual and group behaviour, contradict popular opinion as well as theoretical assumptions from both fields. Gestures are not primarily used to replace speech, nor are they chiefly mimetic. Instead, learners use gestures with speech, and although they do exploit mimetic gestures to solve lexical problems, they also use more abstract gestures to handle discourse-related difficulties and metalinguistic commentary. The influence of factors such as proficiency, task, culture, and strategic competence on gesture use is discussed, and the oral and gestural strategic modes are compared. In the evaluation study, native speakers’ assessments of learners’ gestures, and the potential effect of gestures on evaluations of proficiency are analysed and discussed in terms of individual communicative style.
Compensatory gestures function at multiple communicative levels. This has implications for theories of communication strategies, and an expansion of the existing frameworks is discussed taking both cognitive and interactive aspects into account."

Research paper thumbnail of Gestures in language development

Gestures are prevalent in communication and tightly linked to language and speech. As such they c... more Gestures are prevalent in communication and tightly linked to language and speech. As such they can shed important light on issues of language development across the lifespan. This volume, originally published as a Special Issue of Gesture Volume 8:2 (2008), brings together studies from different disciplines that examine language development in children and adults from varying perspectives. It provides a review of common theoretical and empirical themes, and the contributions address topics such as gesture use in prelinguistic infants, the relationship between gestures and lexical development in typically and atypically developing children and in second language learners, what gestures reveal about discourse, and how all languages that adult second language speakers know can influence each other. The papers exemplify a vibrant new field of study with relevance for multiple disciplines.

Research paper thumbnail of From gesture in conversation to visible action as utterance

Language use is fundamentally multimodal. Speakers use their hands to point to locations, to repr... more Language use is fundamentally multimodal. Speakers use their hands to point to locations, to represent content and to comment on ongoing talk; they position their bodies to show their orientation and stance in interaction; they use facial displays to comment on what is being said; and they engage in mutual gaze to establish intersubjectivity. This volume brings together studies by leading scholars from several fields on gaze and facial displays, on the relationship between gestures, sign, and language, on pointing and other conventionalized forms of manual expression, on gestures and language evolution, and on gestures in child development. The papers in this collection honor Adam Kendon whose pioneering work has laid the theoretical and methodological foundations for contemporary studies of multimodality, gestures, and utterance visible action.

Research paper thumbnail of Time to speak. Cognitive and neural prerequisites for Time in language

Time is a fundamental aspect of human cognition and action. All languages have developed rich mea... more Time is a fundamental aspect of human cognition and action. All languages have developed rich means to express various facets of time, such as bare time spans, their position on the time line, or their duration. This volume explores what we know about the neural and cognitive representations of time that speakers can draw on in language.