Incremental Eects of Mismatch during Picture-Sentence Integration: Evidence from Eye-tracking (original) (raw)
Related papers
Psychological Review, 1976
We consider several recent information^processing models of sentence-picture matching to assess their implications for sentence processing. The representational component of the models describes a task-specific "verification representation" that is derived from a more general representation needed for comprehension. The specific models do not describe the processes by which these representations are derived; nor do the assumptions common to the models shed light on the structure of these verification representations. The models are, at best, detailed descriptions of the processes by which subjects verify sentences that they have already understood against pictures that they have already perceived.
Cognition, 2019
Many studies have shown that sentences implying an object to have a certain shape produce a robust reaction time advantage for shape-matching pictures in the sentence-picture verification task. Typically, this finding has been interpreted as evidence for perceptual simulation, i.e., that access to implicit shape information involves the activation of modality-specific visual processes. It follows from this proposal that disrupting visual processing during sentence comprehension should interfere with perceptual simulation and obliterate the match effect. Here we directly test this hypothesis. Participants listened to sentences while seeing either visual noise that was previously shown to strongly interfere with basic visual processing or a blank screen. Experiments 1 and 2 replicated the match effect but crucially visual noise did not modulate it. When an interference technique was used that targeted high-level semantic processing (Experiment 3) however the match effect vanished. Visual noise specifically targeting high-level visual processes (Experiment 4) only had a minimal effect on the match effect. We conclude that the shape match effect in the sentence-picture verification paradigm is unlikely to rely on perceptual simulation.
Individual differences in the verification of sentence--picture relationships
Journal of Verbal Learning and …, 1978
... VERBAL LEARNING AND VERBAL BEHAVIOR 17, 493--507 (1978) Individual Differences in the Verification of Sentence-Picture Relationships COLIN M. MACLEOD ... ABOVE STAR, and (b) verification time, the time to verify whether a picture of the form ,+ was true with respect ...
When participants follow spoken instructions to pick up and move objects in a visual workspace, their eye movements to the objects are closely time-locked to referential expressions in the instructions. Two experiments used this methodology to investigate the processing of the temporary ambiguities that arise because spoken language unfolds over time. Experiment 1 examined the processing of sentences with a temporarily ambiguous prepositional phrase (e.g., ''Put the apple on the towel in (M.J. Spivey).
Eye movements as a measure of syntactic and semantic incongruity in unambiguous sentences
2002
In one normative and one eye tracking experiment, we investigated the relationship between eye movements and linguistic anomaly detection. We manipulated semantic congruency and two types of syntactic congruency in unambiguous sentences. The three types of manipulations produced different patterns of effects in the first pass measures in the anomalous region. Syntactic congruency affected first pass reading time when the manipulation involved a phrasal category error (substituting a noun phrase for a prepositional phrase), but not when it involved a morphological feature error (subject-verb agreement). Both types of syntactic congruency influenced the likelihood of a first pass regressive eye movement. Semantic congruency did not influence first pass reading time, but it did affect the regression path duration. These findings support a model of language comprehension that marks some distinctions among the construction of syntactic structure, morphological feature-checking, and semantic processing. Furthermore, our findings suggest that we may need to reassess some garden path effects in the literature.
Cognitive Psychology, 2002
When participants follow spoken instructions to pick up and move objects in a visual workspace, their eye movements to the objects are closely time-locked to referential expressions in the instructions. Two experiments used this methodology to investigate the processing of the temporary ambiguities that arise because spoken language unfolds over time. Experiment 1 examined the processing of sentences with a temporarily ambiguous prepositional phrase (e.g., ''Put the apple on the towel in (M.J. Spivey).
EXAMINATION OF SENTENCE PROCESSING WITH CONTINUOUS VS. INTERRUPTED PRESENTATION PARADIGMS
This paper presents three studies which examine the susceptibility of sentence processing to intrusion by extra-sentential target words in two on-line dual-task techniques commonly used to study sentence processing: the cross-modal continuous-sentence (CMCS) paradigm and the unimodal visual sentencei nterruption (UVSI) paradigm. It provides a direct empirical examination of issues raised by (see also Nicol, Fodor, Swinney, 1994 with and about these techniques.
Pictorial encoding of sentences in sentence–-picture comparison
Quarterly Journal of Experimental Psychology
Subjects are shown to verbally encode sentences for simultaneous comparison with pictures but pictorially encode sentences for later comparison with pictures. This is taken as further evidence that subjects adapt their encodings or representations of stimuli to demands of the task.
Eye Movements and Spoken Language Comprehension
1996
When participants follow spoken instructions to pick up and move objects in a visual workspace, their eye movements to the objects are closely time-locked to referential expressions in the instructions. Two experiments used this methodology to investigate the processing of the temporary ambiguities that arise because spoken language unfolds over time. Experiment 1 examined the processing of sentences with a temporarily ambiguous prepositional phrase (e.g., ''Put the apple on the towel in (M.J. Spivey).