Jesús Calvillo Tinoco | Saarland University (original) (raw)

Uploads

Papers by Jesús Calvillo Tinoco

Cognitive Science, 2016

A novel connectionist model of sentence production is presented, which employs rich situation mod... more A novel connectionist model of sentence production is presented, which employs rich situation model representations originally proposed for modeling systematicity in comprehension (Frank, Haselager, & van Rooij, 2009). The high overall performance of our model demonstrates that such representations are not only suitable for comprehension, but also for modeling language production. Further, the model is able to produce novel encodings (active vs. passive) for a particular semantics, as well as generate such encodings for previously unseen situations, thus demonstrating both syntactic and semantic systematicity. Our results provide yet further evidence that such connectionist approaches can achieve systematicity, in production as well as comprehension.

Recent models of language have eliminated syntactic-semantic dividing lines. We explore the psych... more Recent models of language have eliminated syntactic-semantic dividing lines. We explore the psycholinguistic implications of this development by comparing different types of sentence embeddings in their ability to encode syntactic constructions. Our study uses contrasting sentence structures known to cause syntactic priming effects, that is, the tendency in humans to repeat sentence structures after recent exposure. We compare how syntactic alternatives are captured by sentence embeddings produced by a neural language model (BERT) or by the composition of word embeddings (BEAGLE, HHM, GloVe). Dative double object vs. prepositional object and active vs. passive sentences are separable in the high-dimensional space of the sentence embeddings and can be classified with a high degree of accuracy. The results lend empirical support to the modern, computational, integrated accounts of semantics and syntax, and they shed light on the information stored at different layers in deep language models such as BERT.

Information, Aug 16, 2021

This article is an open access article distributed under the terms and conditions of the Creative... more This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY

We present an analysis of the internal mechanism of the recurrent neural model of sentence produc... more We present an analysis of the internal mechanism of the recurrent neural model of sentence production presented by Calvillo et al. (2016). The results show clear patterns of computation related to each layer in the network allowing to infer an algorithmic account, where the semantics activates the semantically related words, then each word generated at each time step activates syntactic and semantic constraints on possible continuations, while the recurrence preserves information through time. We propose that such insights could generalize to other models with similar architecture, including some used in computational linguistics for language modeling, machine translation and image caption generation.

Cognitive Science, 2017

A model of sentence production is presented, which implements a strategy that produces sentences ... more A model of sentence production is presented, which implements a strategy that produces sentences with more uniform surprisal profiles, as compared to other strategies, and in accordance to the Uniform Information Density Hypothesis (Jaeger, 2006; Levy & Jaeger, 2007). The model operates at the algorithmic level combining information concerning word probabilities and sentence lengths, representing a first attempt to model UID as resulting from underlying factors during language production. The sentences produced by this model showed indeed the expected tendency, having more uniform surprisal profiles and lower average word surprisal, in comparison to other production strategies.

Die Entwicklungü berzeugender Argumentationi stebenso wie die Analysegegebener Argumentationsstru... more Die Entwicklungü berzeugender Argumentationi stebenso wie die Analysegegebener Argumentationsstruktureneine wichtige Aufgabeder Rechtswissenschaft.Die Formulierung rechtswissenschaftlicher Argumentationstellt eine anspruchsvolle intellektuelle Aufgabed ar,d ie sicha uf möglichst viele relevante Hintergrundinformationens tützen sollte.E iner ständigw achsendenA nzahl verfügbarer Gerichtsentscheidungen steht dabei die beschränkte menschliche Informationsverarbeitungskapazität gegenüber. Um diesen Problemen zu begegnen,w ird im Rahmen des vomBMBF-geförderten Konsortialprojektes ARGUMENTUM ein Software-Werkzeuge ntwickelt, das eine automatische Identifikationu nd Analyse vonA rgumentationsstrukturen in dene lektronischv erfügbaren Entscheidungen des Bundesverfassungsgerichts unterstützens oll. Im vorliegendenB eitrag werden das Konzept sowie die Architektur des ARGUMENTUM Software-Werkzeuges präsentiert underste Einblickeindie aktuelle Entwicklung des Prototypsgegeben.

This interactive poster session highlights findings from the first two years of the Teachers as L... more This interactive poster session highlights findings from the first two years of the Teachers as Learners initiative, sponsored by the James S. McDonnell Foundation. In 2018, ten research teams were funded to explore cognitive, sociocultural, and systemic dimensions of teachers learning to implement challenging instruction and classroom discourse in service of promoting students’ engagement and agency in the intellectual work of subject matter learning. The quintessential question these projects address is how teachers learn what they need to know and be able to do to create such contexts. Cross-cutting themes address contexts of professional learning, reflective practice, and iterative cycles of design, enactment, and re-design. ICLS 2020 Proceedings 2151 © ISLS

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020

Why do bilinguals switch languages within a sentence? The present observational study asks whethe... more Why do bilinguals switch languages within a sentence? The present observational study asks whether word surprisal and word entropy predict code-switching in bilingual written conversation. We describe and model a new dataset of Chinese-English text with 1476 clean code-switched sentences, translated back into Chinese. The model includes known control variables together with word surprisal and word entropy. We found that word surprisal, but not entropy, is a significant predictor that explains code-switching above and beyond other well-known predictors. We also found sentence length to be a significant predictor, which has been related to sentence complexity. We propose high cognitive effort as a reason for code-switching, as it leaves fewer resources for inhibition of the alternative language. We also corroborate previous findings, but this time using a computational model of surprisal, a new language pair, and doing so for written language.

Datenbank-Spektrum, 2015

ABSTRACT Die Entwicklung überzeugender Argumentation ist - ebenso wie die Analyse gegebener Argum... more ABSTRACT Die Entwicklung überzeugender Argumentation ist - ebenso wie die Analyse gegebener Argumentationsstrukturen - eine wichtige Aufgabe sowohl in der Rechtswissenschaft als auch in der juristischen Praxis. Beide Aufgaben gestalten sich intellektuell anspruchsvoll und sollten sich auf möglichst viele relevante Hintergrundinformationen stützen. Einer ständig wachsenden Anzahl verfügbarer Informationsquellen steht dabei die beschränkte menschliche Informationsverarbeitungskapazität gegenüber. Um diesen Problemen zu begegnen, wird im Rahmen des vom BMBF geförderten Konsortialprojektes ARGUMENTUM ein Software-Werkzeug entwickelt, das eine automatische Identifikation und Analyse von Argumentationsstrukturen in den elektronisch verfügbaren Entscheidungen des Bundesverfassungsgerichts unterstützen soll. Im vorliegenden Beitrag werden Konzept, Architektur und Implementierung des ARGUMENTUM-Werkzeuges präsentiert und Einblicke in mögliche Anwendungen gegeben.

Natural Language Processing and Cognitive Science

This dissertation approaches the task of modeling human sentence production from a connectionist ... more This dissertation approaches the task of modeling human sentence production from a connectionist point of view, and using distributed semantic representations. The main questions it tries to address are: (i) whether the distributed semantic representations defined by Frank et al. (2009) are suitable to model sentence production using artificial neural networks, (ii) the behavior and internal mechanism of a model that uses this representations and recurrent neural networks, and (iii) a mechanistic account of the

Cognitive Science, 2016

A novel connectionist model of sentence production is presented, which employs rich situation mod... more A novel connectionist model of sentence production is presented, which employs rich situation model representations originally proposed for modeling systematicity in comprehension (Frank, Haselager, & van Rooij, 2009). The high overall performance of our model demonstrates that such representations are not only suitable for comprehension, but also for modeling language production. Further, the model is able to produce novel encodings (active vs. passive) for a particular semantics, as well as generate such encodings for previously unseen situations, thus demonstrating both syntactic and semantic systematicity. Our results provide yet further evidence that such connectionist approaches can achieve systematicity, in production as well as comprehension.

Recent models of language have eliminated syntactic-semantic dividing lines. We explore the psych... more Recent models of language have eliminated syntactic-semantic dividing lines. We explore the psycholinguistic implications of this development by comparing different types of sentence embeddings in their ability to encode syntactic constructions. Our study uses contrasting sentence structures known to cause syntactic priming effects, that is, the tendency in humans to repeat sentence structures after recent exposure. We compare how syntactic alternatives are captured by sentence embeddings produced by a neural language model (BERT) or by the composition of word embeddings (BEAGLE, HHM, GloVe). Dative double object vs. prepositional object and active vs. passive sentences are separable in the high-dimensional space of the sentence embeddings and can be classified with a high degree of accuracy. The results lend empirical support to the modern, computational, integrated accounts of semantics and syntax, and they shed light on the information stored at different layers in deep language models such as BERT.

Information, Aug 16, 2021

This article is an open access article distributed under the terms and conditions of the Creative... more This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY

We present an analysis of the internal mechanism of the recurrent neural model of sentence produc... more We present an analysis of the internal mechanism of the recurrent neural model of sentence production presented by Calvillo et al. (2016). The results show clear patterns of computation related to each layer in the network allowing to infer an algorithmic account, where the semantics activates the semantically related words, then each word generated at each time step activates syntactic and semantic constraints on possible continuations, while the recurrence preserves information through time. We propose that such insights could generalize to other models with similar architecture, including some used in computational linguistics for language modeling, machine translation and image caption generation.

Cognitive Science, 2017

A model of sentence production is presented, which implements a strategy that produces sentences ... more A model of sentence production is presented, which implements a strategy that produces sentences with more uniform surprisal profiles, as compared to other strategies, and in accordance to the Uniform Information Density Hypothesis (Jaeger, 2006; Levy & Jaeger, 2007). The model operates at the algorithmic level combining information concerning word probabilities and sentence lengths, representing a first attempt to model UID as resulting from underlying factors during language production. The sentences produced by this model showed indeed the expected tendency, having more uniform surprisal profiles and lower average word surprisal, in comparison to other production strategies.

Die Entwicklungü berzeugender Argumentationi stebenso wie die Analysegegebener Argumentationsstru... more Die Entwicklungü berzeugender Argumentationi stebenso wie die Analysegegebener Argumentationsstruktureneine wichtige Aufgabeder Rechtswissenschaft.Die Formulierung rechtswissenschaftlicher Argumentationstellt eine anspruchsvolle intellektuelle Aufgabed ar,d ie sicha uf möglichst viele relevante Hintergrundinformationens tützen sollte.E iner ständigw achsendenA nzahl verfügbarer Gerichtsentscheidungen steht dabei die beschränkte menschliche Informationsverarbeitungskapazität gegenüber. Um diesen Problemen zu begegnen,w ird im Rahmen des vomBMBF-geförderten Konsortialprojektes ARGUMENTUM ein Software-Werkzeuge ntwickelt, das eine automatische Identifikationu nd Analyse vonA rgumentationsstrukturen in dene lektronischv erfügbaren Entscheidungen des Bundesverfassungsgerichts unterstützens oll. Im vorliegendenB eitrag werden das Konzept sowie die Architektur des ARGUMENTUM Software-Werkzeuges präsentiert underste Einblickeindie aktuelle Entwicklung des Prototypsgegeben.

This interactive poster session highlights findings from the first two years of the Teachers as L... more This interactive poster session highlights findings from the first two years of the Teachers as Learners initiative, sponsored by the James S. McDonnell Foundation. In 2018, ten research teams were funded to explore cognitive, sociocultural, and systemic dimensions of teachers learning to implement challenging instruction and classroom discourse in service of promoting students’ engagement and agency in the intellectual work of subject matter learning. The quintessential question these projects address is how teachers learn what they need to know and be able to do to create such contexts. Cross-cutting themes address contexts of professional learning, reflective practice, and iterative cycles of design, enactment, and re-design. ICLS 2020 Proceedings 2151 © ISLS

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020

Why do bilinguals switch languages within a sentence? The present observational study asks whethe... more Why do bilinguals switch languages within a sentence? The present observational study asks whether word surprisal and word entropy predict code-switching in bilingual written conversation. We describe and model a new dataset of Chinese-English text with 1476 clean code-switched sentences, translated back into Chinese. The model includes known control variables together with word surprisal and word entropy. We found that word surprisal, but not entropy, is a significant predictor that explains code-switching above and beyond other well-known predictors. We also found sentence length to be a significant predictor, which has been related to sentence complexity. We propose high cognitive effort as a reason for code-switching, as it leaves fewer resources for inhibition of the alternative language. We also corroborate previous findings, but this time using a computational model of surprisal, a new language pair, and doing so for written language.

Datenbank-Spektrum, 2015

ABSTRACT Die Entwicklung überzeugender Argumentation ist - ebenso wie die Analyse gegebener Argum... more ABSTRACT Die Entwicklung überzeugender Argumentation ist - ebenso wie die Analyse gegebener Argumentationsstrukturen - eine wichtige Aufgabe sowohl in der Rechtswissenschaft als auch in der juristischen Praxis. Beide Aufgaben gestalten sich intellektuell anspruchsvoll und sollten sich auf möglichst viele relevante Hintergrundinformationen stützen. Einer ständig wachsenden Anzahl verfügbarer Informationsquellen steht dabei die beschränkte menschliche Informationsverarbeitungskapazität gegenüber. Um diesen Problemen zu begegnen, wird im Rahmen des vom BMBF geförderten Konsortialprojektes ARGUMENTUM ein Software-Werkzeug entwickelt, das eine automatische Identifikation und Analyse von Argumentationsstrukturen in den elektronisch verfügbaren Entscheidungen des Bundesverfassungsgerichts unterstützen soll. Im vorliegenden Beitrag werden Konzept, Architektur und Implementierung des ARGUMENTUM-Werkzeuges präsentiert und Einblicke in mögliche Anwendungen gegeben.

Natural Language Processing and Cognitive Science

This dissertation approaches the task of modeling human sentence production from a connectionist ... more This dissertation approaches the task of modeling human sentence production from a connectionist point of view, and using distributed semantic representations. The main questions it tries to address are: (i) whether the distributed semantic representations defined by Frank et al. (2009) are suitable to model sentence production using artificial neural networks, (ii) the behavior and internal mechanism of a model that uses this representations and recurrent neural networks, and (iii) a mechanistic account of the