The Language of Vision* (original) (raw)

PERCEPTION AND PRODUCTION IN A VISUALLY BASED LANGUAGE

Annals of the New York Academy of Sciences, 1975

Whether or not speech is special (as some would argue), it seems to us that there remain questions of great theoretical import concerning the relationship between speech and language; namely, What would language be like without speech? What properties of the complex phenomenon we call language-including its production and perception-are due to the mode of expression, i.e., due to the channel in which the more abstract entities are realized as physical, perceptible stuff, and what properties are due to some broader lingusitic faculty or to cognition in general? What sort of stamp does sound itself put on language? What would be the effect of other possible modes?

Objects and Nouns: An Account of the Vision-Language Interface

Establishing Cognitive Semiotics

This chapter tackles an important question that cognitive semiotics can answer about cross-modal relations among meanings, and the systems that produce them. The answer consists of an account of the relation between the cognitive systems underpinning object recognition in vision and NP interpretation in language. The analysis is based on the notion of “infomorphism”: two systems governed by the same underlying principles exchange information in regular ways. We show that, once we apply this account to classify the properties of each system, we can also model their possible connections. We discuss three phenomena (lexical variation, reference patterns, and semantic memory patterns) that can be modeled via this account.

Introduction to the special issue on language–vision interactions

Journal of Memory and Language, 2007

Researchers in psycholinguistics are increasingly interested in the question of how linguistic and visual information are integrated during language processing. In part, this trend is attributable to the use of the so-called ''visual world paradigm'' in psycholinguistics, in which participants look at and sometimes manipulate objects in a visual world as they listen to spoken utterances or generate utterances of their own. In this introductory article to the Special Issue on Language-Vision Interactions, we briefly describe the history of attempts to look at the integration of language and vision, and we preview the articles appearing in the special issue. From those articles, it is clear that recent work has dramatically expanded our understanding of this important question, a trend that will only accelerate as theoretical and methodological advances continue to be made. Ó 2007 Elsevier Inc. All rights reserved.

Commonality Between Verbal and Visual Language

2020

Islamic architecture is one of the most The term Visual Language is commonly used in the field of art and design to describe a form of communication that uses visual elements and/or symbols as opposed to formal written language to convey a meaning or an idea. The notion of visual language constantly represents one of the author major concerns, what does it mean and how to be understood, how to study and grasp its structure and grammar, what are the basic elements of this language and how it is used as a powerful method of expressing our emotions and our understanding of the world around us in a different way of communication. As a teacher and a practitioner of art and design, the author concern about visual language led him to investigate the relevance between visual language and verbal language suggesting that both languages have similar and common structures that would enable art and design students to understand the visual language more accurately in accordance to their understanding of the verbal one by applying the same structural concepts. This investigation, which took more than 10 years of research, practice and experimental teaching, resulted in developing a methodology that could implement the communality aspects between visual and verbal language in teaching art and design in the academic arena. This paper endeavours to address the advantages of using the visual language structural potentials in relation to the verbal language as a suggested methodology in teaching 2D design for the foundational stages in art and design institutions. The paper also showcases and analyses some students' works that demonstrate how they applied their understanding of visual language suggested methodology in developing their 2D designing skills.

Space and the Vision-Language Interface: A Model-Theoretic Approach

2011

The relation between spatial vision and spatial language has always been a source of controversy. Three problems can be identified as in need of a solution. A first problem pertains to the nature of the minimal information units that make up spatial vision and language. A second problem pertains to the ‘dynamic’ aspects of vision and language, or what visual information to and similar adpositions correspond to. A third problem pertains to how these different types of information are related one another, and what is the status of this ‘interface’, especially within a broader theory of language and cognition. The solution proposed here consists in a formal (model-theoretic) treatment of visual and linguistic information, both static and dynamic, that is couched within (a simplified form of) Discourse Representation Theory. It is shown that this solution is consistent with general theories of cognition and may shed some (novel) light on the nature of the FLN/FLB distinction.