Introducing the diagrammatic mode (original) (raw)

Introducing the diagrammatic semiotic mode

arXiv (Cornell University), 2020

As the use and diversity of diagrams across many disciplines grows, there is an increasing interest in the diagrams research community concerning how such diversity might be documented and explained. In this article, we argue that one way of achieving increased reliability, coverage, and utility for a general classification of diagrams is to draw on recently developed semiotic principles developed within the field of multimodality. To this end, we sketch out the internal details of what may tentatively be termed the diagrammatic semiotic mode. This provides a natural account of how diagrammatic representations may integrate natural language, various forms of graphics, diagrammatic elements such as arrows, lines and other expressive resources into coherent organisations, while still respecting the crucial diagrammatic contributions of visual organisation. We illustrate the proposed approach using two recent diagram corpora and show how a multimodal approach supports the empirical analysis of diagrammatic representations, especially in identifying diagrammatic constituents and describing their interrelations in a manner that may be generalised across diagram types and be used to characterise distinct kinds of functionality.

Cognitive Science Approaches To Understanding Diagrammatic Representations

Thinking with Diagrams, 2001

Through a wide variety of approaches cognitive science has given us various important insights into the nature of diagrammatic representations. This paper surveys the findings, issues and approaches to diagrammatic representations in cognitive science. Important current issues that are highlighted include: the relation between the parts of the representational system that are internal to the mind and in external visual media that presents the diagram; the use of multiple representations which is typical of real contexts of diagram use; the benefits of diagrams in terms of (i) computational offloading, (ii) re-representation and (iii) graphical constraining.

Using perceptual syntax to enhance semantic content in diagrams

IEEE Computer Graphics and Applications, 2001

Diagrams are essential in documenting large information systems. They capture, communicate and leverage knowledge indispensable for solving problems and are conceived to act as "cognitive externalizations". 1 A diagram provides a mapping from the problem domain to the visual representation by supporting cognitive processes that involve perceptual pattern finding and cognitive symbolic operations. 2 However not all mappings are equivalent, and to be effective a diagram's representation needs to be embedded with characteristics such that meaningful patterns can be easily perceived. Consequently a diagram's effectiveness depends, to some extent, on how well it is constructed as an input to our visual system. In our research, we focus on a class of diagrams commonly referred to as graphs or nodelink diagrams. Nodes representing entities, objects or processes, and links or edges representing relationships between the nodes characterize them. Their most common form is that of outline circles or boxes denoting nodes, and lines of different types representing links between the nodes. Entity-relationship diagrams, software structure diagrams and data flow models are examples of node-link diagrams used to model the structure of processes, software, or data.

On Diagrammatic Nature of Representation : Art and Design as Modeling Forms of Thinking

Visio. The Journal of the International Association for Visual Semiotics, 2003

By giving shape to our experiences we make sense of them. Making sense of something means to comprehend it. Such a comprehension is made possible by virtue of linking new information (=sign) with an old and familiar one. The process of linking, that is, a process of semiotic modeling requires connectivity and interconnectedness of our conceptualizations, which form thought patterns. That sense of interconnectedness ever present in the shaping and comprehension of experience leads to the examination of diagrams. This presentation is geared toward a discussion of relationships between diagrammatic reasoning, cognitive psychology, sensory perception, and the process of visual representation. For both an artist and a communication designer, it is essential to acknowledge, that changes in design affect the meaning in such a way, that it becomes a different meaning altogether. This presentation reveals the relationship between the meaning and the sensory characteristics of graphic diagram...

Learning to think and communicate with diagrams: 14 questions to consider

Artificial Intelligence Review, 2001

This paper looks at the particular role which diagrammatic representations, and external representations more generally, play within an educational context. In particular, it considers the way in which the demands on diagrammatic representational systems in educational settings differ with respect to other settings (e.g. professional): in some instances, these demands are increased, while in others, the demands are markedly different.

Seeing the language: a diagrammatic approach to natural discourse

The key idea behind the diagrammatic approach presented in the paper is that the sophisticated mechanisms of human visual construction also play an important role in natural languages. We propose a diagrammatic representation of English, giving examples, translation rules, and semantics. Special attention will be paid to anaphoric phenomena, in particular, the possibility of a uniform treatment of anaphoric pronouns.

The "Diagram" as the Audio-Visual Image

2019

In this research, we will examine another means of "information visualization" and "mapping of information" by use of philosophical concepts. Firstly, by referring to Michel Foucault's This is not a pipe, we will propose the "diagram" is a conceptual entity which separates and re-combines seeing and saying. The corresponding conditions of seeing and saying within several eras and periods were his main focus. Secondly, we shall examine how "information" has forced upon us corresponding ways of seeing and saying, but also ways of thinking and behaving. According to Gilles Deleuze, we need to find the "counter-information" that separates and re-combines the audiovisual image. This type of information will be found in a few of the media arts and visualization techniques employed by them. Finally, we will clarify the relations between science-philosophy-art, and argue that they make it possible to conceive of other ways of understanding our thoughts and bodies. Keywords-component; diagram; cartography; audiovisual image; seeing and saying; thought and body I. INTRODUCTION "Information visualization", "mapping of information", and "infographics" are useful tools for us today. They structure and clarify information, data, and thereby produce knowledge. Nowadays, we are always connected to the internet via smartphones and tablet computers. We will continue to more frequently in our ordinary lives because they help us to obtain and process information on the internet. They make it possible to easily understand complex and difficult information. Explanations reliant only on language can generate confusion, and likewise visuals without a caption are not necessarily clear. The visual and language must help assist one another. "Information visualization" and "mapping of information" functions via the corresponding ways of seeing and saying. If they do not correspond with each other, it will generate confusion in our thinking, and causes rotation in the actions or behavior of the body. However, their function renders data and knowledge merely what is said by someone, and also the sight as the what is simply seen by someone (i.e., the visualization is a point of view by the maker). We will search for the conditions corresponding to seeing and saying. What kind of "Information visualization" and "mapping of information" make possible another way of seeing and saying? We aim to explicate the virtual power of them.

Learning to think and communicate using diagrams: 14 questions to consider.

This paper looks at the particular role which diagrammatic representations, and external representations more generally, play within an educational context. In particular, it considers the way in which the demands on diagrammatic representational systems in educational settings differ with respect to other settings (e.g. professional): in some instances, these demands are increased, while in others, the demands are markedly different. The paper considers three key issues: the question of whether diagrams make certain tasks easier (and whether this is desirable from an educational point of view), the generalisation and transfer of diagrammatic skills once learnt, and the possible problems associated with simultaneously learning domain knowledge and a novel representational system. The paper then considers a number of sub-issues, and concludes by highlighting areas of particular interest for future AI research.