Csaba Veres | University of Bergen (original) (raw)

Papers by Csaba Veres

IEEE Access

This paragraph of the first footnote will contain support information. Will fill in later. ABSTRA... more This paragraph of the first footnote will contain support information. Will fill in later. ABSTRACT Natural Language Processing (NLP) has become one of the leading application areas in the current Artificial Intelligence boom. Transfer learning has enabled large deep learning neural networks trained on the language modeling task to vastly improve performance in almost all language tasks. Interestingly, when the models are trained with data that includes software code, they demonstrate remarkable abilities in generating functioning computer code from natural language specifications. We argue that this creates a conundrum for claims that neural models provide an alternative theory to generative phrase structure grammars in explaining how language works. Since the acceptable syntax of programming languages is determined by phrase structure grammars, successful neural models are apparently uninformative about the theoretical foundations of programming languages, and by extension, natural languages. We argue that the term language model is misleading because deep learning models are not theoretical models of language and propose the adoption of corpus model instead, which better reflects the genesis and contents of the model. INDEX TERMS natural language processing, deep learning, syntax, linguistics, language model, automatic programming, neural networks 1 Chomsky describes his early work as an attempt to create a theoretical apparatus which was rich enough to describe the data. But it was always understood that the initial machinery had to be wrong because such a rich, complex system couldn't meet the criterion for biological evolution. The subsequent years were spent by reducing the complexity of the theoretical machinery https://youtu.be/pUWmTXkpHjE?t=3520 2 MIT150: Brains, Minds and Machines Symposium, June 16, 2011.

Lecture Notes in Computer Science, 2006

... An example showing this stability, as well as the approximate power law curve, and some evide... more ... An example showing this stability, as well as the approximate power law curve, and some evidence for cultural influence in terms of the community uptake of the term "Ajax" is shown in figure 1. (“Ajax” is represented ... But pity the poor soul who calls it 'the square root of a banana'! ...

ArXiv, 2021

Natural Language Processing (NLP) has become one of the leading application areas in the current ... more Natural Language Processing (NLP) has become one of the leading application areas in the current Artificial Intelligence boom. Transfer learning has enabled large deep learning neural networks trained on the language modeling task to vastly improve performance in almost all language tasks. Interestingly, when the models are trained with data that includes software code, they demonstrate remarkable abilities in generating functioning computer code from natural language specifications. We argue that this creates a conundrum for claims that neural models provide an alternative theory to generative phrase structure grammars in explaining how language works. Since the syntax of programming languages is determined by phrase structure grammars, successful neural models are apparently uninformative about the theoretical foundations of programming languages, and by extension, natural languages. We argue that the term language model is misleading because deep learning models are not theoretic...

I have spent the last few years of my life presenting papers on the ways in which cognitive scien... more I have spent the last few years of my life presenting papers on the ways in which cognitive science in general and linguistics in particular can inform research in Information Systems (IS). The works originate firmly in the “Chomskian ” 1 tradition of linguistics, contrary to the general theme of the workshop. But do we really need to take sides in the generative vs. cognitive linguistics debate? I suggest that even though one might prefer a view on theoretical grounds, both camps have valuable contributions to some areas of IS. To begin with, we note that there appears to be a mistaken assumption that, by subscribing to a Chomskian framework, one also subscribes to the notion that “formal semantics ” to do with quantification, negation and so on, is the only possible kind of semantics. That is, semantics as a cognitive fact is supposed not to exist. This is simply not true. Chomsky himself remarks “... if semantics is what is meant by the tradition (say, Peirce or Frege or somebody...

Heider and Simmel [Heider, F., Simmel, M., 1944. An experimental study of apparent behavior. Amer... more Heider and Simmel [Heider, F., Simmel, M., 1944. An experimental study of apparent behavior. American Journal of Psychology 57, 243-259] found that people spontaneously describe depictions of simple moving objects in terms of purposeful and intentional action. Not all intentional beings are objects, however, and people often attribute purposeful activity to non-object individuals such as countries, basketball teams, and families. This raises the question of whether the same effect found by Heider and Simmel would hold for non-object individuals such as groups. We replicate and extend the original study, using both objects and groups as stimuli, and introducing two control conditions with groups that are not engaged in structured movement. We found that under the condition that best promoted the attribution of intentionality, moving groups are viewed as purposeful and goal-directed entities to the same extent that moving objects are. These results suggest that the psychological distinction between the notion of 'intentional entity' and the notion of 'object' can be found even in the perception of moving geometrical figures.

Rights Copyright © is held by the author. Digital access to this material is made possible by the... more Rights Copyright © is held by the author. Digital access to this material is made possible by the University Libraries, University of Arizona. Further transmission, reproduction or presentation (such as public display or performance) of protected items is prohibited except with permission of the author. Downloaded 21-Feb-2016 04:01:53 Link to item

This paper discusses the concept ‘metadata’, and shows its importance in in-formation collection ... more This paper discusses the concept ‘metadata’, and shows its importance in in-formation collection and dissemination activities. We also show that the in-formation management components of maritime patrol and response man-date the effective use of metadata. We then propose an approach based on Semantic Technologies including the Resource Description Framework (RDF) and Upper Ontologies, for the implementation of metadata based dissemina-tion services for AIR 7000. A preliminary architecture is proposed. While the architecture is not yet operational, it highlights the challenges that need to be overcome in any solution to the information management tasks of AIR 7000, and provides a possible form for the solution.

AI 2019: Advances in Artificial Intelligence, 2019

Just over thirty years ago the prospect of modelling human knowledge with parallel distributed pr... more Just over thirty years ago the prospect of modelling human knowledge with parallel distributed processing systems without explicit rules, became a possibility. In the past five years we have seen remarkable progress with artificial neural network (ANN) based systems being able to solve previously difficult problems in many cognitive domains. With a focus on Natural Language Processing (NLP), we argue that the progress is in part illusory because the benchmarks that measure progress have become task oriented, and have lost sight of the goal to model knowledge. Task oriented benchmarks are not informative about the reasons machine learning succeeds, or fails. We propose a new dataset in which the correct answers to entailments and grammaticality judgements depend crucially on specific items of knowledge about verb semantics, and therefore errors on performance can be directly traced to deficiencies in knowledge. If this knowledge is not learnable from the provided input, then it must be provided as an innate prior.

There have been a growing number of publications suggesting that philosophical ontologies will de... more There have been a growing number of publications suggesting that philosophical ontologies will define a rigorous basis for conceptual modelling, particularly for data modelling methods and notations. An examination of an underlying psychological assumption of the conceptual modelling process is used to show that philosophical ontologies are being used as a ‘telescope’ to view the products of yet another ‘telescope’ and this undermines their reliability by being too far removed from the actual modelling process. An ontology of conceptual structure, derived through linguistic analysis provides a psychologically realistic alternative to the philosophical ontologies that is as close to its mental interpretation as possible and is a more promising approach to understanding the modelling process.

A primary contextual source for today’s context-sensitive mobile phone apps is the user’s locatio... more A primary contextual source for today’s context-sensitive mobile phone apps is the user’s location. The recent surge in the availability of open linked data can provide location-oriented semantic context, still wanting to be explored in innovative ways. In PediaCloud, the Android tool described here, we show how we can use the associative structure of the Semantic Web at a geographical location, visualize location information with tag clouds, and allow users to follow the associations of the Semantic Web enabled by the tag cloud, with the aim of enabling the users to construct an understanding of the “place” around them. The data we use are found through DBpedia, a project a project aimed to lift the information in WikiPedia into the Semantic Web.

MapXplore is an attempt to build a mainstream, useful, and easy to use application that uses link... more MapXplore is an attempt to build a mainstream, useful, and easy to use application that uses linked data at its core. As such, it will be one of very few such apps on the Apple, or for that matter any of the mobile app stores. The purpose of the application is to allow users to browse any part of the globe, and identify points of interest from DBPedia articles. They can then drill down into traditional as well as linked data sources to get a comprehensive view of the points of interest. We note some difficulties in working with current linked data resources, and suggest some methods to help pave a future rich in popular, easy to use mobile semantic applications. By using these guidelines we aim to keep refining MapXplore to make it a showcase application for the power of linked data.

AutoMind Creator is an iOS application that lets users interact with linked data to produce custo... more AutoMind Creator is an iOS application that lets users interact with linked data to produce customized views. These can be exported as graphical visualizations we call Semantic Mind Maps, as well as rich text (RTF), outline (OPML) and Freemind. We present a new technique for linked data visualisation called Semantic Mind Maps which are rich Mind Maps whose nodes are semantically grounded with a defining URI. The maps are essentially a compact knowledge representation format from which users can further explore information of interest. This paper describes the implementation of AutoMind, and highlights some particular pitfalls in programming for a commercial application with linked data, especially on the Apple ecosystem.

Free form tagging was one of the most useful contributions of "Web2.0" toward the probl... more Free form tagging was one of the most useful contributions of "Web2.0" toward the problem of content management and discovery on the web. Semantic tagging is a more recent but much less successful innovation borne of frustration at the limitations of free form tagging. In this paper we present LexiTags, a new platform designed to help realize the potential of semantic tagging for content management, and as a tool for crowdsourcing semantic metadata. We describe the operation of the LexiTags semantic bookmarking service, and present results from tools that exploit the semantic tags. These tools show that crowdsourcing can be used to model the taxonomy of an information space, and to semantically annotate resources within the space.

One of the most important early developments of the “Web2.0” era was social tagging. However, it ... more One of the most important early developments of the “Web2.0” era was social tagging. However, it soon became evident that tagging as practiced in the early days had significant problems which would limit its scalability. LexiTags was developed to help overcome some of these limitations. One key challenge is to disambiguate the possible word meanings such that the top option would almost always correspond to the one intended by the user in the given context. The work briefly reported here is an attempt to do just that. We conducted an experiment to measure the relative effectiveness of the algorithms for disambiguation in the sense selection task. Although correct disambiguation proved difficult, we have some indication that a specialized algorithm which can makeuse of the unique elements of context available in the Web Browser based context, might outperform standard disambiguation algorithms.

Finding good angles on news events is a central journalistic and editorial skill. As news work be... more Finding good angles on news events is a central journalistic and editorial skill. As news work becomes increasingly computer-assisted and big-data based, journalistic tools therefore need to become better able to support news angles too. This paper outlines a big-data platform that is able to suggest appropriate angles on news events to journalists. We first clarify and discuss the central characteristics of news angles. We then proceed to outline a big-data architecture that can propose news angles. Important areas for further work include: representing news angles formally; identifying interesting and unexpected angles on unfolding events; and designing a big-data architecture that works on a global scale.

This paper argues that data modeling for information systems cannot be divorced from human percep... more This paper argues that data modeling for information systems cannot be divorced from human perception, and is therefore marked by the subtle and often unconscious vagaries of cognition. In the absence of a formal semantics for modeling languages, this can result in models that are subjective, ambiguous, and difficult to interpret. Philosophical ontologies that provide a taxonomy of elements in the world have been proposed as a foundation to ground the symbols in various notational systems. Contrary to this view we show that models represent a designer's psychological perception of the world rather than some idealized, philosophical description of that world. A precise ontology of cognitive perceptions is therefore more relevant for the design of diagrammatic notations for use in documenting and unambiguously communicating the analysis of a domain. INTRODUCTION This paper reports an experiment which attempts to uncover some deep, universal principles of cognition that can have so...

AutoExtend is a method for learning unambiguous vector embeddings for word senses. We visualise t... more AutoExtend is a method for learning unambiguous vector embeddings for word senses. We visualise these word embeddings with t-SNE, which further compresses the vectors to the x,y plane. We show that the t-SNE co-ordinates can be used to reveal interesting semantic relations between word senses, and propose a new method that uses the simple x,y coordinates to compute semantic similarity. This can be used to propose new links and alterations to existing ones in WordNet. We plan to add this approach to the existing toolbox of methods in an attempt to understand learned semantic relations in word embeddings.

The schema.org initiative was designed to introduce machine readable metadata into the World Wide... more The schema.org initiative was designed to introduce machine readable metadata into the World Wide Web. This paper investigates conceptual biases in the schema through a mapping exercise between schema.org types and WordNet synsets. We create a mapping ontology which establishes the relationship between schema metadata types and the corresponding everyday concepts. This in turn can be used to enhance metadata annotation to include a more complete description of knowledge on the Web of data.

IEEE Access

This paragraph of the first footnote will contain support information. Will fill in later. ABSTRA... more This paragraph of the first footnote will contain support information. Will fill in later. ABSTRACT Natural Language Processing (NLP) has become one of the leading application areas in the current Artificial Intelligence boom. Transfer learning has enabled large deep learning neural networks trained on the language modeling task to vastly improve performance in almost all language tasks. Interestingly, when the models are trained with data that includes software code, they demonstrate remarkable abilities in generating functioning computer code from natural language specifications. We argue that this creates a conundrum for claims that neural models provide an alternative theory to generative phrase structure grammars in explaining how language works. Since the acceptable syntax of programming languages is determined by phrase structure grammars, successful neural models are apparently uninformative about the theoretical foundations of programming languages, and by extension, natural languages. We argue that the term language model is misleading because deep learning models are not theoretical models of language and propose the adoption of corpus model instead, which better reflects the genesis and contents of the model. INDEX TERMS natural language processing, deep learning, syntax, linguistics, language model, automatic programming, neural networks 1 Chomsky describes his early work as an attempt to create a theoretical apparatus which was rich enough to describe the data. But it was always understood that the initial machinery had to be wrong because such a rich, complex system couldn't meet the criterion for biological evolution. The subsequent years were spent by reducing the complexity of the theoretical machinery https://youtu.be/pUWmTXkpHjE?t=3520 2 MIT150: Brains, Minds and Machines Symposium, June 16, 2011.

Lecture Notes in Computer Science, 2006

... An example showing this stability, as well as the approximate power law curve, and some evide... more ... An example showing this stability, as well as the approximate power law curve, and some evidence for cultural influence in terms of the community uptake of the term "Ajax" is shown in figure 1. (“Ajax” is represented ... But pity the poor soul who calls it 'the square root of a banana'! ...

ArXiv, 2021

Natural Language Processing (NLP) has become one of the leading application areas in the current ... more Natural Language Processing (NLP) has become one of the leading application areas in the current Artificial Intelligence boom. Transfer learning has enabled large deep learning neural networks trained on the language modeling task to vastly improve performance in almost all language tasks. Interestingly, when the models are trained with data that includes software code, they demonstrate remarkable abilities in generating functioning computer code from natural language specifications. We argue that this creates a conundrum for claims that neural models provide an alternative theory to generative phrase structure grammars in explaining how language works. Since the syntax of programming languages is determined by phrase structure grammars, successful neural models are apparently uninformative about the theoretical foundations of programming languages, and by extension, natural languages. We argue that the term language model is misleading because deep learning models are not theoretic...

I have spent the last few years of my life presenting papers on the ways in which cognitive scien... more I have spent the last few years of my life presenting papers on the ways in which cognitive science in general and linguistics in particular can inform research in Information Systems (IS). The works originate firmly in the “Chomskian ” 1 tradition of linguistics, contrary to the general theme of the workshop. But do we really need to take sides in the generative vs. cognitive linguistics debate? I suggest that even though one might prefer a view on theoretical grounds, both camps have valuable contributions to some areas of IS. To begin with, we note that there appears to be a mistaken assumption that, by subscribing to a Chomskian framework, one also subscribes to the notion that “formal semantics ” to do with quantification, negation and so on, is the only possible kind of semantics. That is, semantics as a cognitive fact is supposed not to exist. This is simply not true. Chomsky himself remarks “... if semantics is what is meant by the tradition (say, Peirce or Frege or somebody...

Heider and Simmel [Heider, F., Simmel, M., 1944. An experimental study of apparent behavior. Amer... more Heider and Simmel [Heider, F., Simmel, M., 1944. An experimental study of apparent behavior. American Journal of Psychology 57, 243-259] found that people spontaneously describe depictions of simple moving objects in terms of purposeful and intentional action. Not all intentional beings are objects, however, and people often attribute purposeful activity to non-object individuals such as countries, basketball teams, and families. This raises the question of whether the same effect found by Heider and Simmel would hold for non-object individuals such as groups. We replicate and extend the original study, using both objects and groups as stimuli, and introducing two control conditions with groups that are not engaged in structured movement. We found that under the condition that best promoted the attribution of intentionality, moving groups are viewed as purposeful and goal-directed entities to the same extent that moving objects are. These results suggest that the psychological distinction between the notion of 'intentional entity' and the notion of 'object' can be found even in the perception of moving geometrical figures.

Rights Copyright © is held by the author. Digital access to this material is made possible by the... more Rights Copyright © is held by the author. Digital access to this material is made possible by the University Libraries, University of Arizona. Further transmission, reproduction or presentation (such as public display or performance) of protected items is prohibited except with permission of the author. Downloaded 21-Feb-2016 04:01:53 Link to item

This paper discusses the concept ‘metadata’, and shows its importance in in-formation collection ... more This paper discusses the concept ‘metadata’, and shows its importance in in-formation collection and dissemination activities. We also show that the in-formation management components of maritime patrol and response man-date the effective use of metadata. We then propose an approach based on Semantic Technologies including the Resource Description Framework (RDF) and Upper Ontologies, for the implementation of metadata based dissemina-tion services for AIR 7000. A preliminary architecture is proposed. While the architecture is not yet operational, it highlights the challenges that need to be overcome in any solution to the information management tasks of AIR 7000, and provides a possible form for the solution.

AI 2019: Advances in Artificial Intelligence, 2019

Just over thirty years ago the prospect of modelling human knowledge with parallel distributed pr... more Just over thirty years ago the prospect of modelling human knowledge with parallel distributed processing systems without explicit rules, became a possibility. In the past five years we have seen remarkable progress with artificial neural network (ANN) based systems being able to solve previously difficult problems in many cognitive domains. With a focus on Natural Language Processing (NLP), we argue that the progress is in part illusory because the benchmarks that measure progress have become task oriented, and have lost sight of the goal to model knowledge. Task oriented benchmarks are not informative about the reasons machine learning succeeds, or fails. We propose a new dataset in which the correct answers to entailments and grammaticality judgements depend crucially on specific items of knowledge about verb semantics, and therefore errors on performance can be directly traced to deficiencies in knowledge. If this knowledge is not learnable from the provided input, then it must be provided as an innate prior.

There have been a growing number of publications suggesting that philosophical ontologies will de... more There have been a growing number of publications suggesting that philosophical ontologies will define a rigorous basis for conceptual modelling, particularly for data modelling methods and notations. An examination of an underlying psychological assumption of the conceptual modelling process is used to show that philosophical ontologies are being used as a ‘telescope’ to view the products of yet another ‘telescope’ and this undermines their reliability by being too far removed from the actual modelling process. An ontology of conceptual structure, derived through linguistic analysis provides a psychologically realistic alternative to the philosophical ontologies that is as close to its mental interpretation as possible and is a more promising approach to understanding the modelling process.

A primary contextual source for today’s context-sensitive mobile phone apps is the user’s locatio... more A primary contextual source for today’s context-sensitive mobile phone apps is the user’s location. The recent surge in the availability of open linked data can provide location-oriented semantic context, still wanting to be explored in innovative ways. In PediaCloud, the Android tool described here, we show how we can use the associative structure of the Semantic Web at a geographical location, visualize location information with tag clouds, and allow users to follow the associations of the Semantic Web enabled by the tag cloud, with the aim of enabling the users to construct an understanding of the “place” around them. The data we use are found through DBpedia, a project a project aimed to lift the information in WikiPedia into the Semantic Web.

MapXplore is an attempt to build a mainstream, useful, and easy to use application that uses link... more MapXplore is an attempt to build a mainstream, useful, and easy to use application that uses linked data at its core. As such, it will be one of very few such apps on the Apple, or for that matter any of the mobile app stores. The purpose of the application is to allow users to browse any part of the globe, and identify points of interest from DBPedia articles. They can then drill down into traditional as well as linked data sources to get a comprehensive view of the points of interest. We note some difficulties in working with current linked data resources, and suggest some methods to help pave a future rich in popular, easy to use mobile semantic applications. By using these guidelines we aim to keep refining MapXplore to make it a showcase application for the power of linked data.

AutoMind Creator is an iOS application that lets users interact with linked data to produce custo... more AutoMind Creator is an iOS application that lets users interact with linked data to produce customized views. These can be exported as graphical visualizations we call Semantic Mind Maps, as well as rich text (RTF), outline (OPML) and Freemind. We present a new technique for linked data visualisation called Semantic Mind Maps which are rich Mind Maps whose nodes are semantically grounded with a defining URI. The maps are essentially a compact knowledge representation format from which users can further explore information of interest. This paper describes the implementation of AutoMind, and highlights some particular pitfalls in programming for a commercial application with linked data, especially on the Apple ecosystem.

Free form tagging was one of the most useful contributions of "Web2.0" toward the probl... more Free form tagging was one of the most useful contributions of "Web2.0" toward the problem of content management and discovery on the web. Semantic tagging is a more recent but much less successful innovation borne of frustration at the limitations of free form tagging. In this paper we present LexiTags, a new platform designed to help realize the potential of semantic tagging for content management, and as a tool for crowdsourcing semantic metadata. We describe the operation of the LexiTags semantic bookmarking service, and present results from tools that exploit the semantic tags. These tools show that crowdsourcing can be used to model the taxonomy of an information space, and to semantically annotate resources within the space.

One of the most important early developments of the “Web2.0” era was social tagging. However, it ... more One of the most important early developments of the “Web2.0” era was social tagging. However, it soon became evident that tagging as practiced in the early days had significant problems which would limit its scalability. LexiTags was developed to help overcome some of these limitations. One key challenge is to disambiguate the possible word meanings such that the top option would almost always correspond to the one intended by the user in the given context. The work briefly reported here is an attempt to do just that. We conducted an experiment to measure the relative effectiveness of the algorithms for disambiguation in the sense selection task. Although correct disambiguation proved difficult, we have some indication that a specialized algorithm which can makeuse of the unique elements of context available in the Web Browser based context, might outperform standard disambiguation algorithms.

Finding good angles on news events is a central journalistic and editorial skill. As news work be... more Finding good angles on news events is a central journalistic and editorial skill. As news work becomes increasingly computer-assisted and big-data based, journalistic tools therefore need to become better able to support news angles too. This paper outlines a big-data platform that is able to suggest appropriate angles on news events to journalists. We first clarify and discuss the central characteristics of news angles. We then proceed to outline a big-data architecture that can propose news angles. Important areas for further work include: representing news angles formally; identifying interesting and unexpected angles on unfolding events; and designing a big-data architecture that works on a global scale.

This paper argues that data modeling for information systems cannot be divorced from human percep... more This paper argues that data modeling for information systems cannot be divorced from human perception, and is therefore marked by the subtle and often unconscious vagaries of cognition. In the absence of a formal semantics for modeling languages, this can result in models that are subjective, ambiguous, and difficult to interpret. Philosophical ontologies that provide a taxonomy of elements in the world have been proposed as a foundation to ground the symbols in various notational systems. Contrary to this view we show that models represent a designer's psychological perception of the world rather than some idealized, philosophical description of that world. A precise ontology of cognitive perceptions is therefore more relevant for the design of diagrammatic notations for use in documenting and unambiguously communicating the analysis of a domain. INTRODUCTION This paper reports an experiment which attempts to uncover some deep, universal principles of cognition that can have so...

AutoExtend is a method for learning unambiguous vector embeddings for word senses. We visualise t... more AutoExtend is a method for learning unambiguous vector embeddings for word senses. We visualise these word embeddings with t-SNE, which further compresses the vectors to the x,y plane. We show that the t-SNE co-ordinates can be used to reveal interesting semantic relations between word senses, and propose a new method that uses the simple x,y coordinates to compute semantic similarity. This can be used to propose new links and alterations to existing ones in WordNet. We plan to add this approach to the existing toolbox of methods in an attempt to understand learned semantic relations in word embeddings.

The schema.org initiative was designed to introduce machine readable metadata into the World Wide... more The schema.org initiative was designed to introduce machine readable metadata into the World Wide Web. This paper investigates conceptual biases in the schema through a mapping exercise between schema.org types and WordNet synsets. We create a mapping ontology which establishes the relationship between schema metadata types and the corresponding everyday concepts. This in turn can be used to enhance metadata annotation to include a more complete description of knowledge on the Web of data.