Query Interpretation-an Application of Semiotics in Image Retrieval (original) (raw)
Related papers
Applying Semantic Reasoning in Image Retrieval
2015
Abstract—With the growth of open sensor networks, multiple applications in different domains make use of a large amount of sensor data, resulting in an emerging need to search semantically over heterogeneous datasets. In semantic search, an important challenge consists of bridging the semantic gap between the high-level natural language query posed by the users and the low-level sensor data. In this paper, we show that state-of-the-art techniques in Semantic Modelling, Computer Vision and Human Media Interaction can be combined to apply semantic reasoning in the field of image retrieval. We propose a system, GOOSE, which is a general-purpose search engine that allows users to pose natural language queries to retrieve corresponding images. User queries are interpreted using the Stanford Parser, semantic rules and the Linked Open Data source ConceptNet. Interpreted queries are presented to the user as an intuitive and insightful graph in order to collect feedback that is used for furt...
International Journal of Computer Vision, 2004
image semantics resists all forms of modeling, very much like any kind of intelligence does. However, in order to develop more satisfying image navigation systems, we need tools to construct a semantic bridge between the user and the database. In this paper we present an image indexing scheme and a query language, which allow the user to introduce cognitive dimension to the search. At an abstract level, this approach consists of: (1) learning the "natural language" that humans speak to communicate their semantic experience of images, (2) understanding the relationships between this language and objective measurable image attributes, and then (3) developing corresponding feature extraction schemes. More precisely, we have conducted a number of subjective experiments in which we asked human subjects to group images, and then explain verbally why they did so. The results of this study indicated that a part of the abstraction involved in image interpretation is often driven by semantic categories, which can be broken into more tangible semantic entities, i.e. objective semantic indicators. By analyzing our experimental data, we have identified some candidate semantic categories (i.e. portraits, people, crowds, cityscapes, landscapes, etc.) and their underlying semantic indicators (i.e. skin, sky, water, object, etc.). These experiments also helped us derive important low-level image descriptors, accounting for our perception of these indicators. We have then used these findings to develop an image feature extraction and indexing scheme. In particular, our feature set has been carefully designed to match the way humans communicate image meaning. This led us to the development of a "semantic-friendly" query language for browsing and searching diverse collections of images. We have implemented our approach into an Internet search engine, and tested it on a large number of images. The results we obtained are very promising.
Utilising semantic technologies for intelligent indexing and retrieval of digital images
Computing, 2013
The proliferation of digital media has led to a huge interest in classifying and indexing media objects for generic search and usage. In particular, we are witnessing colossal growth in digital image repositories that are difficult to navigate using free-text search mechanisms, which often return inaccurate matches as they in principle rely on statistical analysis of query keyword recurrence in the image annotation or surrounding text. In this paper we present a semantically-enabled image annotation and retrieval engine that is designed to satisfy the requirements of the commercial image collections market in terms of both accuracy and efficiency of the retrieval process. Our search engine relies on methodically structured ontologies for image annotation, thus allowing for more intelligent reasoning about the image content and subsequently obtaining a more accurate set of results and a richer set of alternatives matchmaking the original query. We also show how our well-analysed and designed domain ontology contributes to the implicit expansion of user queries as well as the exploitation of lexical databases for explicit semantic-based query expansion.
Image and Its Semantic Role in Search Problem
2008
The world has now shrunk and information today exists in many forms ranging from text to videos. An overloaded World Wide Web, full of data makes it difficult to extract information from that data and this has given birth to a new phenomenon in the computer industry which is the search engine technology. Image that contains dense information has not yet found its real interpretation over search engines. In this paper we practice application of Semantic Web concepts and propose a standard dimension in image structures in order to improve searching ability in image search engines. An image annotation tool, called "SemImage", was developed which allows users to annotate an image with various ontologies and JPEG was taken as a case-study. This work describes our initial research efforts in semantics-based searching driven by ontological standards for images which we refer to as Image SemSearch.
Semantically Relevant Image Retrieval by Combining Image and Linguistic Analysis
Lecture Notes in Computer Science, 2006
In this paper, we introduce a novel approach to image-based information retrieval by combining image analysis with linguistic analysis of associated annotation information. While numerous Content Based Image Retrieval (CBIR) systems exist, most of them are constrained to use images as the only source of information. In contrast, recent research, especially in the area of web-search has also used techniques that rely purely on textual information associated with an image. The proposed research adopts a conceptually different philosophy. It utilizes the information at both the image and annotation level, if it detects a strong semantic coherence between them. Otherwise, depending on the quality of information available, either of the media is selected to execute the search. Semantic similarity is defined through the use of linguistic relationships in WordNet as well as through shape, texture, and color. Our investigations lead to results that are of significance in designing multimedia information retrieval systems. These include technical details on designing cross-media retrieval strategies as well as the conclusion that combining information modalities during retrieval not only leads to more semantically relevant performance but can also help capture highly complex issues such as the emergent semantics associated with images.
EVALUATION OF SEMANTIC SEARCH USING THE IMAGENOTION APPLICATION
2008
ABSTRACT Semantic search techniques have a big potential in exploring image archives as they provide better search results than traditional full-text search. Within our ImageNotion application, we develop and combine these techniques to improve end user experience by providing innovative query refinement and navigation features.
Using Linguistic Models for Image Retrieval
Lecture Notes in Computer Science, 2005
This research addresses the problem of image retrieval by exploring the semantic relationships that exist between image annotations. This is done by using linguistic relationships encoded in WordNet, a comprehensive lexical repository. Additionally, we propose the use of a reflective user-interface where users can interactively query-explore semantically related images by varying a simple parameter that does not require knowledge about the underlying information structure. This facilitates query-retrieval in context of the emergent nature of semantics that complex media, such as images have. Experiments show the efficacy and promise of this approach which can play a significant role in applications varying from multimedia information management to web-based image search.
An Integrative Semantic Framework for Image Annotation and Retrieval
IEEE/WIC/ACM International Conference on Web Intelligence (WI'07), 2007
Most public image retrieval engines utilise free-text search mechanisms, which often return inaccurate matches as they in principle rely on statistical analysis of query keyword recurrence in the image annotation or surrounding text. In this paper we present a semantically-enabled image annotation and retrieval engine that relies on methodically structured ontologies for image annotation, thus allowing for more intelligent reasoning about the image content and subsequently obtaining a more accurate set of results and a richer set of alternatives matchmaking the original query. Our semantic retrieval technology is designed to satisfy the requirements of the commercial image collections market in terms of both accuracy and efficiency of the retrieval process. We also present our efforts in further improving the recall of our retrieval technology by deploying an efficient query expansion technique.