Review: Automatic Image Annotation for Semantic Image Retrieval (original) (raw)

On the need for annotation-based image retrieval

2004

ABSTRACT Compared with content-based image retrieval, annotationbased image retrieval is more practical in some application domains. Users' information needs and the semantic contents of images can be represented by textual information more easily. We describe two problems which are unique to annotation-based image retrieval and would be worthy of further research. Contextual information embedded in data may be used to address these problems.

Automatic image annotation for semantic image retrieval

2007

This paper addresses the challenge of automatic annotation of images for semantic image retrieval. In this research, we aim to identify visual features that are suitable for semantic annotation tasks. We propose an image classification system that combines MPEG-7 visual descriptors and support vector machines. The system is applied to annotate cityscape and landscape images. For this task, our analysis shows that the colour structure and edge histogram descriptors perform best, compared to a wide range of MPEG-7 visual descriptors. On a dataset of 7200 landscape and cityscape images representing real-life varied quality and resolution, the MPEG-7 colour structure descriptor and edge histogram descriptor achieve a classification rate of 82.8% and 84.6%, respectively. By combining these two features, we are able to achieve a classification rate of 89.7%. Our results demonstrate that combining salient features can significantly improve classification of images.

Annotated Image search: Annotated Image Search using Text and Image Features

As the diversity and size of digital image collections grow exponentially, efficient image retrieval is becoming increasingly important. In general, current automatic image retrieval systems can be characterized into two categories: text-based and image content-based. For text-based image retrieval, the images are searched using the annotated text. In this framework, manual image annotation is extremely laborious and the visual content of images are difficult to be described precisely by a limited set of text terms. To overcome these difficulties, content-based image retrieval systems index images by their visual content, such as color, shape, texture, etc. It is a remarkable fact that, neither searching the images based on the content of the image nor searching the images using the annotated text may lead to an accurate result but jointly they tend to produce a perfect result; this is probably because the writers of text descriptions of images tend to leave out what is visually obvious (the color of flowers, etc.) and to mention properties that are very difficult to infer using vision (the species of the flower, say) and the content of the image depicts the description that may be left out by the writer. An efficient image retrieval system is highly desired. An algorithm which can combine both the retrieval systems i.e. Text Based and Content Based search and then filter out the common images can provide the exact solution for the underlying problem of the retrieval system. Our approach strives to implement the content based search by color and texture features of the objects present in the image using DWT, RGB color filter and color moments and text based search using the simple string match algorithm, and later using both results a similarity comparison is carried out to come up with a final result of the retrieval system. Our image extraction algorithm is based on both the content and the text based retrieval system with high recall rate.

Annotated Image search : Annotated Image Search using Text and Image Features Prof

2013

As the diversity and size of digital image collections grow exponentially, efficient image retrieval is becoming increasingly important. In general, current automatic image retrieval systems can be characterized into two categories: textbased and image content-based. For text-based image retrieval, the images are searched using the annotated text. In this framework, manual image annotation is extremely laborious and the visual content of images are difficult to be described precisely by a limited set of text terms. To overcome these difficulties, content-based image retrieval systems index images by their visual content, such as color, shape, texture, etc. It is a remarkable fact that, neither searching the images based on the content of the image nor searching the images using the annotated text may lead to an accurate result but jointly they tend to produce a perfect result; this is probably because the writers of text descriptions of images tend to leave out what is visually obvi...

Semantically Relevant Image Retrieval by Combining Image and Linguistic Analysis

Lecture Notes in Computer Science, 2006

In this paper, we introduce a novel approach to image-based information retrieval by combining image analysis with linguistic analysis of associated annotation information. While numerous Content Based Image Retrieval (CBIR) systems exist, most of them are constrained to use images as the only source of information. In contrast, recent research, especially in the area of web-search has also used techniques that rely purely on textual information associated with an image. The proposed research adopts a conceptually different philosophy. It utilizes the information at both the image and annotation level, if it detects a strong semantic coherence between them. Otherwise, depending on the quality of information available, either of the media is selected to execute the search. Semantic similarity is defined through the use of linguistic relationships in WordNet as well as through shape, texture, and color. Our investigations lead to results that are of significance in designing multimedia information retrieval systems. These include technical details on designing cross-media retrieval strategies as well as the conclusion that combining information modalities during retrieval not only leads to more semantically relevant performance but can also help capture highly complex issues such as the emergent semantics associated with images.

Semantic image retrieval and auto-annotation by converting keyword space to image space

Multi-Media Modelling Conference …, 2006

In this paper, we propose a novel strategy at an abstract level by combining textual and visual clustering results to retrieve images using semantic keywords and auto-annotate images based on similarity with existing keywords. Our main hypothesis is that images that fall in to the same text-cluster can be described with common visual features of those images. In order to implement this hypothesis, we set out to estimate the common visual features in the textually clustered images. When given an un-annotated image, we find the best image match in the different textual clusters by processing their low-level features. Experiments have demonstrated that good accuracy of proposal and its high potential use in annotation of images and for improvement of content based image retrieval.

An Integrative Semantic Framework for Image Annotation and Retrieval

IEEE/WIC/ACM International Conference on Web Intelligence (WI'07), 2007

Most public image retrieval engines utilise free-text search mechanisms, which often return inaccurate matches as they in principle rely on statistical analysis of query keyword recurrence in the image annotation or surrounding text. In this paper we present a semantically-enabled image annotation and retrieval engine that relies on methodically structured ontologies for image annotation, thus allowing for more intelligent reasoning about the image content and subsequently obtaining a more accurate set of results and a richer set of alternatives matchmaking the original query. Our semantic retrieval technology is designed to satisfy the requirements of the commercial image collections market in terms of both accuracy and efficiency of the retrieval process. We also present our efforts in further improving the recall of our retrieval technology by deploying an efficient query expansion technique.

Review: Automatic Semantic Image Annotation

INTERNATIONAL JOURNAL OF COMPUTERS & TECHNOLOGY, 2016

There are many approaches for automatic annotation in digital images. Nowadays digital photography is a common technology for capturing and archiving images because of the digital cameras and storage devices reasonable price. As amount of the digital images increase, the problem of annotating a specific image becomes a critical issue. Automated image annotation is creating a model capable of assigning terms to an image in order to describe its content. There are many image annotation techniques that seek to find the correlation between words and image features such as color, shape, and texture to provide an automatically correct annotation words to images which provides an alternative to the time consuming work of manual image annotation. This paper aims to cover a review on different Models (MT, CRM, CSD-Prop, SVD-COS and CSD-SVD) for automating the process of image annotation as an intermediate step in image retrieval process using Corel 5k images.

Image and annotation retrieval via image contents and tags

At present, tags are extensively used to describe the images, so that utilizing these tags for image retrieval system is today’s need. So, our system incorporates image similarity graph with image-tag bipartite graph using visual features of image like color, shape & texture .For color feature extraction HSV model, for shape feature extraction Sobel with mean and median filter, for texture feature extraction Framelet methods are used. Initially we extract features by these three methods and all features are combined for matching between images in the database and query image. Combination of the features of the three methods along with CBIR and TBIR are used to balance the influence between image content tags. The system is able to retrieve the images related to the query as well as annotating the query image.