Semantic Gap between People: An Experimental Investigation Based on Image Annotation (original) (raw)
Related papers
Image Tagging and Search – A Gender Oriented Study
Social computing sites constitute a valuable source of user generated content for user modeling. Whereas user generated content and the mining of such content are well studied, little attention has been given in the literature to modeling the relationship between users’ personal information and content. Here we analyze the relation of user gender to the choice of tags to describe a photo. A large user sample is examined to produce gender-related tagging vocabularies and tag representations. 1000 salient tags’ male and female representations are compared and results indicate that there are important differences between gender based term choices in a large majority of cases. To test the influence of gender on retrieval, we built a gender sensitive image search prototype and tested it, using a survey. Results show that around two thirds of participants tend to prefer image search results obtained using tag representations of their own gender and that a third of participants have a clear preference for their own gender’s results.
Image annotation: the effects of content, lexicon and annotation method
International Journal of Multimedia Information Retrieval, 2020
Image annotation is the process of assigning metadata to images, allowing effective retrieval by text-based search techniques. Despite the lots of efforts in automatic multimedia analysis, automatic semantic annotation of multimedia is still inefficient due to the problems in modelling high level semantic terms. In this paper we examine the factors affecting the quality of annotations collected through crowdsourcing platforms. An image dataset was manually annotated utilizing: (i) a vocabulary consists of pre-selected set of keywords, (ii) an hierarchical vocabulary, and (iii) free keywords. The results show that the annotation quality is affected by the image content itself and the used lexicon. As we expected while annotation using the hierarchical vocabulary is more representative, the use of free keywords leads to increased invalid annotation. Finally it is shown that images requiring annotations that are not directly related to their content (i.e. annotation using abstract concepts), lead to accrue annotator inconsistency revealing in that way the difficulty in annotating such kind of images is not limited to automatic annotation, but it is generic problem of annotation.
Semantic enrichment by non-experts: usability of manual annotation tools
2012
Most of the semantic content available has been generated automatically by using annotation services for existing content. Automatic annotation is not of sufficient quality to enable focused search and retrieval: either too many or too few terms are semantically annotated. User-defined semantic enrichment allows for a more targeted approach. We developed a tool for semantic annotation of digital documents and conducted an end-user study to evaluate its acceptance by and usability for non-expert users.
"Image tagging in Internet is becoming a crucial aspect in the search activity of many users all over the world, as online content evolves from being mainly text based, to being multi-media based (text, images, sound, …). In this paper we present a study carried out for native and non native English language taggers, with the objective of providing user support depending on the detected language skills and characteristics of the user. In order to do this, we analyze the differences between how users tag objectively (using what we call ‘see’ type tags) and subjectively (by what we call ‘evoke’ type tags). We study the data using bivariate correlation, visual inspection and rule induction. We find that the objective/subjective factors are discriminative for native/non native users and can be used to create a data model. This information can be utilized to help and support the user during the tagging process."
Analysis of user image descriptions and automatic image indexing vocabularies: An exploratory study
2004
Abstract This study explores the terms assigned by users to index, manage, and describe images and compares them to indexing terms derived automatically by systems for image retrieval. Results of this study indicate that userderived indexing vocabulary largely reflects what users see in the image or what they perceive as the overall topic of an image. This is in contrast to system-derived indexing wherein terms are extracted from existing text surrounding the image.
Lecture Notes in Computer Science, 2015
The practice of annotation is a secular and omnipresent activity. We find the annotation in several areas such as learning, semantic web, social networks, digital library, bioinformatics, etc. Thus, since the year 1989 and with the emergence of information technology, several annotation systems have been developed in human-computer environment adapted for various contexts and for various roles. These ubiquitous annotation systems allow users to annotate with digital information several electronic resources such as: web pages, text files, databases, images, videos, etc. Even though this topic has already been partially studied by other researchers, the previous works have left some open issues. It concern essentially the lack of how to organize all the developed annotation systems according to formal criteria in order to facilitate to the users the choice of an annotation system in a well-defined context and according to unified requirements. This problem is mainly due to the fact that annotation systems have only been developed for specific purposes. As a result, there is only a fragmentary picture of these annotation tools in the literature. The aim of this article is to provide a unified and integrated picture of all the annotation systems in human-computer environment. Therefore, we present a classification of sixty annotation tools developed by industry and academia during the last twenty-five years. This organization of annotation tools is built on the basis of five generic criteria. Observations and discussion of open issues conclude this survey.
Comparing the Language Used in Flickr, General Web Pages, Yahoo Images and Wikipedia
Words can be associated with images in different ways. Google and Yahoo use text found around a photo on a web page, Flickr image uploaders add their own tags. How do the annotations differ when they are extracted from text and when they are manually created? How does these language populations compare to written text? Here we continue our exploration of the differences in these languages.
Interaction Issues in Computer Aided Semantic Annotation of Multimedia
cs.bham.ac.uk
The CASAM project aims to provide a tool for more efficient and effective annotation of multimedia documents through collaboration between a user and a system performing an automated analysis of the media content. A critical part of the project is to develop a user interface which best supports both the user and the system through optimal human-computer interaction. In this paper we discuss the work undertaken, the proposed user interface and underlying interaction issues which drove its development.
2007
We describe a series of studies aimed at identifying specifications for a text extraction module of an image indexer’s toolkit. The materials used in the studies consist of images paired with paragraph sequences that describe the images. We administered a pilot survey to visual resource center professionals at three universities to determine what types of paragraphs would be preferred for metadata selection. Respondents generally showed a strong preference for one of two paragraphs they were presented with, indicating that not all paragraphs that describe images are seen as good sources of metadata. We developed a set of semantic category labels to assign to spans of text in order to distinguish between different types of information about the images, thus to classify metadata contexts. Human agreement on metadata is notoriously variable. In order to maximize agreement, we conducted four human labeling experiments using the seven semantic category labels we developed. A subset of ou...
The Assignment of Tags to Images in Internet: Language Skill Evaluation for Tag Recommendation
Users who tag images in Internet using the English language, an be native in that language or non native. Also, they can have different levels of tagging skills, in semantic terms (richness of vocabulary) and syntactic terms (errors incurred while defining the tags). If we can identify the ‘best’ taggers, we can use their work to help taggers whose skills are not so good. In this paper we present a study carried out for native and non native English language taggers, with the objective of providing user support depending on the detected language skills and characteristics of the user. In order to do this, we study the different types of syntactic errors that taggers commit, and analyze different semantic factors related to objective and subjective tagging, given that our hypothesis is that the latter is in general more difficult. We find that the syntactic and semantic factors together, allow us to profile users in terms of their skill level. This would allow us to keep the tag sessions of the best users and provide their tags to users who have a lower skill level.