A Survey of Semantic Image and Video Annotation Tools (original) (raw)

Using a Multimedia Ontology Infrastructure for Semantic Annotation of Multimedia Content

2005

In this paper we discuss the use of knowledge for the automatic extraction of semantic metadata from multimedia content. For the representation of knowledge we extended and enriched current general-purpose ontologies to include low-level visual features. More specifically, we implemented a tool that links MPEG-7 visual descriptors to high-level, domain-specific concepts. For the exploitation of this knowledge infrastructure we developed an experimentation platform, that allows us to analyze multimedia content and automatically create the associated semantic metadata, as well as to test, validate and refine the ontologies built. We pursued a tight and functional integration of the knowledge base and the analysis modules putting them in a loop of constant interaction instead of being the one just a pre-or post-processing step of the other.

Semantic Annotations for Digital Video

2005

This paper describes a functioning system that associates semantic annotations (in the form of a table of triples of identifiers) with digital video. The first part describes a subsystem (in existence since 2002, desktop version 1995) of adding annotations to digital video. The second part builds on that subsystem, creating additional semantic annotations that enable semantic-web retrieval. The key component of the second subsystem is a glossary for the text being indexed, created by the user in interaction with Princeton's WordNet. The resulting glossary is a microformat within an XHTML document, which we validate using our microformat validator. Additional techniques for harvesting metadata from the domain expert are described.

Multimedia Annotation using Semantic Web Technologies

Due to the progressively increasing amount of multimedia on the Web, the need for efficient metadata formats describing that content has become increasingly evident. This paper gives an overview of the different approaches and methods for creation and retrieval of semantic rich multimedia metadata. Semantic web and its most important technologies XML, RDF and ontologies used for multimedia annotation are defined. An overview of various multimedia metadata vocabularies and formats that vary in their size and purpose is provided. Multimedia metadata is a type of metadata used for describing different aspects of multimedia content. All formats of multimedia metadata are not compatible with each other and most of it do not provide enough semantics. New Semantic Web technologies provide well-defined information meaning so different multimedia metadata can be more easily processed by computers.

Ontologies for Multimedia Annotation: An overview

In recent years, along with the expansion of Web 2.0 and social networks, an extreme growth of multimedia content on the Web is registered. That multimedia content is mostly in the form of images and videos. To enable enhanced use, reuse and retrieval of multimedia content from the Web, that content needs to be annotated. Several multimedia metadata standards and a number of vocabularies commonly used for annotating multimedia content exist today. Semantic Web technologies, like RDF and ontologies, provide well-defined meaning for the multimedia content, enabling better processing of their annotations by computers and applications. Formal language OWL, along with its sublanguages, is used for defining ontologies on the Semantic Web. In this paper a brief overview of ontologies in general and selected specialized multimedia ontologies that can be used for semantically rich multimedia annotation is presented.

Multimedia Annotations on the Semantic Web

IEEE Multimedia, 2006

Multimedia and the Semantic Web: in theory, it is a perfect match. The Semantic Web, on the one hand, provides a stack of languages and technologies for annotating Web resources, enabling machine processing of metadata describing semantics of web content. Multimedia applications, on the other hand, require metadata descriptions of their media items to facilitate search and retrieval, intelligent processing and effective presentation of multimedia information. This need for multimedia metadata was recognized by the media industry long ago. Semantic Web technologies, however, still play a very minor role within multimedia applications and most approaches employ non-RDF based techniques. This paper describes a number of current approaches to multimedia metadata and provides an inventory of the open issues to achieve a practical integration of multimedia metadata into the Semantic Web.

Semi-automatic Video Content Annotation

Lecture Notes in Computer Science, 2002

Video modeling and annotating are indispensable operations necessary for creating and populating a video database. To annotate video data effectively and accurately, a video content description ontology is first proposed in this paper, we then introduce a semi-automatic annotation strategy which utilize various video processing techniques to help the annotator explore video context or scenarios for annotation. Moreover, a video scene detection algorithm which joints visual and semantics is proposed to visualize and refine the annotation results. With the proposed strategy, a more reliable and efficient video content description could be achieved. It is better than manual manner in terms of efficiency, and better than automatic scheme in terms of accuracy.

Towards bridging the semantic gap in multimedia annotation and retrieval

2006

We present a systematic approach to the design of multimedia ontologies based on the MPEG-7 standard and domain-specific vocabularies. The limitations of MPEG-7 in describing the semantics of highly structured domains like sports or medicine has led to an upsurge of interest in adopting an integrated approach to the design of ontologies. We follow suit and use MPEG-7 to model structural and low-level aspects of multimedia documents. The high-level semantics are modeled using a domain-specific ontology designed for soccer games. The integration of these ontologies is achieved by providing appropriate links to the individual ontologies. As a proof-of-concept, we describe a video annotation tool implemented as a plugin for the widely used Protégé ontology editor. The advantage of our methodologies lies in the fact that we use semantic web compliant languages and tools that results in alleviating the interoperability issues currently plaguing the multimedia and the semantic web communities.

A framework for automatic semantic video annotation

Multimedia Tools and Applications, 2013

Your article is protected by copyright and all rights are held exclusively by Springer Science +Business Media New York. This e-offprint is for personal use only and shall not be selfarchived in electronic repositories. If you wish to self-archive your work, please use the accepted author's version for posting to your own website or your institution's repository. You may further deposit the accepted author's version on a funder's repository at a funder's request, provided it is not made publicly available until 12 months after publication.