Semantic Representation of Multimedia Content - Knowledge Representation and Semantic Indexing (original) (raw)
Related papers
Towards Semantic Multimedia Indexing by Classification & Reasoning on Textual Metadata
2007
The task of multimedia document categorization forms a well-known problem in information retrieval. The task is to assign a multimedia document to one or more categories, based on its contents. In this case, effective management and thematic categorization requires usually the extraction of the underlying semantics. The proposed approach utilizes as input, analyzes and exploits the textual annotation that accompanies a multimedia document, in order to extract its underlying semantics, construct a semantic index and finally classify the documents to thematic categories. This process is based on a unified knowledge and semantics representation model introduced, as well as basic principles of fuzzy relational algebra. On top of that the fuzzy extension of expressive description logic language SHIN , f-SHIN and its reasoning services are used to further refine and optimize the initial categorization results. The proposed approach was tested on a set of real-life multimedia documents, derived from the Internet 1 , as well as personal databases and shows rather promising results.
An Extensible Platform for Semantic Classification And Retrieval of Multimedia Resources
Swap, 2006
This paper introduces a possible solution to the problem of semantic indexing, searching and retrieving heterogeneous resources, from textual as in most of modern search engines, to multimedia. The idea of "anchor" as information unit is here introduced to view resources from different perspectives and to access existing resources and metadata archives. Moreover, the platform uses an ontology as a conceptual representation of a well-defined domain in order to semantically classify and retrieve anchors (and the related resources). Specifically, the architecture of the proposed platform aims at being as modular and easily extensible as possible, in order to permit the inclusion of state-of-the-art techniques for the classification and retrieval of multimedia resources. Eventually, the adoption of Web Services as interface technology facilitates the exposition of the semantic functionalities and of content management to web application designers and users without any additional overload on the content creation and maintenance workflow.
Soft Computing in …, 2006
The semantic gap is the main problem of content based multimedia retrieval. This refers to the extraction of the semantic content of multimedia documents, the understanding of user information needs and requests, as well as to the matching between the two. In this chapter we focus on the analysis of multimedia documents for the extraction of their semantic content. Our approach is based on fuzzy algebra, as well as fuzzy ontological information. We start by outlining the methodologies that may lead to the creation of a semantic index; these methodologies are integrated in a video annotating environment. Based on the semantic index, we then explain how multimedia content may be analyzed for the extraction of semantic information in the form of thematic categorization. The latter relies on stored knowledge and a fuzzy hierarchical clustering algorithm that uses a similarity measure that is based on the notion of context.
A survey of semantic multimedia retrieval systems
2011
A growing number of research approaches are focusing on combining multimedia retrieval processing with semantics and knowledge based methods in order to achieve higher-level understanding of multimedia content. This research direction, often called semantic multimedia, combines techniques such as low-level multimedia feature extraction and common semantic representation schemes for features and concepts, thus making possible to manage query based on semantics that is a way for better supporting end user searches and result visualization in multimedia retrieval. Since low-level representations of media greatly differ from the higher level concepts associated with them, understanding the semantics of a query required a further insight in multimedia retrieval to bridge the semantic gap. In this paper we review the state-of-the-art techniques in semantic multimedia retrieval by discussing how relevant multimedia retrieval systems incorporate a semantic layer to improve the system perfor...
Representation of user preferences and adaptation to context in multimedia content–based retrieval
Proceedings of the Workshop on Multimedia Semantics, SOFSEM, 2002
The task of content -based retrieval is to provide users with the multimedia documents that best match their wishes. This process is not free of uncertainty; the role of the user profile is to remove a part of this uncertainty, using the information it contains concerning the user's preferences, and thus improve the precision. For this aim MPEG-7 introduced description schemes representing users' preferences and tools supporting user interaction. In this paper, using a formal representation of user preferences and a semantic knowledge base, we expand the user profile representation introduced in MPEG-7 in order to acquire more semantic information about the user. Furthermore, using the notion of context, we identify the part of the profile that is related to the user's query, thus preventing irrelevant interests from affecting the process of content -based retrieval.
Squiggle: a semantic search engine for indexing and retrieval of multimedia content
2006
Search engines are becoming such an easy way to find textual resources that we wish to use them also for multimedia content; however, syntactic techniques, even if promising, are not up to the task: future search engines must consider new approaches. Experimental prototypes of this search engine of the future are appearing. Most of them employs "smart machines" able to directly elaborate multimedia resources, but we believe that the solution should embrace also "smart data", able to capture lexical and conceptual characteristics of a domain in an ontology. In order to prove that Semantic Web technologies provide real benefits to end users in terms of an easier and more effective access to information, we developed ËÕÙ Ð , a Semantic Web framework that eases the deployment of semantic search engines. Following a model-driven approach to application development, ËÕÙ Ð makes ontologies (both the SKOS model and the domain knowledge) part of the running code. We evaluate the advantages of ËÕÙ Ð against traditional approaches in two real world deployments: one to search images of skiers for Torino 2006 Winter Olympic Games and one to search music files.
2012
2 Abstract: With the emergence of the low cost multimedia enable devices and storage the progression of the multimedia increases at an incredible velocity. These advancements in the technology have culminated with the immense amount of data. Effective utilization of this colossal data requires a need for the effective and intelligent system and technique to store, annotate, manage, search and retrieve. Semantically analyzing and interpreting the multimedia data is an open challenge. Multimedia analysis starts from the text based system to the content based and finally to the semantic based retrieval systems.
Using ontologies and fuzzy relations in multimedia personalization
Semantic Media Adaptation and …, 2006
In this paper we extend on previous work in order to address the problem of multimedia personalization at a semantic level. In different previous works we have developed algorithms to address computationally efficient handling of large but sparse fuzzy relations, and theory to address knowledge representation, thematic categorization and user modeling. In this work we take two further steps: i) we integrate ontologies in our original knowledge modeling approach and ii) we explain how these diverse algorithms and methodologies can be combined in order to approach a greater goal, that of semantic multimedia personalization.
A media agent for automatically building a personalized semantic index of Web media objects
Journal of the American Society for Information Science and Technology, 2001
A novel idea of media agent is briefly presented, which can automatically build a personalized semantic index of Web media objects for each particular user. Because the Web is a rich source of multimedia data and the text content on the Web pages is usually semantically related to those media objects on the same pages, the media agent can automatically collect the URLs and related text, and then build the index of the multimedia data, on behalf of the user whenever and wherever she accesses these multimedia data or their container Web pages. Moreover, the media agent can also use an off-line crawler to build the index for those multimedia objects that are relevant to the user's favorites but have not accessed by the user yet. When the user wants to find these multimedia data once again, the semantic index facilitates text-based search for her.