IM(S)2: Interactive movie summarization system (original) (raw)
Related papers
IM (S)< sup> 2 : Interactive movie summarization system
2010
The need of summarization methods and systems has become more and more crucial as the audio-visual material continues its critical growth. This paper presents a novel vision and a novel system for movies summarization. A video summary is an audio-visual document displaying the essential parts of an original document. However, the definition of the term ''essential" is user-dependent. The advantage of this work, unlike the others, is the involvement of users in the summarization process. By means of IM(S) 2 , people generate on the fly customized video summaries responding to their preferences. IM(S) 2 is made up of an offline part and an online part. In the offline, we segment the movies into shots and we compute features describing them. In the online part users inform about their preferences by selecting interesting shots. After that, the system will analyze the selected shots to bring out the user's preferences. Finally the system will generate a summary from the whole movie which will provide more focus on the user's preferences. To show the efficiency of IM(S) 2 , it was tested on the database of the European project MUSCLE made up of five movies. We invited 10 users to evaluate the usability of our system by generating for every movie of the database a semi-supervised summary and to judge at the end its quality. Obtained results are encouraging and show the merits of our approach.
What do you wish to see? A summarization system for movies based on user preferences
Information Processing & Management, 2015
Video summarization aims at producing a compact version of a full-length video while preserving the significant content of the original video. Movie summarization condenses a full-length movie into a summary that still retains the most significant and interesting content of the original movie. In the past, several movie summarization systems have been proposed to generate a movie summary based on low-level video features such as color, motion, texture, etc. However, a generic summary, which is common to everyone and is produced based only on low-level video features will not satisfy every user. As users' preferences for the summary differ vastly for the same movie, there is a need for a personalized movie summarization system nowadays. To address this demand, this paper proposes a novel system to generate semantically meaningful video summaries for the same movie, which are tailored to the preferences and interests of a user. For a given movie, shots and scenes are automatically detected and their high-level features are semi-automatically annotated. Preferences over high-level movie features are explicitly collected from the user using a query interface. The user preferences are generated by means of a stored-query. Movie summaries are generated at shot level and scene level, where shots or scenes are selected for summary skim based on the similarity measured between shots and scenes, and the user's preferences. The proposed movie summarization system is evaluated subjectively using a sample of 20 subjects with eight movies in the English language. The quality of the generated summaries is assessed by informativeness, enjoyability, relevance, and acceptance metrics and Quality of Perception measures. Further, the usability of the proposed summarization system is subjectively evaluated by conducting a questionnaire survey. The experimental results on the performance of the proposed movie summarization approach show the potential of the proposed system.
Automatic Summarization for Generic Audiovisual Content
Nowadays, with the explosion of multimedia content availability, the selectiveness of its consumption increases in importance. Audiovisual content is no longer brought to us only by the television, being available through many other systems, like Personal Video Recorders and Video on Demand systems, on each one's Personal Computer or, obviously, in the Internet. The exponential growth of websites like YouTube shows that people assign great relevance to audiovisual content in these days. Moreover, common people can easily produce, store, distribute and view those contents as it is not required any specialized skills to do so. To browse for a specific content in any of these systems can be a long, painful task. To manually summarize videos for the user's own purposes, as showing a summary of his/her vacations to friends and family also takes much time and it is a complex task. As people's time is getting more precious and scarce every day, an application capable of saving the time spent in these tasks by automatically summarizing audiovisual generic content arises as useful. Motivated by these factors, this report describes the developed solution for automatic audiovisual summarization for generic content, its motivations, the architecture as well as the process for summarization designed and implemented in the course of this work. To evaluate the quality of the created summaries, a user evaluation study was conducted with encouraging results, showing that the developed application is able, with relative success, to summarize audiovisual generic content.
Automatic and user-centric approaches to video summary evaluation
SPIE: Multimedia Content Access: …, 2007
Automatic video summarization has become an active research topic in content-based video processing. However, not much emphasis has been placed on developing rigorous summary evaluation methods and developing summarization systems based on a clear understanding of user needs, obtained through user centered design. In this paper we address these two topics and propose an automatic video summary evaluation algorithm adapted from teh text summarization domain.
A novel user-centered design for personalized video summarization
2014 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), 2014
In the past, several automatic video summarization systems had been proposed to generate video summary. However, a generic video summary that is generated based only on audio, visual and textual saliencies will not satisfy every user. This paper proposes a novel system for generating semantically meaningful personalized video summaries, which are tailored to the individual user's preferences over video semantics. Each video shot is represented using a semantic multinomial which is a vector of posterior semantic concept probabilities. The proposed system stitches video summary based on summary time span and top-ranked shots that are semantically relevant to the user's preferences. The proposed summarization system is evaluated using both quantitative and subjective evaluation metrics. The experimental results on the performance of the proposed video summarization system are encouraging.
VSUMM: A mechanism designed to produce static video summaries and a novel evaluation method
Pattern Recognition Letters, 2011
The fast evolution of digital video has brought many new multimedia applications and, as a consequence, has increased the amount of research into new technologies that aim at improving the effectiveness and efficiency of video acquisition, archiving, cataloging and indexing, as well as increasing the usability of stored videos. Among possible research areas, video summarization is an important topic that potentially enables faster browsing of large video collections and also more efficient content indexing and access. Essentially, this research area consists of automatically generating a short summary of a video, which can either be a static summary or a dynamic summary. In this paper, we present VSUMM, a methodology for the production of static video summaries. The method is based on color feature extraction from video frames and k-means clustering algorithm. As an additional contribution, we also develop a novel approach for the evaluation of video static summaries. In this evaluation methodology, video summaries are manually created by users. Then, several user-created summaries are compared both to our approach and also to a number of different techniques in the literature. Experimental results show -with a confidence level of 98% -that the proposed solution provided static video summaries with superior quality relative to the approaches to which it was compared.
Auto-summarization of audio-video presentations
1999
As streaming audio-video technology becomes widespread, there is a dramatic increase in the amount of multimedia content available on the net. Users face a new challenge: How to examine large amounts of multimedia content quickly. One technique that can enable quick overview of multimedia is video summaries; that is, a shorter version assembled by picking important segments from the original.
An extended framework for adaptive playback-based video summarization
Internet Multimedia Management Systems IV, 2003
In our previous work, we described an adaptive fast playback framework for video summarization where we changed the playback rate using the motion activity feature so as to maintain a constant pace. This method provides an effective way of skimming through video, especially when the motion is not too complex and the background is mostly still, such as in surveillance video. In this paper, we present an extended summarization framework that, in addition to motion activity, uses semantic cues such as face or skin color appearance, speech and music detection, or other domain dependent semantically significant events to control the playback rate. The semantic features we use are computationally inexpensive and can be computed in compressed domain, yet are robust, reliable, and have a wide range of applicability across different content types. The presented framework also allows for adaptive summaries based on preference, for example, to include more dramatic vs. action elements, or vice versa. The user can switch at any time between the skimming and the normal playback modes. The continuity of the video is preserved, and complete omission of segments that may be important to the user is avoided by using adaptive fast playback instead of skipping over long segments. The rule-set and the input parameters can be further modified to fit a certain domain or application. Our framework can be used by itself, or as a subsequent presentation stage for a summary produced by any other summarization technique that relies on generating a sub-set of the content.
2006
In this paper, we present a new approach for the automatic construction of video summaries. We introduce the Simulated User Principle to evaluate the quality of a video summary in a way which is automatic, yet related to user perception. We present experimental results to support our ideas.
A Proposed Methodology for Subjective Evaluation of Video and Text Summarization
Cryptology and Network Security, 2018
To evaluate a system that automatically summarizes video files (image and audio), it should be taken into account how the system works and which are the part of the process that should be evaluated, as two main topics to be evaluated can be differentiated: the video summary and the text summary. So, in the present article it is presented a complete way in order to evaluate this type of systems efficiently. With this objective, the authors have performed two types of evaluation: objective and subjective (the main focus of this paper). The objective evaluation is mainly done automatically, using established and proven metrics or frameworks, but it may need in some way the participation of humans, while the subjective evaluation is based directly on the opinion of people, who evaluate the system by answering a set of questions, which are then processed in order to obtain the targeted conclusions. The obtained general results from both evaluation systems will provide valuable information about the completeness and coherence, as well as the correctness of the generated summarizations from different points of view, as the lexical, semantical, etc. perspective. Apart from providing information about the state of the art, it will be presented an experimental proposal too, including the parameters of the experiment and the evaluation methods to be applied.