Qualitative Monitoring of Video Quality of Experience (original) (raw)
Related papers
Estimation of Subjective Video Quality as Feedback to Content Providers
2010 Fifth International Conference on Systems and Networks Communications, 2010
Concerning video transmission on the Internet, we present a model for estimating the subjective quality from objective measurements at the transmission receivers and on the network. The model reflects the quality degradation subject to parameters like packet loss ratio and bit rate and is calibrated using the results from subjective quality assessments. Besides the model and the calibration, the main achievement of this paper is the model's validation by implementation in a monitoring tool. It can be used by content and network providers to help swiftly localise the causes of a possibly poor quality of experience (QoE). It also can help content providers make decisions regarding the adjustment of vital parameters, such as bit rate and other error correction mechanisms.
Automatic quality of experience measuring on video delivering networks
ACM SIGMETRICS Performance Evaluation Review, 2008
This article describes a full video delivery network monitoring suite. Our monitoring tool offers a new view of a video delivery network, based on the quality as perceived by final users (what is nowadays called Quality of Experience, in short QoE). We measure the perceived quality at the client side by means of the recently proposed PSQA technology, by studying the video flows at the frame level. The developed monitoring suite is a completely free-software application, based on well-known technologies such as Simple Network Management Protocol or Round Robin Databases, which can be executed in various operating systems. In this short article we explain the tool implementation and we present some of the measurements performed with it.
A Proposed Measurement for Video Quality of Experience
Al-Nahrain journal of science, 2019
Technological development in the last years leads to increase the access speed in the networks that allow a huge number of users watching videos online. The Quality of Experience (QoE) Knowledge of services that provide from the network is a very critical matter to have a strong design of multimedia streaming networks. This paper provides a video streaming QoE prediction metric that does not require any information on the reference video. The proposed system extract numbers of features from videos that used to train the neural network and finally prediction the QoE value. Verify models prediction using 10-fold cross-validation that in a regular way split dataset (training set and test set) with multiple percentages. The proposed system verifies the best result.
Estimations and Remedies for Quality of Experience in Multimedia Streaming
Advances in Human- …
Managing multimedia network services in a Usercentric manner provides for more delivered quality to the users, whilst maintaining a limited footprint on the network resources. For efficient User-centric management it is imperative to have a precise metric for perceived quality. Quality of Experience (QoE) is such a metric, which captures many different aspects that compose the perception of quality. The drawback of using QoE is that due to its subjectiveness, accurate measurement necessitates execution of cumbersome subjective studies. In this work we propose a method that uses Machine Learning techniques to build QoE prediction models based on limited subjective data. Using those models we have developed an algorithm that generates the remedies for improving the QoE of observed multimedia stream. Selecting the optimal remedy is done by comparing the costs in resources associated to each of them. Coupling the QoE estimation and calculation of remedies produces a tool for effective implementation of a User-centric management loop for multimedia streaming services.
Computer Communications, 2017
The adoption of smart phones, the increased access to mobile broadband networks and the availability of public cloud infrastructures are aligning to the next generation of truly ubiquitous multimedia services, known as Cloud Mobile Media (CMM) services offering mobile video. Nevertheless, due to an inherit higher and variable end to end delay mainly as a result of the virtualization process, new challenges appear. One challenge is given by live video streaming applications when trying to keep a good Quality of Experience of the delivered video, measured in terms of a subjective video quality metric, named Mean Opinion Score (MOS). Our goal is to estimate and predict this subjective metric in a holistic manner using different estimation techniques, such as Artificial Neural Networks, Factor Analysis and Multinomial Linear Regression, with Full Reference and Non Reference approaches. For this, we have analyzed and measured different variables related to Quality of Service, bit stream and basic video quality metrics, throughout the CMM infrastructure. With these variables, we apply the mentioned techniques which allows us to estimate MOS of the delivered video in a robust and reliable way, achieving an average estimation error between 0.46 and 15.94% depending on the technique used. The real MOS has been evaluated through surveys. Finally, we compare the accuracy of the estimated MOS against well known publicly available video quality algorithms, following the recommendations given by Video Quality Experts Group.
Subjective QoE assessment on video service: Laboratory controllable approach
2017
This paper introduces research that addresses the subjective assessment of Quality of Experience (QoE) during the entire life cycle of a video session. We define a video session life cycle as the time from when a user attempts to initiate playback, until such time that the video ends either from normal video conclusion or through a network-induced failure. We provide a detailed description of our assessment methodology designed to discern whether a user's QoE would be impacted by the presence of failures. To accomplish this, we carefully select various test conditions to take into consideration the rating scale used, the types of impairments and failures seen by the user, and whether impaired videos are seen together with failed videos in multi-video sessions. The selection and creation of source video sequences are also discussed, as well as the use of between-subjects and within-subjects approaches for running our experiments in a controlled laboratory setting. Statistical analysis was carried out to interpret our experimental results. We compared the results of the between-subjects measures and the results of the within-subjects measures, and concluded that the introduction of a scale with an extended lower bound enabled subjects to more clearly express their dissatisfaction of videos with failures when compared to the traditional ITU 5-point rating scale. In addition, we observed that videos that were simply impaired but concluded normally did not have a statistically significant difference when an extended scale was used.
An Open Source Platform for Perceived Video Quality Evaluation
Proceedings of the 11th ACM Symposium on QoS and Security for Wireless and Mobile Networks, 2015
To ensure the best multimedia service quality in order to well address users' expectations, a new concept named Quality of Experience (QoE) has appeared. Two methods can be used to evaluate the user satisfaction, a subjective one and an objective one. The subjective approach is based on measured real data. The problem is there is no dataset large enough and can be used to well evaluate the QoE. In this, work we present our approach to build a data set for subjective evaluation based on a categorization approach and open source software.
Quality of Experience Models for Multimedia Streaming
International Journal of …, 2010
Understanding how quality is perceived by the viewers of multimedia streaming services is essential for their management. Quality of Experience (QoE) is a subjective metric that quantifies the perceived quality and therefore crucial in the process of optimizing the tradeoff between quality and resources. But accurate estimation of QoE usually entails cumbersome subjective studies that are long and expensive to execute. This paper presents a QoE estimation methodology for developing Machine Learning prediction models based on initial restricted-size subjective tests. Experimental results on subjective data from streaming multimedia tests show that the Machine Learning models outperform other statistical methods achieving accuracy greater than 90%. These models are suitable for real-time use due to their small computational complexity. Even though they have high accuracy, these models are static and cannot adapt to changes in the environment. To maintain the accuracy of the prediction models we have adopted Online Learning techniques that update the models on data from subjective viewer feedback. Overall this method provides accurate and adaptive QoE prediction models that can become indispensible component of a QoE-aware management service.
Realization of subjective tests in the environment of streaming services
2011
Increasing requirements on video quality seem to be essential while designing any video-oriented services. The methods in the user-centered design of services are fairly labor intensive and have to consider resulting value of user experience. However, user experience is a term that is currently very hard to be defined. There are different approaches to user experience assessment, which lack an ultimate method to predict expected user experience. In this article, we introduce a system that enables web service providers to measure quality of service provided to end-users while playing online video content that is approached via http progressive streaming. This tool is also suitable for future educational purposes in the field of video quality evaluation.