Shengbin Meng - Academia.edu (original) (raw)

Papers by Shengbin Meng

2020 IEEE International Conference on Visual Communications and Image Processing (VCIP)

Short video applications, like TikTok, have attracted many users across the world. It can feed sh... more Short video applications, like TikTok, have attracted many users across the world. It can feed short videos based on users' preferences and allow users to slide the boring content anywhere and anytime. To reduce the loading time and keep playback smoothness, most of the short video apps will preload the recommended short videos in advance. However, these apps preload short videos in fixed size and fixed order, which can lead to huge playback stall and huge bandwidth waste. To deal with these problems, we present an Adaptive Preloading mechanism for short videos based on Lyapunov Optimization, also called APL, to achieve near-optimal playback experience, i.e., maximizing playback smoothness and minimizing bandwidth waste considering users' sliding behaviors. Specifically, we make three technical contributions: (1) We design a novel short video streaming framework which can dynamically preload the recommended short videos before the current video is downloaded completely. (2) We formulate the preloading problem into a playback experience optimization problem to maximize the playback smoothness and minimize the bandwidth waste. (3) We transform the playback experience optimization problem during the whole viewing process into a single-step greedy algorithm based on the Lyapunov optimization theory to make the online decisions during playback. Through extensive experiments based on the real datasets that generously provided by TikTok, we demonstrate that APL can reduce the stall ratio by 81%/12% and bandwidth waste by 11%/31% compared with no-preloading/fixed-preloading mechanism.

2020 IEEE International Conference on Visual Communications and Image Processing (VCIP)

On a platform of user-generated content (UGC), the uploaded videos need to be encoded again befor... more On a platform of user-generated content (UGC), the uploaded videos need to be encoded again before distribution. For this specific encoding scenario, we propose a novel dataset and a corresponding learning-based scheme that is able to achieve significant bitrate saving without decreasing perceptual quality. In the dataset, each video’s label indicates whether it can be encoded with a much lower bitrate while still keeps the same perceptual quality. Models trained on this dataset can then be used to classify the input video and adjust its final encoding parameters accordingly. With enough classification accuracy, more than 20% average bitrate saving can be obtained through the proposed scheme. The dataset will be further expanded to facilitate the study on this problem.

MultiMedia Modeling, 2022

2020 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), 2020

2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 2021

2016 IEEE International Conference on Multimedia and Expo (ICME), 2016

TCP-based application-layer protocols are increasingly applied to commercial live video streaming... more TCP-based application-layer protocols are increasingly applied to commercial live video streaming systems. However, in unstable mobile networks, the throughput of TCP may fluctuate rapidly due to its transmission mechanism, causing undesired playback interruption. In this paper, we propose a general application-layer rate adaptation approach to cope with the variability in TCP throughput. We analyze the transmission process and evaluate the network condition using a multi-buffer model. With information obtained in this model, an algorithm based on the Proportional-Integral-Derivative (PID) controller is proposed to dynamically adjust the video bitrate in response to the throughput change. We have implemented an experimental mobile live streaming system employing this approach and achieved a significant improvement in playback continuity and bandwidth utilization.

IEEE Transactions on Multimedia, 2021

2018 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 2018

Recently, the 360-degree video has become a hot topic in multimedia area. However, the requiremen... more Recently, the 360-degree video has become a hot topic in multimedia area. However, the requirements of high bitrate, low Internet interactive latency and high perceived quality limit its further applications. So in this paper, we design a tile- based QoE-driven HTTP/2 streaming system for 360 video. It uses a new projection method, Sinusoidal Projection (SP), to reduce the bitrate of tiles. A novel cross-user's behavior learning method is also used to predict viewpoint. To improve bandwidth utilization, it pushes multiple tiles in one request by HTTP/2. Besides, by using a QoE-driven framework, our approach can significantly improve users' perceived quality. The numerous experiment results have demonstrated the efficiency of the proposed system. Compared with the legacy methods, the transmission bitrate drops about 17%, the view- port prediction accuracy improves 30%, the Viewport-PSNR improve 22% and the transmission latency drops about 30%.

IEEE Transactions on Multimedia, 2016

To cope with the challenges brought about by bandwidth fluctuation and improve the experience of ... more To cope with the challenges brought about by bandwidth fluctuation and improve the experience of watching online videos, an adaptive video streaming system that can adjust video quality according to actual network conditions is proposed based on the scalable video coding (SVC) extension of H.264/AVC. First, a simple and effective linear error model is proposed and verified for quality scalability of SVC. The model exploits the linear feature of pixel value errors and can be used to accurately estimate the distortion caused by discarding any combination of enhancement data packets in an SVC bitstream. On that basis, a greedy-like algorithm is designed to assign each data packet a priority value according to its rate-distortion (R-D) impact, thus enabling R-D optimized bitstream extraction under certain bitrate constraints. Finally, the proportional-integral-derivative (PID) method is utilized to control the video quality adjustment and determine a suitable bitrate for transmission. By monitoring and predicting the past, current, and future bandwidth information, the PID-based quality control algorithm is able to reduce quality fluctuation, while still preserving a high quality level. Experimental results show that compared with the baseline software, the proposed system that integrates the above algorithms can achieve much lower video quality fluctuation, with PSNR variance reduced from 1.24 to 0.69, and at the same time deliver higher video quality, with the PSNR average increased by 0.83 dB.

2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2014

This paper investigates the no-reference SVC bitstream extraction problem and presents an efficie... more This paper investigates the no-reference SVC bitstream extraction problem and presents an efficient solution to approximate the “optimal” extracted sub-stream. First, we introduce a linear error model to accurately estimate the distortion caused by discarding any combination of packets, even when the original sequence is not available. Then we propose a greedy algorithm to decide each packet's priority according to its R-D impact. The priority value of packets can be stored in the bitstream and used for R-D optimized extraction. Experimental results show that our bitstream extraction method can achieve a significant PSNR gain compared to the extractors of JSVM, without computational complexity increment. Comparison with other methods also demonstrates the advantage of the proposed method.

2014 IEEE International Conference on Communications (ICC), 2014

Scalable Video Coding (SVC) makes it possible to change video quality dynamically according to re... more Scalable Video Coding (SVC) makes it possible to change video quality dynamically according to real-time bandwidth. For quality control algorithms of SVC video streaming, the biggest challenge is to keep a video quality that is both smooth and as good as possible. In this paper, we first introduce a combined quality level scheme to describe SVC video quality in a unified way. Then an effective and efficient quality control algorithm for SVC video streaming is proposed based on the Proportional-Integral-Derivative (PID) control method. Extensive experiments show that the proposed algorithm improves 8.6% in video quality with 24.8% reduction in quality fluctuation compared with the existing packet delay feedback algorithm. The proposed algorithm has also been implemented in online video website www.7dlive.com and performs well in applications.

2014 IEEE 16th International Workshop on Multimedia Signal Processing (MMSP), 2014

In this paper, we propose a novel design and optimized implementation of the HEVC decoder. First,... more In this paper, we propose a novel design and optimized implementation of the HEVC decoder. First, a novel decoder prototype with refined decoding workflow and efficient memory management is designed. Then on this basis, a series of single-instruction-multiple-data (SIMD) based algorithms are used to speed up several time-consuming modules in HEVC decoding. Finally, a frame-based parallel framework is applied to exploit the multi-threading technology on multicore processors. With the highly optimized HEVC decoder, decoding speed of 246fps on Intel i7-2400 3.4GHz quad-core processor for 1080p videos and 52fps on ARM Cortex-A9 1.2GHz dual-core processor for 720p videos can be achieved in our experiments.

Lecture Notes in Computer Science, 2015

In this demonstration, we showcase a complete software encoding and decoding solution for the new... more In this demonstration, we showcase a complete software encoding and decoding solution for the new High Efficiency Video Coding (HEVC) standard. The encoder is optimized for x86 processors using SSE instruction set extension and multi-thread technology, and achieves high efficiency at a significantly reduced computation load. We have integrated the encoder library into the widely-used media framework FFmpeg and developed transcoding and recording applications for HEVC. The decoder is highly optimized for both x86 and ARM architecture. With novel single-instruction-multiple-data (SIMD) algorithms and a frame-based parallel framework for multi-core CPUs, decoding speed of 46FPS for 1080p videos on ARM Cortex-A9 1.5GHz dual-core processor and 75FPS for 4K (3840x2160) videos on Intel i7-2600 3.4GHz quad-core processor can be achieved. We have also integrated the decoder library into FFmpeg and made an Android video player based on that. The software solution can well meet the demand of producing and watching HEVC videos on existing devices, showing promising future of HEVC applications.

2020 IEEE International Conference on Visual Communications and Image Processing (VCIP)

Short video applications, like TikTok, have attracted many users across the world. It can feed sh... more Short video applications, like TikTok, have attracted many users across the world. It can feed short videos based on users' preferences and allow users to slide the boring content anywhere and anytime. To reduce the loading time and keep playback smoothness, most of the short video apps will preload the recommended short videos in advance. However, these apps preload short videos in fixed size and fixed order, which can lead to huge playback stall and huge bandwidth waste. To deal with these problems, we present an Adaptive Preloading mechanism for short videos based on Lyapunov Optimization, also called APL, to achieve near-optimal playback experience, i.e., maximizing playback smoothness and minimizing bandwidth waste considering users' sliding behaviors. Specifically, we make three technical contributions: (1) We design a novel short video streaming framework which can dynamically preload the recommended short videos before the current video is downloaded completely. (2) We formulate the preloading problem into a playback experience optimization problem to maximize the playback smoothness and minimize the bandwidth waste. (3) We transform the playback experience optimization problem during the whole viewing process into a single-step greedy algorithm based on the Lyapunov optimization theory to make the online decisions during playback. Through extensive experiments based on the real datasets that generously provided by TikTok, we demonstrate that APL can reduce the stall ratio by 81%/12% and bandwidth waste by 11%/31% compared with no-preloading/fixed-preloading mechanism.

2020 IEEE International Conference on Visual Communications and Image Processing (VCIP)

On a platform of user-generated content (UGC), the uploaded videos need to be encoded again befor... more On a platform of user-generated content (UGC), the uploaded videos need to be encoded again before distribution. For this specific encoding scenario, we propose a novel dataset and a corresponding learning-based scheme that is able to achieve significant bitrate saving without decreasing perceptual quality. In the dataset, each video’s label indicates whether it can be encoded with a much lower bitrate while still keeps the same perceptual quality. Models trained on this dataset can then be used to classify the input video and adjust its final encoding parameters accordingly. With enough classification accuracy, more than 20% average bitrate saving can be obtained through the proposed scheme. The dataset will be further expanded to facilitate the study on this problem.

MultiMedia Modeling, 2022

2020 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), 2020

2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 2021

2016 IEEE International Conference on Multimedia and Expo (ICME), 2016

TCP-based application-layer protocols are increasingly applied to commercial live video streaming... more TCP-based application-layer protocols are increasingly applied to commercial live video streaming systems. However, in unstable mobile networks, the throughput of TCP may fluctuate rapidly due to its transmission mechanism, causing undesired playback interruption. In this paper, we propose a general application-layer rate adaptation approach to cope with the variability in TCP throughput. We analyze the transmission process and evaluate the network condition using a multi-buffer model. With information obtained in this model, an algorithm based on the Proportional-Integral-Derivative (PID) controller is proposed to dynamically adjust the video bitrate in response to the throughput change. We have implemented an experimental mobile live streaming system employing this approach and achieved a significant improvement in playback continuity and bandwidth utilization.

IEEE Transactions on Multimedia, 2021

2018 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 2018

Recently, the 360-degree video has become a hot topic in multimedia area. However, the requiremen... more Recently, the 360-degree video has become a hot topic in multimedia area. However, the requirements of high bitrate, low Internet interactive latency and high perceived quality limit its further applications. So in this paper, we design a tile- based QoE-driven HTTP/2 streaming system for 360 video. It uses a new projection method, Sinusoidal Projection (SP), to reduce the bitrate of tiles. A novel cross-user's behavior learning method is also used to predict viewpoint. To improve bandwidth utilization, it pushes multiple tiles in one request by HTTP/2. Besides, by using a QoE-driven framework, our approach can significantly improve users' perceived quality. The numerous experiment results have demonstrated the efficiency of the proposed system. Compared with the legacy methods, the transmission bitrate drops about 17%, the view- port prediction accuracy improves 30%, the Viewport-PSNR improve 22% and the transmission latency drops about 30%.

IEEE Transactions on Multimedia, 2016

To cope with the challenges brought about by bandwidth fluctuation and improve the experience of ... more To cope with the challenges brought about by bandwidth fluctuation and improve the experience of watching online videos, an adaptive video streaming system that can adjust video quality according to actual network conditions is proposed based on the scalable video coding (SVC) extension of H.264/AVC. First, a simple and effective linear error model is proposed and verified for quality scalability of SVC. The model exploits the linear feature of pixel value errors and can be used to accurately estimate the distortion caused by discarding any combination of enhancement data packets in an SVC bitstream. On that basis, a greedy-like algorithm is designed to assign each data packet a priority value according to its rate-distortion (R-D) impact, thus enabling R-D optimized bitstream extraction under certain bitrate constraints. Finally, the proportional-integral-derivative (PID) method is utilized to control the video quality adjustment and determine a suitable bitrate for transmission. By monitoring and predicting the past, current, and future bandwidth information, the PID-based quality control algorithm is able to reduce quality fluctuation, while still preserving a high quality level. Experimental results show that compared with the baseline software, the proposed system that integrates the above algorithms can achieve much lower video quality fluctuation, with PSNR variance reduced from 1.24 to 0.69, and at the same time deliver higher video quality, with the PSNR average increased by 0.83 dB.

2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2014

This paper investigates the no-reference SVC bitstream extraction problem and presents an efficie... more This paper investigates the no-reference SVC bitstream extraction problem and presents an efficient solution to approximate the “optimal” extracted sub-stream. First, we introduce a linear error model to accurately estimate the distortion caused by discarding any combination of packets, even when the original sequence is not available. Then we propose a greedy algorithm to decide each packet's priority according to its R-D impact. The priority value of packets can be stored in the bitstream and used for R-D optimized extraction. Experimental results show that our bitstream extraction method can achieve a significant PSNR gain compared to the extractors of JSVM, without computational complexity increment. Comparison with other methods also demonstrates the advantage of the proposed method.

2014 IEEE International Conference on Communications (ICC), 2014

Scalable Video Coding (SVC) makes it possible to change video quality dynamically according to re... more Scalable Video Coding (SVC) makes it possible to change video quality dynamically according to real-time bandwidth. For quality control algorithms of SVC video streaming, the biggest challenge is to keep a video quality that is both smooth and as good as possible. In this paper, we first introduce a combined quality level scheme to describe SVC video quality in a unified way. Then an effective and efficient quality control algorithm for SVC video streaming is proposed based on the Proportional-Integral-Derivative (PID) control method. Extensive experiments show that the proposed algorithm improves 8.6% in video quality with 24.8% reduction in quality fluctuation compared with the existing packet delay feedback algorithm. The proposed algorithm has also been implemented in online video website www.7dlive.com and performs well in applications.

2014 IEEE 16th International Workshop on Multimedia Signal Processing (MMSP), 2014

In this paper, we propose a novel design and optimized implementation of the HEVC decoder. First,... more In this paper, we propose a novel design and optimized implementation of the HEVC decoder. First, a novel decoder prototype with refined decoding workflow and efficient memory management is designed. Then on this basis, a series of single-instruction-multiple-data (SIMD) based algorithms are used to speed up several time-consuming modules in HEVC decoding. Finally, a frame-based parallel framework is applied to exploit the multi-threading technology on multicore processors. With the highly optimized HEVC decoder, decoding speed of 246fps on Intel i7-2400 3.4GHz quad-core processor for 1080p videos and 52fps on ARM Cortex-A9 1.2GHz dual-core processor for 720p videos can be achieved in our experiments.

Lecture Notes in Computer Science, 2015

In this demonstration, we showcase a complete software encoding and decoding solution for the new... more In this demonstration, we showcase a complete software encoding and decoding solution for the new High Efficiency Video Coding (HEVC) standard. The encoder is optimized for x86 processors using SSE instruction set extension and multi-thread technology, and achieves high efficiency at a significantly reduced computation load. We have integrated the encoder library into the widely-used media framework FFmpeg and developed transcoding and recording applications for HEVC. The decoder is highly optimized for both x86 and ARM architecture. With novel single-instruction-multiple-data (SIMD) algorithms and a frame-based parallel framework for multi-core CPUs, decoding speed of 46FPS for 1080p videos on ARM Cortex-A9 1.5GHz dual-core processor and 75FPS for 4K (3840x2160) videos on Intel i7-2600 3.4GHz quad-core processor can be achieved. We have also integrated the decoder library into FFmpeg and made an Android video player based on that. The software solution can well meet the demand of producing and watching HEVC videos on existing devices, showing promising future of HEVC applications.