Jinwoo Jeong - Academia.edu (original) (raw)

Papers by Jinwoo Jeong

IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, 2013

IEEE Access

Single-image super-resolution technology has been widely studied in various applications to impro... more Single-image super-resolution technology has been widely studied in various applications to improve the quality and resolution of degraded images acquired from noise-sensitive low-resolution sensors. As most studies on single-image super-resolution focused on the development of deep learning networks operating on high-performance GPUs, this study proposed an efficient and lightweight super-resolution network that enables real-time performance on mobile devices. To replace the relatively slow element-wise addition layer on mobile devices, we introduced a skip connection layer by directly concatenating a low-resolution input image with an intermediate feature map. In addition, we introduced weighted clipping to reduce the quantization errors commonly encountered during float-to-int8 model conversion. Moreover, a reparameterization method was selectively applied without increasing the cost in terms of inference time and number of parameters. Based on the contributions, the proposed network has been recognized as the best solution in Mobile AI & AIM 2022 Real-Time Single-Image Super-Resolution Challenge with PSNR of 30.03 dB and NPU runtime of 19.20 ms.

Advanced Science and Technology Letters, 2017

In this paper, we propose a fast motion estimation technique using chroma interpolation filter fo... more In this paper, we propose a fast motion estimation technique using chroma interpolation filter for High Efficiency Video Coding (HEVC) encoder. In order to reduce the computation time of motion estimation process, the proposed method utilizes 4-tap chroma interpolation filter for motion estimation and 8-tap luma interpolation filter for motion compensation, separately, instead of applying the same 8-tap luma interpolation filter to both motion estimation and motion compensation process. From the experimental results, the proposed algorithm achieves 4.63% time-saving with 0.06% BD-bitrate increase on average, compared to x265 encoder, which is the most computationally efficient HEVC encoder in the industrial field.

2009 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting, 2009

To provide heterogeneous device capability in the video multicasting/broadcasting, the layered mu... more To provide heterogeneous device capability in the video multicasting/broadcasting, the layered multicast can be exploited. However, previous statistical multiplexing system do not consider to support layered multicast. In order to support heterogenous capability for broadcasting, this paper presents statistical multiplexing system for layered multicast. In proposed system, video sequences for multiple programs are encoded to one base layer stream and several enhancement layer streams, and base layer streams of each program are multiplexed into one single CBR channel and multiple enhancement layers of each program are separately multiplexed into multiple single CBR channels. Furthermore, we allocated different bandwidth for each program using complexity measure in order to achieve similar visual quality among all programs in all layers. Experimental results show that the proposed statistical multiplexing system can efficiently support multiple multiplexed streams for layered multicast and provides a smooth video quality change across programs.

Symmetry, 2019

Visual quality and algorithm efficiency are two main interests in video frame interpolation. We p... more Visual quality and algorithm efficiency are two main interests in video frame interpolation. We propose a hybrid task-based convolutional neural network for fast and accurate frame interpolation of 4K videos. The proposed method synthesizes low-resolution frames, then reconstructs high-resolution frames in a coarse-to-fine fashion. We also propose edge loss, to preserve high-frequency information and make the synthesized frames look sharper. Experimental results show that the proposed method achieves state-of-the-art performance and performs 2.69x faster than the existing methods that are operable for 4K videos, while maintaining comparable visual and quantitative quality.

In many applications, video sequences must be transmitted constant bit rate channels (CBR). There... more In many applications, video sequences must be transmitted constant bit rate channels (CBR). Therefore, rate control has to be used to regulate the variable bit rate of coded stream. In this paper, we propose an effective bit-rate control algorithm in H.264/AVC for CBR transmission. Our algorithm jointly decides mode and quantization using rate distortion optimization (RDO) technique. And to reduce computational complexity, the proposed algorithm uses modified version of rate and distortion model calculated from the histogram of DCT coefficients. Since the proposed algorithm induces the rate and distortion model from the current macroblock instead of the previous coded frame or macroblock, it can effectively prevents the fluctuation of the output bits especially when scene change occurs, while keeping the quality in same level. And it can not only achieve the improvement of the rate control but also meet the target bit rate more accurately.

IEICE Electronics Express, 2009

Inter-frame coding using inter-and intra-predictions plays an important role in achieving high co... more Inter-frame coding using inter-and intra-predictions plays an important role in achieving high compression efficiency in H.264/AVC. However, most intra-predictions are unnecessary, since the intra-coding mode occupies less than 5% of the overall coding in interframe coding. In this paper, we propose an intra mode skip algorithm for inter-frame coding in H.264/AVC as a means of obtaining fast intra mode decision. The algorithm uses the joint entropy (JE) and mutual information (MI) to extract the temporal correlations between the current block and the reconstructed block based on 8 × 8 motion estimation. All or part of the intra-prediction search is omitted by using JE and MI. Experimental results show that the proposed algorithm is capable of reducing the overall coding time by 15-37% and the overall intra-prediction time by 32-78% compared to full search of the reference software, with negligible loss of quality.

IEICE Electronics Express, 2010

In video coding, a portion of skipped macroblocks is highly effective to the coding performance, ... more In video coding, a portion of skipped macroblocks is highly effective to the coding performance, especially, at low bitrates since, for a skip mode, no additional coding parameters are transmitted except mode index itself. To increase the skip mode occurrence, in this paper, we propose an efficient skip mode competition scheme based on vector clustering of multiple predictive motion vectors (PMV) and boundary detection of moving objects. The proposed scheme selects the best PMV of skip mode optimally among representative PMV candidates which are obtained after the clustering process and switches a more appropriate competition scheme adaptively according to object boundary conditions. Experimental results show that the proposed scheme reduces average 4.31% in BDRATE compared to the KTA 2.6 software.

IEICE Electronics Express, 2011

For coding of quantized transform coefficients, a coded block pattern (CBP) is a key syntax eleme... more For coding of quantized transform coefficients, a coded block pattern (CBP) is a key syntax element to represent the existence of coefficients in a macroblock efficiently. In H.264/AVC, the CBP is coded by using a fixed variable length coding table, called Exp-Golomb codes. However, since the CBP is highly affected by the change of quantization parameter (QP), in this paper, we show the CBP distributions according to the change of QP and propose an context-adaptive CBP coding method. The proposed scheme selects the coding table adaptively based on the context of neighbors' CBP. Experimental results show that the proposed scheme reduces average 1.15% in total bit rate and 12.54% in CBP bit rate compared to the anchor, H.264/AVC.

2019 International Conference on Information and Communication Technology Convergence (ICTC), 2019

AV1 is an emerging open-source and royalty-free video compression format, which is jointly develo... more AV1 is an emerging open-source and royalty-free video compression format, which is jointly developed and finalized in June 2018 by the Alliance for Open Media (AOMedia) industry consortium. AV1 achieves significant improvements in coding efficiency compared with HECV and VP9 thanks to new coding tools. However, encoding complexity is extremely high by selecting best mode among new features. This paper proposes a fast chroma prediction mode decision algorithm based on luma prediction mode for AV1 intra coding. Experiment results show that the proposed algorithm achieves an 15.86% TS with 0.44% BDBR increase compared with AV1 reference software.

IEIE Transactions on Smart Processing & Computing, 2018

Symmetry, 2019

Recently, video frame interpolation research developed with a convolutional neural network has sh... more Recently, video frame interpolation research developed with a convolutional neural network has shown remarkable results. However, these methods demand huge amounts of memory and run time for high-resolution videos, and are unable to process a 4K frame in a single pass. In this paper, we propose a fast 4K video frame interpolation method, based upon a multi-scale optical flow reconstruction scheme. The proposed method predicts low resolution bi-directional optical flow, and reconstructs it into high resolution. We also proposed consistency and multi-scale smoothness loss to enhance the quality of the predicted optical flow. Furthermore, we use adversarial loss to make the interpolated frame more seamless and natural. We demonstrated that the proposed method outperforms the existing state-of-the-art methods in quantitative evaluation, while it runs up to 4.39× faster than those methods for 4K videos.

In this paper, we propose a fast sample adaptive offset (SAO) parameter estimation algorithm base... more In this paper, we propose a fast sample adaptive offset (SAO) parameter estimation algorithm based on the block-based edge detection in High Efficiency Video Coding (HEVC) encoder. Instead of the exhaustive search to find the best SAO parameters among all massive combinations, the proposed algorithm decides the best edge offset by analyzing the input block's edge characteristics in advance. From the experimental results, the proposed algorithm achieves 61.21% time-saving with 1.09% BD-rate increase on average, compared to HEVC reference software HM 16.9.

In this paper, we propose a bit-rate control algorithm for H.264/AVC using the rate and distortio... more In this paper, we propose a bit-rate control algorithm for H.264/AVC using the rate and distortion model estimated by the histogram of DCT coefficients. Since the proposed algorithm induces the rate and distortion model from the current macroblock instead of the previous coded frame or macroblock, it can effectively prevent the fluctuation of the output bits especially when scene change occurs, while keeping the quality in same level.

2008 2nd International Conference on Signal Processing and Communication Systems, 2008

The H.264/AVC uses the rate distortion optimization (RDO) method as a measure to select the optim... more The H.264/AVC uses the rate distortion optimization (RDO) method as a measure to select the optimal coding mode. However, the computational complexity for RDO calculation is increased due to the various coding modes. In this paper, we analyzes that the best mode decision for 4 × 4 intra-prediction can be affected according to the relation of most probable mode (MPM) and prediction mode. With this understanding, we propose an algorithm for fast 4 × 4 intraprediction mode selection based on the MPM in the H.264/AVC encoder. A defined rate distortion (RD) cost, including the minimum bit for signaling the prediction modes except the MPM, is used as a threshold for fast 4 × 4 intraprediction. Experimental results show that the proposed algorithm is capable of reducing the overall encoding time by 27-29% and the overall intra-prediction time by 35% compared to full search of the reference software, without loss of quality.

2013 International Conference on Information Science and Cloud Computing Companion, 2013

In this paper, we propose a novel technique for film grain noise removal, which can be adopted in... more In this paper, we propose a novel technique for film grain noise removal, which can be adopted in high fidelity video coding. Film grain noise enhances the natural appearance of high fidelity video, therefore it is should be preserved. However, film grain noise is a burden to typical video compression systems because it has relatively large energy level in the high frequency region. In order to improve the coding performance while preserving film grain noise, the noise removal and synthesis process is used. We propose a film grain noise removal technology in the pre-processing step. In pre-processing step, film grain noise is removed by using temporal, spatial and inter-color correlation. Specially, color image denoisng using inter color prediction provides good denoising performance in noise concentrated B plane because film grain noise has inter-color correlation in the RGB domain. The results show that the coding gain of denoised video is higher than for previous works,while the visual quality of the final reconstructed video is well preserved.

Optical Engineering, 2011

ABSTRACT Intra coding of an RGB video is important to many high fidelity multimedia applications ... more ABSTRACT Intra coding of an RGB video is important to many high fidelity multimedia applications because video acquisition is mostly done in RGB space, and the coding of decorrelated color video loses its virtue in high quality ranges. In order to improve the compression performance of an RGB video, this paper proposes an inter color prediction using adaptive weights. For making full use of spatial, as well as inter color correlation of an RGB video, the proposed scheme is based on a residual prediction approach, and thus the incorporated prediction is performed on the transformed frequency components of spatially predicted residual data of each color plane. With the aid of efficient prediction employing frequency domain inter color residual correlation, the proposed scheme achieves up to 24.3% of bitrate reduction, compared to the common mode of H.264/AVC high 4:4:4 intra profile.

IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, 2013

IEEE Access

Single-image super-resolution technology has been widely studied in various applications to impro... more Single-image super-resolution technology has been widely studied in various applications to improve the quality and resolution of degraded images acquired from noise-sensitive low-resolution sensors. As most studies on single-image super-resolution focused on the development of deep learning networks operating on high-performance GPUs, this study proposed an efficient and lightweight super-resolution network that enables real-time performance on mobile devices. To replace the relatively slow element-wise addition layer on mobile devices, we introduced a skip connection layer by directly concatenating a low-resolution input image with an intermediate feature map. In addition, we introduced weighted clipping to reduce the quantization errors commonly encountered during float-to-int8 model conversion. Moreover, a reparameterization method was selectively applied without increasing the cost in terms of inference time and number of parameters. Based on the contributions, the proposed network has been recognized as the best solution in Mobile AI & AIM 2022 Real-Time Single-Image Super-Resolution Challenge with PSNR of 30.03 dB and NPU runtime of 19.20 ms.

Advanced Science and Technology Letters, 2017

In this paper, we propose a fast motion estimation technique using chroma interpolation filter fo... more In this paper, we propose a fast motion estimation technique using chroma interpolation filter for High Efficiency Video Coding (HEVC) encoder. In order to reduce the computation time of motion estimation process, the proposed method utilizes 4-tap chroma interpolation filter for motion estimation and 8-tap luma interpolation filter for motion compensation, separately, instead of applying the same 8-tap luma interpolation filter to both motion estimation and motion compensation process. From the experimental results, the proposed algorithm achieves 4.63% time-saving with 0.06% BD-bitrate increase on average, compared to x265 encoder, which is the most computationally efficient HEVC encoder in the industrial field.

2009 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting, 2009

To provide heterogeneous device capability in the video multicasting/broadcasting, the layered mu... more To provide heterogeneous device capability in the video multicasting/broadcasting, the layered multicast can be exploited. However, previous statistical multiplexing system do not consider to support layered multicast. In order to support heterogenous capability for broadcasting, this paper presents statistical multiplexing system for layered multicast. In proposed system, video sequences for multiple programs are encoded to one base layer stream and several enhancement layer streams, and base layer streams of each program are multiplexed into one single CBR channel and multiple enhancement layers of each program are separately multiplexed into multiple single CBR channels. Furthermore, we allocated different bandwidth for each program using complexity measure in order to achieve similar visual quality among all programs in all layers. Experimental results show that the proposed statistical multiplexing system can efficiently support multiple multiplexed streams for layered multicast and provides a smooth video quality change across programs.

Symmetry, 2019

Visual quality and algorithm efficiency are two main interests in video frame interpolation. We p... more Visual quality and algorithm efficiency are two main interests in video frame interpolation. We propose a hybrid task-based convolutional neural network for fast and accurate frame interpolation of 4K videos. The proposed method synthesizes low-resolution frames, then reconstructs high-resolution frames in a coarse-to-fine fashion. We also propose edge loss, to preserve high-frequency information and make the synthesized frames look sharper. Experimental results show that the proposed method achieves state-of-the-art performance and performs 2.69x faster than the existing methods that are operable for 4K videos, while maintaining comparable visual and quantitative quality.

In many applications, video sequences must be transmitted constant bit rate channels (CBR). There... more In many applications, video sequences must be transmitted constant bit rate channels (CBR). Therefore, rate control has to be used to regulate the variable bit rate of coded stream. In this paper, we propose an effective bit-rate control algorithm in H.264/AVC for CBR transmission. Our algorithm jointly decides mode and quantization using rate distortion optimization (RDO) technique. And to reduce computational complexity, the proposed algorithm uses modified version of rate and distortion model calculated from the histogram of DCT coefficients. Since the proposed algorithm induces the rate and distortion model from the current macroblock instead of the previous coded frame or macroblock, it can effectively prevents the fluctuation of the output bits especially when scene change occurs, while keeping the quality in same level. And it can not only achieve the improvement of the rate control but also meet the target bit rate more accurately.

IEICE Electronics Express, 2009

Inter-frame coding using inter-and intra-predictions plays an important role in achieving high co... more Inter-frame coding using inter-and intra-predictions plays an important role in achieving high compression efficiency in H.264/AVC. However, most intra-predictions are unnecessary, since the intra-coding mode occupies less than 5% of the overall coding in interframe coding. In this paper, we propose an intra mode skip algorithm for inter-frame coding in H.264/AVC as a means of obtaining fast intra mode decision. The algorithm uses the joint entropy (JE) and mutual information (MI) to extract the temporal correlations between the current block and the reconstructed block based on 8 × 8 motion estimation. All or part of the intra-prediction search is omitted by using JE and MI. Experimental results show that the proposed algorithm is capable of reducing the overall coding time by 15-37% and the overall intra-prediction time by 32-78% compared to full search of the reference software, with negligible loss of quality.

IEICE Electronics Express, 2010

In video coding, a portion of skipped macroblocks is highly effective to the coding performance, ... more In video coding, a portion of skipped macroblocks is highly effective to the coding performance, especially, at low bitrates since, for a skip mode, no additional coding parameters are transmitted except mode index itself. To increase the skip mode occurrence, in this paper, we propose an efficient skip mode competition scheme based on vector clustering of multiple predictive motion vectors (PMV) and boundary detection of moving objects. The proposed scheme selects the best PMV of skip mode optimally among representative PMV candidates which are obtained after the clustering process and switches a more appropriate competition scheme adaptively according to object boundary conditions. Experimental results show that the proposed scheme reduces average 4.31% in BDRATE compared to the KTA 2.6 software.

IEICE Electronics Express, 2011

For coding of quantized transform coefficients, a coded block pattern (CBP) is a key syntax eleme... more For coding of quantized transform coefficients, a coded block pattern (CBP) is a key syntax element to represent the existence of coefficients in a macroblock efficiently. In H.264/AVC, the CBP is coded by using a fixed variable length coding table, called Exp-Golomb codes. However, since the CBP is highly affected by the change of quantization parameter (QP), in this paper, we show the CBP distributions according to the change of QP and propose an context-adaptive CBP coding method. The proposed scheme selects the coding table adaptively based on the context of neighbors' CBP. Experimental results show that the proposed scheme reduces average 1.15% in total bit rate and 12.54% in CBP bit rate compared to the anchor, H.264/AVC.

2019 International Conference on Information and Communication Technology Convergence (ICTC), 2019

AV1 is an emerging open-source and royalty-free video compression format, which is jointly develo... more AV1 is an emerging open-source and royalty-free video compression format, which is jointly developed and finalized in June 2018 by the Alliance for Open Media (AOMedia) industry consortium. AV1 achieves significant improvements in coding efficiency compared with HECV and VP9 thanks to new coding tools. However, encoding complexity is extremely high by selecting best mode among new features. This paper proposes a fast chroma prediction mode decision algorithm based on luma prediction mode for AV1 intra coding. Experiment results show that the proposed algorithm achieves an 15.86% TS with 0.44% BDBR increase compared with AV1 reference software.

IEIE Transactions on Smart Processing & Computing, 2018

Symmetry, 2019

Recently, video frame interpolation research developed with a convolutional neural network has sh... more Recently, video frame interpolation research developed with a convolutional neural network has shown remarkable results. However, these methods demand huge amounts of memory and run time for high-resolution videos, and are unable to process a 4K frame in a single pass. In this paper, we propose a fast 4K video frame interpolation method, based upon a multi-scale optical flow reconstruction scheme. The proposed method predicts low resolution bi-directional optical flow, and reconstructs it into high resolution. We also proposed consistency and multi-scale smoothness loss to enhance the quality of the predicted optical flow. Furthermore, we use adversarial loss to make the interpolated frame more seamless and natural. We demonstrated that the proposed method outperforms the existing state-of-the-art methods in quantitative evaluation, while it runs up to 4.39× faster than those methods for 4K videos.

In this paper, we propose a fast sample adaptive offset (SAO) parameter estimation algorithm base... more In this paper, we propose a fast sample adaptive offset (SAO) parameter estimation algorithm based on the block-based edge detection in High Efficiency Video Coding (HEVC) encoder. Instead of the exhaustive search to find the best SAO parameters among all massive combinations, the proposed algorithm decides the best edge offset by analyzing the input block's edge characteristics in advance. From the experimental results, the proposed algorithm achieves 61.21% time-saving with 1.09% BD-rate increase on average, compared to HEVC reference software HM 16.9.

In this paper, we propose a bit-rate control algorithm for H.264/AVC using the rate and distortio... more In this paper, we propose a bit-rate control algorithm for H.264/AVC using the rate and distortion model estimated by the histogram of DCT coefficients. Since the proposed algorithm induces the rate and distortion model from the current macroblock instead of the previous coded frame or macroblock, it can effectively prevent the fluctuation of the output bits especially when scene change occurs, while keeping the quality in same level.

2008 2nd International Conference on Signal Processing and Communication Systems, 2008

The H.264/AVC uses the rate distortion optimization (RDO) method as a measure to select the optim... more The H.264/AVC uses the rate distortion optimization (RDO) method as a measure to select the optimal coding mode. However, the computational complexity for RDO calculation is increased due to the various coding modes. In this paper, we analyzes that the best mode decision for 4 × 4 intra-prediction can be affected according to the relation of most probable mode (MPM) and prediction mode. With this understanding, we propose an algorithm for fast 4 × 4 intraprediction mode selection based on the MPM in the H.264/AVC encoder. A defined rate distortion (RD) cost, including the minimum bit for signaling the prediction modes except the MPM, is used as a threshold for fast 4 × 4 intraprediction. Experimental results show that the proposed algorithm is capable of reducing the overall encoding time by 27-29% and the overall intra-prediction time by 35% compared to full search of the reference software, without loss of quality.

2013 International Conference on Information Science and Cloud Computing Companion, 2013

In this paper, we propose a novel technique for film grain noise removal, which can be adopted in... more In this paper, we propose a novel technique for film grain noise removal, which can be adopted in high fidelity video coding. Film grain noise enhances the natural appearance of high fidelity video, therefore it is should be preserved. However, film grain noise is a burden to typical video compression systems because it has relatively large energy level in the high frequency region. In order to improve the coding performance while preserving film grain noise, the noise removal and synthesis process is used. We propose a film grain noise removal technology in the pre-processing step. In pre-processing step, film grain noise is removed by using temporal, spatial and inter-color correlation. Specially, color image denoisng using inter color prediction provides good denoising performance in noise concentrated B plane because film grain noise has inter-color correlation in the RGB domain. The results show that the coding gain of denoised video is higher than for previous works,while the visual quality of the final reconstructed video is well preserved.

Optical Engineering, 2011

ABSTRACT Intra coding of an RGB video is important to many high fidelity multimedia applications ... more ABSTRACT Intra coding of an RGB video is important to many high fidelity multimedia applications because video acquisition is mostly done in RGB space, and the coding of decorrelated color video loses its virtue in high quality ranges. In order to improve the compression performance of an RGB video, this paper proposes an inter color prediction using adaptive weights. For making full use of spatial, as well as inter color correlation of an RGB video, the proposed scheme is based on a residual prediction approach, and thus the incorporated prediction is performed on the transformed frequency components of spatially predicted residual data of each color plane. With the aid of efficient prediction employing frequency domain inter color residual correlation, the proposed scheme achieves up to 24.3% of bitrate reduction, compared to the common mode of H.264/AVC high 4:4:4 intra profile.