Learning to Enhance Low-Light Image via Zero-Reference Deep Curve Estimation (original) (raw)

1 Nanyang Technological University, Singapore
2 Nankai University, Tianjin, China

Abstract

This work is an extension of our earlier conference version that has appeared in CVRP2020 (Zero-DCE). We further present an accelerated and light version of Zero-DCE, called Zero-DCE++, that takes advantage of a tiny network with just 10K parameters. Zero-DCE++ has a fast inference speed (1000/11 FPS on single GPU/CPU for an image with a size of 1200*900*3) while keeping the enhancement performance of Zero-DCE.

Highlights

  1. We propose the first low-light enhancement network that is independent of paired and unpaired training data, thus avoiding the risk of overfitting. As a result, our method generalizes well to various lighting conditions.
  2. We design an image-specific curve that is able to approximate pixel-wise and higher-order curves by iteratively applying itself. Such image-specific curve can effectively perform mapping within a wide dynamic range.
  3. We show the potential of training a deep image enhancement model in the absence of reference images through task-specific non-reference loss functions that indirectly evaluate enhancement quality.
  4. The proposed Zero-DCE can be accelerated considerably while still keeping impressive enhancement performance. We provide multiple options to balance the enhancement performance and the cost of computational resources.

Modifications

  1. We investigate the relations between the enhancement performance and the network structure, curve estimation, and input sizes. According to the investigation, we re-design the network structure, reformulate the curve formation, and control the sizes of input image, and thus present an accelerated and light version Zero-DCE, called Zero-DCE++, which is more suitable for real-time enhancement on resource-limited devices.
  2. Comparing to our earlier work, without compromising the enhancement performance, the trainable parameters (79K) and foating point operations (FLOPs) (84.99G) for an input image of size 1200*900*3 of Zero-DCE are reduced to 10K and 0.115G on Zero-DCE++. This translates to two times in runtime speed up, from 500 FPS in Zero-DCE to 1000 FPS in Zero-DCE++, for processing an image of size 1200*900*3 on a single NVIDIA 2080Ti GPU. In addition, even only with Intel Core i9-10920X CPU@3.5GHz, the processing time of Zero-DCE also can be reduced from 10s to 0.09s on Zero-DCE++, a 111 times speed up on a single CPU setting. The training time is also reduced from 30 minutes to 20 minutes.
  3. We perform more experiments, design analysis, and ablation studies to demonstrate the advantages of zero-reference learning for low-light image enhancement and show the effectiveness of our method over existing state-of-the-art methods.
  4. We conduct a more comprehensive literature survey on low-light image enhancement and discuss the advantages and limitations of current methods.

Citation

@Article{Zero-DCE++, author ={Li, Chongyi and Guo, Chunle and Loy, Chen Change}, title = {Learning to Enhance Low-Light Image via Zero-Reference Deep Curve Estimation}, journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence}, pape={}, year = {2021}, doi={10.1109/TPAMI.2021.3063604} }

@Article{Zero-DCE, author = {Guo, Chunle and Li, Chongyi and Guo, Jichang and Loy, Chen Change and Hou, Junhui and Kwong, Sam and Cong Runmin}, title = {Zero-reference deep curve estimation for low-light image enhancement}, journal = {CVPR}, pape={1780-1789}, year = {2020} }