The Family of Onion Convolutions for Image Inpainting (original) (raw)

References

• Abadi, M., Agarwal, A., Barham, P. et al. (2015). TensorFlow: Large-scale machine learning on heterogeneous systems. https://www.tensorflow.org/, software available from tensorflow.org
• Adobe Inc (2019). Adobe photoshop. https://www.adobe.com/products/photoshop.html
• Ashikhmin, M. (2001). Synthesizing natural textures. In: Proceedings of the 2001 Symposium on Interactive 3D Graphics, Association for Computing Machinery, New York, NY, USA, I3D –01, p 217–226, https://doi.org/10.1145/364338.364405
• Bahdanau, D., Cho, K., & Bengio, Y. (2015). Neural machine translation by jointly learning to align and translate. CoRR abs/1409.0473
• Barnes, C., Shechtman, E., Finkelstein, A., & Goldman, D. B. (2009). PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics (Proc SIGGRAPH) 28(3)
• Bello, I., Zoph, B., Le, Q., Vaswani, A., & Shlens, J. (2019). Attention augmented convolutional networks. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp 3285–3294
• Bertalmío, M., Sapiro, G., Caselles, V., & Ballester, C. (2000). Image inpainting. In: SIGGRAPH ’00
• Bertalmio, M., Vese, L., Sapiro, G., & Osher, S. (2003). Simultaneous structure and texture image inpainting. IEEE Transactions on Image Processing, 12(8), 882–889.
  Article Google Scholar
• Boykov, Y., Veksler, O., & Zabih, R. (2001). Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(11), 1222–1239.
  Article Google Scholar
• Canny, J. (1986). A computational approach to edge detection. IEEE Transactions Pattern Analysis and Machine Intelligence, 8(6), 679–698. https://doi.org/10.1109/TPAMI.1986.4767851
  Article Google Scholar
• Chan, T. F., & Shen, J. (2000). Non-texture inpainting by curvature-driven diffusions (cdd). Journal of visual communication and image representation, 12, 436–449.
  Article Google Scholar
• Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018). Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds) ECCV (7), Springer, Lecture Notes in Computer Science, vol 11211, pp 833–851, http://dblp.uni-trier.de/db/conf/eccv/eccv2018-7.html#ChenZPSA18
• Chen, T.Q., & Schmidt, M. (2016). Fast patch-based style transfer of arbitrary style. arXiv preprint arXiv:1612.04337
• Clevert, D.A., Unterthiner, T., & Hochreiter, S. (2015). Fast and accurate deep network learning by exponential linear units (elus). CoRR abs/1511.07289
• Cordonnier, J.B., Loukas, A., & Jaggi, M. (2020). On the relationship between self-attention and convolutional layers. In: International Conference on Learning Representations, https://openreview.net/forum?id=HJlnC1rKPB
• Criminisi, A., Pérez, P., & Toyama, K. (2003). Object removal by exemplar-based inpainting., 2, 721–728. https://doi.org/10.1109/CVPR.2003.1211538
• Criminisi, A., Perez, P., & Toyama, K. (2004). Region filling and object removal by exemplar-based image inpainting. Transactions on Image Processing, 13(9), 1200–1212. https://doi.org/10.1109/TIP.2004.833105
  Article Google Scholar
• Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., & Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, IEEE, pp 248–255
• Efros, A., & Leung, T. (1999). Texture synthesis by non-parametric sampling. In: International Conference on Computer Vision, pp 1033–1038
• Efros, A.A., & Freeman, W.T. (2001). Image quilting for texture synthesis and transfer. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, Association for Computing Machinery, New York, NY, USA, SIGGRAPH ’01, p 341-346, https://doi.org/10.1145/383259.383296,
• Fu, J., Liu, J., Tian, H., Li, Y., Bao, Y., Fang, Z., & Lu, H. (2019). Dual attention network for scene segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3146–3154
• Garber, D.D. (1981). Computational models for texture analysis and texture synthesis. PhD thesis, USA, aAI0551115
• Gatys, L.A., Ecker, A.S., & Bethge, M. (2016). Image style transfer using convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2414–2423
• Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014). Generative adversarial nets. In: Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, MIT Press, Cambridge, MA, USA, NIPS’14, p 2672-2680
• Gregor, K., Danihelka, I., Graves, A., Rezende, D., & Wierstra, D. (2015). Draw: A recurrent neural network for image generation. In: Bach F, Blei D (eds) Proceedings of the 32nd International Conference on Machine Learning, PMLR, Lille, France, Proceedings of Machine Learning Research, vol 37, pp 1462–1471, http://proceedings.mlr.press/v37/gregor15.html
• Harrison, P. (2001). A non-hierarchical procedure for re-synthesis of complex textures. In: Skala V (ed) _Proceedings of the 9-th international conference in central europe on computer graphics, visualization and computer vision_’2001, University of West Bohemia, pp 190 – 197, http://wscg.zcu.cz/wscg2001/wscg2001.htm, international Conferences in Central Europe on Computer Graphics, Visualization and Computer Vision 2001, WSCG 2001 ; Conference date: 05-02-2001 Through 09-02-2001
• Hertzmann, A., Jacobs, C.E., Oliver, N., Curless, B., & Salesin, D.H. (2001). Image analogies. In: Proceedings of the 28th annual conference on computer graphics and interactive techniques, pp 327–340
• Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017). Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems 30
• Hu, J., Shen, L., & Sun, G. (2018). Squeeze-and-excitation networks. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, pp 7132–7141
• Huang, J. B., Kang, S. B., Ahuja, N., & Kopf, J. (2014). Image completion using planar structure guidance. ACM Transactions on Graphics, 10(1145/2601097), 2601205.
  Google Scholar
• Hung, J., Chun-Hong, H., Yi-Chun, L., Tang, N., & Ta-Jen, C. (2008). Exemplar-based image inpainting base on structure construction. Journal of Software. https://doi.org/10.4304/jsw.3.8.57-64
  Article Google Scholar
• Iizuka, S., Simo-Serra, E., & Ishikawa, H. (2017). Globally and locally consistent image completion. ACM Transactions on Graphics, 36, 1–14. https://doi.org/10.1145/3072959.3073659
  Article Google Scholar
• Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2017). Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196
• Kingma, D.P., & Ba, J. (2015). Adam: A method for stochastic optimization. CoRR abs/1412.6980
• Kingma, D.P., & Welling, M. (2013). Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114
• Köhler, R., Schuler, C., Schölkopf, B., & Harmeling, S. (2014). Mask-specific inpainting with deep neural networks. German Conference on Pattern Recognition (pp. 523–534). Cham: Springer International Publishing.
  Google Scholar
• Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 25, 1097–1105.
  Google Scholar
• Kwatra, V., Schödl, A., Essa, I., Turk, G., & Bobick, A. (2003). Graphcut textures: Image and video synthesis using graph cuts. ACM Transactions on Graphics, 22, 277–286. https://doi.org/10.1145/1201775.882264
  Article Google Scholar
• Levin, Zomet, Weiss (2003). Learning how to inpaint from global image statistics. In: Proceedings Ninth IEEE international conference on computer vision, pp 305–312 vol.1
• Li, C., & Wand, M. (2016). Combining markov random fields and convolutional neural networks for image synthesis. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 2479–2486
• Li, J., Wang, N., Zhang, L., Du, B., & Tao, D. (2020a). Recurrent feature reasoning for image inpainting. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR)
• Li, Y., & Lu, H. (2020). Natural image matting via guided contextual attention. arXiv preprint arXiv:2001.04069
• Li, Y., Xu, Q., & Lu, H. (2020b). Hierarchical opacity propagation for image matting. arXiv preprint arXiv:2004.03249
• Liao, J., Yao, Y., Yuan, L., Hua, G., & Kang, S. B. (2017). Visual attribute transfer through deep image analogy. ACM Transactions on Graphics, 10(1145/3072959), 3073683.
  Google Scholar
• Liu, G., Reda, F.A., Shih, K.J., Wang, T.C., Tao, A., & Catanzaro, B. (2018). Image inpainting for irregular holes using partial convolutions. In: ECCV
• Luong, T., Pham, H., & Manning, C.D. (2015). Effective approaches to attention-based neural machine translation. In: proceedings of the 2015 conference on empirical methods in natural language processing, Association for Computational Linguistics, Lisbon, Portugal, pp 1412–1421, https://doi.org/10.18653/v1/D15-1166,https://www.aclweb.org/anthology/D15-1166
• Miyato, T., Kataoka, T., Koyama, M., & Yoshida, Y. (2018). Spectral normalization for generative adversarial networks. In: International conference on learning representations, https://openreview.net/forum?id=B1QRgziT-
• Navasardyan, S., & Ohanyan, M. (2020). Image inpainting with onion convolutions. In: proceedings of the asian conference on computer vision
• Nazeri, K., Ng, E., Joseph, T., Qureshi, F., & Ebrahimi, M. (2019). Edgeconnect: Structure guided image inpainting using edge prediction. In: The IEEE international conference on computer vision (ICCV) Workshops
• Oh, S.W., Lee, S., Lee, J.Y., & Kim, S.J. (2019). Onion-peel networks for deep video completion. In: proceedings of the IEEE/cvf international conference on computer vision, pp 4403–4412
• Park, D.Y., & Lee, K.H. (2019). Arbitrary style transfer with style-attentional networks. In: Proceedings of the IEEE/cvf conference on computer vision and pattern recognition, pp 5880–5888
• Pathak, D., Krähenbühl, P., Donahue, J., Darrell, T., & Efros, A.A. (2016). Context encoders: Feature learning by inpainting. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 2536–2544
• Ramachandran, P., Parmar, N., Vaswani, A., Bello, I., Levskaya, A., & Shlens, J. (2019). Stand-alone self-attention in vision models. In: NeurIPS
• Rane, S. D., Sapiro, G., & Bertalmio, M. (2003). Structure and texture filling-in of missing image blocks in wireless transmission and compression applications. Transactions on Image Processing, 12(3), 296–303. https://doi.org/10.1109/TIP.2002.804264
  Article MathSciNet Google Scholar
• Ren, J. S. J., Xu, L., Yan, Q., & Sun, W. (2015). Shepard convolutional neural networks. In: NIPS
• Shen, J., & Chan, T. (2002). Mathematical models for local nontexture inpaintings. SIAM Journal on Applied Mathematics, 62, 1019–1043. https://doi.org/10.1137/S0036139900368844
  Article MathSciNet MATH Google Scholar
• Simonyan, K., & Zisserman, A. (2015). Very deep convolutional networks for large-scale image recognition. In: Bengio Y, LeCun Y (eds) 3rd International conference on learning representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, arxiv:1409.1556
• Song, Y., Yang, C., Lin, Z., Liu, X., Huang, Q., Li, H., & Jay Kuo, C.C. (2018a). Contextual-based image inpainting: Infer, match, and translate. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 3–19
• Song, Y., Yang, C., Shen, Y., Wang, P., Huang, Q., & Kuo, C.J. (2018b). Spg-net: Segmentation prediction and guidance network for image inpainting. CoRR abs/1805.03356, arxiv:1805.03356 ,
• Suin, M., Purohit, K., & Rajagopalan, A. (2020). Spatially-attentive patch-hierarchical network for adaptive motion deblurring. In: Proceedings of the IEEE/cvf conference on computer vision and pattern recognition, pp 3606–3615
• Sun, J., Yuan, L., Jia, J., & Shum, H. Y. (2005). Image completion with structure propagation. ACM Transaction Graphic, 24(3), 861–868. https://doi.org/10.1145/1073204.1073274
  Article Google Scholar
• Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L.u., &Polosukhin, I. (2017). Attention is all you need. In: Advances in neural information processing systems 30, Curran Associates, Inc., pp 5998–6008, http://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf
• Wei, L.Y., & Levoy, M. (2000). Fast texture synthesis using tree-structured vector quantization. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, ACM Press/Addison-Wesley Publishing Co., USA, SIGGRAPH ’00, p 479-488, https://doi.org/10.1145/344779.345009,
• Williams, R. J. (1992). Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8(3–4), 229–256. https://doi.org/10.1007/BF00992696
  Article MATH Google Scholar
• Xie, C., Liu, S., Li, C., Cheng, M., Zuo, W., Liu, X., Wen, S., & Ding, E. (2019). Image inpainting with learnable bidirectional attention maps. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp 8857–8866
• Xie, J., Xu, L., & Chen, E. (2012). Image denoising and inpainting with deep neural networks. In: Proceedings of the 25th international conference on neural information processing systems - volume 1, curran associates inc., Red Hook, NY, USA, NIPS’12, p 341-349
• Xiong, W., Yu, J., Lin, Z., Yang, J., Lu, X., Barnes, C., & Luo, J. (2019). Foreground-aware image inpainting. In: The IEEE conference on computer vision and pattern recognition (CVPR)
• Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhudinov, R., Zemel, R., & Bengio, Y. (2015). Show, attend and tell: Neural image caption generation with visual attention. In: International conference on machine learning, pp 2048–2057
• Xu, Z., & Sun, J. (2010). Image inpainting by patch propagation using patch sparsity. IEEE Transactions on Image Processing, 19(5), 1153–1165.
  Article MathSciNet Google Scholar
• Yan, Z., Li, X., Li, M., Zuo, W., & Shan, S. (2018). Shift-net: Image inpainting via deep feature rearrangement. In: The European Conference on Computer Vision (ECCV)
• Yang, C., Lu, X., Lin, Z., Shechtman, E., Wang, O., & Li, H. (2016). High-resolution image inpainting using multi-scale neural patch synthesis. arXiv preprint arXiv:1611.09969v2
• Yao, Y., Ren, J., Xie, X., Liu, W., Liu, Y.J., & Wang, J. (2019). Attention-aware multi-stroke style transfer. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1467–1475
• Yeh, R.A., Chen, C., Lim, T.Y., Schwing, A.G., Hasegawa-Johnson, M., & Do, M.N. (2017). Semantic image inpainting with deep generative models. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 6882–6890
• Yi, Z., Tang, Q., Azizi, S., Jang, D., & Xu, Z. (2020). Contextual residual aggregation for ultra high-resolution image inpainting. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 7508–7517
• Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., & Huang, T.S. (2018). Generative image inpainting with contextual attention. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, pp 5505–5514
• Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., & Huang, T. (2019). Free-form image inpainting with gated convolution. In: 2019 IEEE/CVF international conference on computer vision (ICCV), pp 4470–4479
• Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., Yang, M.H., & Shao, L. (2021). Multi-stage progressive image restoration. In: CVPR
• Zhang, H., Goodfellow, I., Metaxas, D., & Odena, A. (2019). Self-attention generative adversarial networks. In: Proceedings of the 36th international conference on machine learning, PMLR, long beach, California, USA, Proceedings of machine learning research, vol 97, pp 7354–7363, http://proceedings.mlr.press/v97/zhang19d.html
• Zhang, R., Isola, P., Efros, A.A., Shechtman, E., & Wang, O. (2018). The unreasonable effectiveness of deep features as a perceptual metric. In: CVPR
• Zheng, C., Cham, T.J., & Cai, J. (2019). Pluralistic image completion. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp 1438–1447
• Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., & Torralba, A. (2017). Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(6), 1452–1464.
  Article Google Scholar
• Zhou, W., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612.

Download references