Style Transfer Generator for Dataset Testing Classification (original) (raw)
Related papers
Revisiting Artistic Style Transfer for Data Augmentation in A Real-Case Scenario
2022 IEEE International Conference on Image Processing (ICIP)
A tremendous number of techniques have been proposed to transfer artistic style from one image to another. In particular, techniques exploiting neural representation of data; from Convolutional Neural Networks to Generative Adversarial Networks. However, most of these techniques do not accurately account for the semantic information related to the objects present in both images or require a considerable training set. In this paper, we provide a data augmentation technique that is as faithful as possible to the style of the reference artist, while requiring as few training samples as possible, as artworks containing the same semantics of an artist are usually rare. Hence, this paper aims to improve the state-of-the-art by first applying semantic segmentation on both images to then transfer the style from the painting to a photo while preserving common semantic regions. The method is exemplified on Van Gogh's paintings, shown to be challenging to segment.
Unpaired High-Resolution and Scalable Style Transfer Using Generative Adversarial Networks
2018
Neural networks have proven their capabilities by outperforming many other approaches on regression or classification tasks on various kinds of data. Other astonishing results have been achieved using neural nets as data generators, especially in settings of generative adversarial networks (GANs). One special application is the field of image domain translations. Here, the goal is to take an image with a certain style (e.g. a photography) and transform it into another one (e.g. a painting). If such a task is performed for unpaired training examples, the corresponding GAN setting is complex, the neural networks are large, and this leads to a high peak memory consumption during, both, training and evaluation phase. This sets a limit to the highest processable image size. We address this issue by the idea of not processing the whole image at once, but to train and evaluate the domain translation on the level of overlapping image subsamples. This new approach not only enables us to tran...
STaDA: Style Transfer as Data Augmentation
Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications
The success of training deep Convolutional Neural Networks (CNNs) heavily depends on a significant amount of labelled data. Recent research has found that neural style transfer algorithms can apply the artistic style of one image to another image without changing the latter's high-level semantic content, which makes it feasible to employ neural style transfer as a data augmentation method to add more variation to the training dataset. The contribution of this paper is a thorough evaluation of the effectiveness of the neural style transfer as a data augmentation method for image classification tasks. We explore the state-of-the-art neural style transfer algorithms and apply them as a data augmentation method on Caltech 101 and Caltech 256 dataset, where we found around 2% improvement from 83% to 85% of the image classification accuracy with VGG16, compared with traditional data augmentation strategies. We also combine this new method with conventional data augmentation approaches to further improve the performance of image classification. This work shows the potential of neural style transfer in computer vision field, such as helping us to reduce the difficulty of collecting sufficient labelled data and improve the performance of generic image-based deep learning algorithms.
Style synthesizing conditional generative adversarial networks
2020
Neural style transfer (NST) models aim to transfer a particular visual style to a image while preserving its content using neural networks. Style transfer models that can apply arbitrary styles without requiring style-specific models or architectures are called universal style transfer (UST) models. Typically a UST model takes a content image and a style image as inputs and outputs the corresponding stylized image. It is, therefore, required to have a style image with the required characteristics to facilitate the transfer. However, in practical applications, where the user wants to apply variations of a style class or a mixture of multiple style classes, such style images may be difficult to find or simply non-existent.
Improved ArtGAN for Conditional Synthesis of Natural Image and Artwork
IEEE Transactions on Image Processing, 2019
This paper proposes a series of new approaches to improve Generative Adversarial Network (GAN) for conditional image synthesis and we name the proposed model as "ArtGAN". One of the key innovation of ArtGAN is that, the gradient of the loss function w.r.t. the label (randomly assigned to each generated image) is back-propagated from the categorical discriminator to the generator. With the feedback from the label information, the generator is able to learn more efficiently and generate image with better quality. Inspired by recent works, an autoencoder is incorporated into the categorical discriminator for additional complementary information. Last but not least, we introduce a novel strategy to improve the image quality. In the experiments, we evaluate ArtGAN on CIFAR-10 and STL-10 via ablation studies. The empirical results showed that our proposed model outperforms the state-of-the-art results on CIFAR-10 in terms of Inception score. Qualitatively, we demonstrate that ArtGAN is able to generate plausible-looking images on Oxford-102 and CUB-200, as well as able to draw realistic artworks based on style, artist, and genre. The source code and models are available at: https://github.com/cs-chan/ArtGAN.
Generative AI Model for Artistic Style Transfer Using Convolutional Neural Networks
Journal of Computer Science and Technology Studies
Artistic style transfer, a captivating application of generative artificial intelligence, involves fusing the content of one image with the artistic style of another to create unique visual compositions. This paper presents a comprehensive overview of a novel technique for style transfer using Convolutional Neural Networks (CNNs). By leveraging deep image representations learned by CNNs, we demonstrate how to separate and manipulate image content and style, enabling the synthesis of high-quality images that combine content and style in a harmonious manner. We describe the methodology, including content and style representations, loss computation, and optimization, and showcase experimental results highlighting the effectiveness and versatility of the approach across different styles and content.
IEEE Access, 2021
In this paper, we tackle the well-known problem of dataset construction from the point of its generation using generative adversarial networks (GAN). As semantic information of the dataset should have a proper alignment with images, controlling the image generation process of GAN comes to the first position. Considering this, we focus on conditioning the generative process by solely utilizing conditional information to achieve reliable control over the image generation. Unlike the existing works that consider the input (noise or image) in conjunction with conditions, our work considers transforming the input directly to the conditional space by utilizing the given conditions only. By doing so, we reveal the relations between conditions to determine their distinct and reliable feature space without the impact of input information. To fully leverage the conditional information, we propose a novel architectural framework (i.e., conditional transformation) that aims to learn features only from a set of conditions for guiding a generative model by transforming the input to the generator. Such an approach enables controlling the generator by setting its inputs according to the specific conditions necessary for semantically correct image generation. Given that the framework operates at the initial stage of generation, it can be plugged into any existing generative models and trained in an end-to-end manner together with the generator. Extensive experiments on various tasks, such as novel image synthesis and image-to-image translation, demonstrate that the conditional transformation of inputs facilitates solid control over the image generation process and thus shows its applicability for use in dataset construction.
ArtGAN: Artwork synthesis with conditional categorical GANs
2017 IEEE International Conference on Image Processing (ICIP), 2017
This paper proposes an extension to the Generative Adversarial Networks (GANs), namely as ARTGAN to synthetically generate more challenging and complex images such as artwork that have abstract characteristics. This is in contrast to most of the current solutions that focused on generating natural images such as room interiors, birds, flowers and faces. The key innovation of our work is to allow backpropagation of the loss function w.r.t. the labels (randomly assigned to each generated images) to the generator from the discriminator. With the feedback from the label information, the generator is able to learn faster and achieve better generated image quality. Empirically, we show that the proposed ARTGAN is capable to create realistic artwork, as well as generate compelling real world images that globally look natural with clear shape on CIFAR-10.
edge2art: Edges to Artworks Translation with Conditional Generative Adversarial Networks
2019
This paper presents an application of the pix2pix model [3], which presents a solution to the image to image translation problem by using cGANs. The main objective of our research consists in the evaluation of several artificial artworks that were generated by the cGAN, taking a scribble of edges as input. This evaluation covers different artistic movements and art styles such as Rococo, Ukiyo-e, Fauvism and Cubism. The set of the trained models of these different styles is called edge2art. Each art style was trained over more than 2000 artworks examples taken from the wikiart dataset used in ArtGAN [4]-[5]. The experiments consists in giving scribbles images to the cGAN, and depending on the selected style, the network will give an colored and stylized artwork as output. Comparison between the generated artworks and the target artworks are measured by Mean Squared Error and Structural Similarity Measure.
GANmera: Reproducing Aesthetically Pleasing Photographs Using Deep Adversarial Networks
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
Generative adversarial networks (GANs) have become increasingly popular in recent years owing to its ability to synthesize and transfer. The image enhancement task can also be modeled as an image-to-image translation problem. In this paper, we propose GANmera, a deep adversarial network which is capable of performing aesthetically-driven enhancement of photographs. The network adopts a 2-way GAN architecture and is semi-supervised with aestheticbased binary labels (good and bad). The network is trained with unpaired image sets, hence eliminating the need for strongly supervised before-after pairs. Using CycleGAN as the base architecture, several fine-grained modifications are made to the loss functions, activation functions and resizing schemes, to achieve improved stability in the generator. Two training strategies are devised to produce results with varying aesthetic output. Quantitative evaluation on the recent benchmark MIT-Adobe-5K dataset demonstrate the capability of our method in achieving state-of-the-art PSNR results. We also show qualitatively that the proposed approach produces aesthetically-pleasing images. This work is a shortlisted submission to the CVPR 2019 NTIRE Image Enhancement Challenge.