Convolutional Neural Networks as a Computational Model for the Underlying Processes of Aesthetics Perception (original) (raw)
Related papers
Learning Photography Aesthetics with Deep CNNs
arXiv (Cornell University), 2017
Automatic photo aesthetic assessment is a challenging arti cial intelligence task. Existing computational approaches have focused on modeling a single aesthetic score or class (good or bad photo), however these do not provide any details on why the photograph is good or bad; or which attributes contribute to the quality of the photograph. To obtain both accuracy and human-interpretability, we advocate learning the aesthetic attributes along with the prediction of the overall score. For this purpose, We propose a novel multitask deep convolution neural network (DCNN), which jointly learns eight aesthetic attributes along with the overall aesthetic score. We report near-human performance in the prediction of the overall aesthetic score. To understand the internal representation of these attributes in the learned model, we also develop the visualization technique using back propagation of gradients. These visualizations highlight the important image regions for the corresponding attributes, thus providing insights about model's understanding of these attributes. We showcase the diversity and complexity associated with di erent attributes through a qualitative analysis of the activation maps.
Measuring photography aesthetics with deep CNNs
IET Image Processing, 2020
In spite of the recent advancements of deep learning based techniques, automatic photo aesthetic assessment still remains a challenging computer vision task. Existing approaches used to focus on providing a single aesthetic score or category ("good" or "bad") of photograph, rather than quantifying "goodness" or "badness". The existing algorithms often ignore the importance of different attributes contributing to the artistic quality of the photograph. To obtain the human-interpretability of aesthetic score of photo, we advocate learning the aesthetic attributes alongwith the prediction of the general aesthetic score. We propose a multi-task deep CNN, that collectively learns aesthetic attributes alongwith a general aesthetic score for the photograph. To understand the mathematical representation of the attributes in the proposed model, a visualization technique is proposed using back propagation of gradients. These visualization of attributes correspond to the location of objects in the images in order to find out which part of an image "triggers" the classification outcome, thus providing the insights about the model's understanding of these attributes. This paper proposes an aesthetic feature vector based on the relative foreground position of the object in the image. The proposed aesthetic features outperform the state-of-art methods especially for Rule of Thirds attribute.
Brain-Inspired Deep Networks for Image Aesthetics Assessment
ArXiv, 2016
Image aesthetics assessment has been challenging due to its subjective nature. Inspired by the scientific advances in the human visual perception and neuroaesthetics, we design Brain-Inspired Deep Networks (BDN) for this task. BDN first learns attributes through the parallel supervised pathways, on a variety of selected feature dimensions. A high-level synthesis network is trained to associate and transform those attributes into the overall aesthetics rating. We then extend BDN to predicting the distribution of human ratings, since aesthetics ratings are often subjective. Another highlight is our first-of-its-kind study of label-preserving transformations in the context of aesthetics assessment, which leads to an effective data augmentation approach. Experimental results on the AVA dataset show that our biological inspired and task-specific BDN model gains significantly performance improvement, compared to other state-of-the-art models with the same or higher parameter capacity.
Image Aesthetics Assessment using Deep Learning (based on High Level Image Attributes)
International journal of engineering research and technology, 2020
Aesthetic assessment of images has been getting a lot of attention for the past decade in the field of computer vision. Large amounts of social media and advertising data in the form of images is continuously analyzed to assign it an aesthetic quality value to improve businesses as well as for gaining more popularity across the web. Visual perception by humans cannot be fully replicated by a machine and continuously more work is being published on aesthetic classification of images. In this paper, we have presented a convolutional neural network model which automatically extracts high level features and distinguishes a set of images into pleasing and non-pleasing categories. Our dataset has been compiled from a variety of sources on the web to make it as diverse as possible. Compared to the traditional handcrafted methods and other machine learning models, our CNN model has provided a better classification accuracy of 68% on our dataset.
IJERT-Image Aesthetics Assessment using Deep Learning (based on High Level Image Attributes)
International Journal of Engineering Research and Technology (IJERT), 2020
https://www.ijert.org/image-aesthetics-assessment-using-deep-learning-based-on-high-level-image-attributes https://www.ijert.org/research/image-aesthetics-assessment-using-deep-learning-based-on-high-level-image-attributes-IJERTCONV8IS05028.pdf Aesthetic assessment of images has been getting a lot of attention for the past decade in the field of computer vision. Large amounts of social media and advertising data in the form of images is continuously analyzed to assign it an aesthetic quality value to improve businesses as well as for gaining more popularity across the web. Visual perception by humans cannot be fully replicated by a machine and continuously more work is being published on aesthetic classification of images. In this paper, we have presented a convolutional neural network model which automatically extracts high level features and distinguishes a set of images into pleasing and non-pleasing categories. Our dataset has been compiled from a variety of sources on the web to make it as diverse as possible. Compared to the traditional handcrafted methods and other machine learning models, our CNN model has provided a better classification accuracy of 68% on our dataset.
Artificial intelligence has emerged as a powerful computational tool to create artworks. One application is Neural Style Transfer, which allows to transfer the style of one image, such as a painting, onto the content of another image, such as a photograph. In the present study, we ask how Neural Style Transfer affects objective image properties and how beholders perceive the novel (style-transferred) stimuli. In order to focus on the subjective perception of artistic style, we minimized the confounding effect of cognitive processing by eliminating all representational content from the input images. To this aim, we transferred the styles of 25 diverse abstract paintings onto 150 colored random-phase patterns with six different Fourier spectral slopes. This procedure resulted in 150 style-transferred stimuli. We then computed eight statistical image properties (complexity, self-similarity, edge-orientation entropy, variances of neural network features, and color statistics) for each image. In a rating study, we asked participants to evaluate the images along three aesthetic dimensions (Pleasing, Harmonious, and Interesting). Results demonstrate that not only objective image properties, but also subjective aesthetic preferences transferred from the original artworks onto the styletransferred images. The image properties of the style-transferred images explain 50-69% of the variance in the ratings. In the multidimensional space of statistical image properties, participants considered style-transferred images to be more Pleasing and Interesting if they were closer to a "sweet spot" where traditional Western paintings (JenAesthetics dataset) are represented. We conclude that NST is a useful tool to create novel artistic stimuli that preserve the image properties of the input style images. In the novel stimuli, we found a strong relationship between statistical image properties and subjective ratings, suggesting a prominent role of perceptual processing in the aesthetic evaluation of abstract images.
Image Aesthetics Assessment Using Multi Channel Convolutional Neural Networks
Computer Vision and Image Processing, 2020
Image Aesthetics Assessment is one of the emerging domains in research. The domain deals with classification of images into categories depending on the basis of how pleasant they are for the users to watch. In this article, the focus is on categorizing the images in high quality and low quality image. Deep convolutional neural networks are used to classify the images. Instead of using just the raw image as input, different crops and saliency maps of the images are also used, as input to the proposed multi channel CNN architecture. The experiments reported on widely used AVA database show improvement in the aesthetic assessment performance over existing approaches.
Image Aesthetic Assessment: A Comparative Study of Hand-Crafted & Deep Learning Models
IEEE Access
Automatic image aesthetics assessment is a computer vision problem dealing with categorizing images into different aesthetic levels. The categorization is usually done by analyzing an input image and computing some measure of the degree to which the image adheres to the fundamental principles of photography such as balance, rhythm, harmony, contrast, unity, look, feel, tone, and texture. Due to its diverse applications in many areas, automatic image aesthetic assessment has gained significant research attention in recent years. This article presents a comparative study of different automatic image aesthetics assessment techniques from the year 2005 to 2021. A number of conventional hand-crafted as well as modern deep learning-based approaches are reviewed and analyzed for their performance on various publicly available datasets. Additionally, critical aspects of different features and models have also been discussed to analyze their performance and limitations in different situations. The comparative analysis reveals that deep learning based approaches excel hand-crafted based techniques in image aesthetic assessment.
Hierarchical aesthetic quality assessment using deep convolutional neural networks
Signal Processing: Image Communication, 2016
Aesthetic image analysis has attracted much attention in recent years. However, assessing the aesthetic quality and assigning an aesthetic score are challenging problems. In this paper, we propose a novel framework for assessing the aesthetic quality of images. Firstly, we divide the images into three categories: "scene", "object" and "texture". Each category has an associated convolutional neural network (CNN) which learns the aesthetic features for the category in question. The object CNN is trained using the whole images and a salient region in each image. The texture CNN is trained using small regions in the original images. Furthermore, an A&C CNN is developed to simultaneously assess the aesthetic quality and identify the category for overall images. For each CNN, classification and regression models are developed separately to predict aesthetic class (high or low) and to assign an aesthetic score. Experimental results on a recently published large-scale dataset show that the proposed method can outperform the state-of-the-art methods for each category.
Designing Deep Leaning frameworks for Image Aesthetics
International Journal For Research In Applied Science & Engineering Technology, 2020
Aesthetic is the measure or appreciation of beauty. In photography, it usually means that an image appeals to the eye. The existing method of machine learning is used to analyze the image aesthetics but this method has limited to the rules of photography. That's why we decided to develop an application to find whether the image is aesthetically pleasing or not and rate image Aesthetics using Convolutional Neural Network(CNN) models. In our project, we are using a double CNN module. We build our dataset for first CNN and FLICKER-AES dataset is used to train second CNN. Dataset generation for cnn1 is done by using a handcrafted mechanism and deep learning approach, outliers are removed manually. In the handcrafted mechanism, we have used various photography rules (Rule of Third, Figure to Ground, Depth of Field) in this mechanism. The CNN-1 model is trained using the generated dataset of labeled images to classify the image into one of six classes. The CNN-2 model is trained using the dataset of rated images (FLICKER-AES) to rate images based on their aesthetics. This trained CNN model helps us to rate our images. We use CNN because it is very effective in areas such as image recognition and classification. The main goal of this project is to classify the image into 5 photography rules to suggest improvements in the image and rate image aesthetics.