Two-Stage Deep Learning Approach to the Classification of Fine-Art Paintings (original) (raw)
Related papers
Towards automated classification of fine-art painting style: A comparative study
Proceedings of the 21st International Conference on Pattern Recognition, 2012
This thesis presents a comparative study of different classification methodologies for the task of fine-art genre classification. The problem of painting classification involves classifying new unknown paintings among different art genres. Two-level comparative study is performed for this classification problem. The first level reviews the performance of discriminative vs. generative models while the second level touches the features aspect of the paintings and compares Semantic-level features vs low-level and intermediate-level features present in the painting. Three models are studied and compared, namely-1) A Discriminative model using a Bag-of-Words (BoW) approach; 2) A Generative model using BoW; 3) Discriminative model using Semantic-level features. Various experiments and techniques like Bag of Words model, Topic models and Classeme features are employed to get insights into potential of these automatic classification techniques for painting styles.
Genre classification of paintings
2016 International Symposium ELMAR, 2016
Extensive digitization efforts in the recent years have led to a large increase of digitized and online available fine-art collections. With digitization of artworks, we aim to preserve all those valuable evidences of various human creative expressions, as well as make them available to a broader audience. The digitalization process of artworks should not constrain only to fulfilling the purpose of preservation, but also serve as a starting point for exploring of this type of data in a novel way, which is made possible with the rise of new achievements in computer vision. In the domain of computer analysis of visual art there are various ongoing research challenges. In this paper, we explore different image feature extraction methods and their applicability in the task of classifying painting by genre. Our dataset includes paintings of various styles grouped in seven genre categories. We achieved an accuracy of 77.57% for the task of genre classification. We concluded that the best performance is achieved when using features derived from a pretrained deep convolutional neural network.
A Deep Approach for Classifying Artistic Media from Artworks
KSII Transactions on Internet and Information Systems, 2019
We present a deep CNN-based approach for classifying artistic media from artwork images. We aim to classify most frequently used artistic media including oilpaint brush, watercolor brush, pencil and pastel, etc. For this purpose, we extend VGGNet, one of the most widely used CNN structure, by substituting its last layer with a fully convolutional layer, which reveals class activation map (CAM), the region of classification. We build two artwork image datasets: YMSet that collects more than 4K artwork images for four most frequently used artistic media from various internet websites and WikiSet that collects almost 9K artwork images for ten most frequently used media from WikiArt. We execute a human baseline experiment to compare the classification performance. Through our experiments, we conclude that our classifier is superior in classifying artistic media to human.
PEARC '20: Practice and Experience in Advanced Research Computing, 2020
Art historians are interested in possible methods and visual criteria for determining the style and authorship of artworks. One approach, developed by Giovanni Morelli in the late nineteenth century, focused on abstracting, extracting and comparing details of recognizable human forms, although he never prescribed what exactly to look for. In this work, we asked what could a contemporary method like convolution networks contribute or reveal about such a humanistic method that is not fully determined, but that is also so clearly aligned with computation? Convolution networks have become very successful in object recognition because they learn general features to distinguish and classify large sets of objects. Thus, we wanted to explore what features are present in these networks that have some discriminatory power for distinguishing paintings. We input the digitized art into a large-scale convolutional network that was pre-trained for object recognition from naturalistic images. Because we do not have labels, we extracted activations from the network and ran K-means clustering. We contrasted and evaluated discriminatory power between shallow and deeper layers. We also compared predetermined features from standard computer vision techniques of edge detection. It turns out that the deep network individual feature maps are highly generic and do not easily map onto obvious authorship interpretations, but in the aggregate can have strong discriminating power that are intuitive. Although this does not directly test issues of attribution, the application can inform humanistic perspectives regarding what counts as features that make up visual elements of paintings.
Fine-tuning Convolutional Neural Networks for fine art classification
Expert Systems with Applications
The increasing availability of large digitized fine art collections opens new research perspectives in the intersection of artificial intelligence and art history. Motivated by the successful performance of Convolutional Neural Networks (CNN) for a wide variety of computer vision tasks, in this paper we explore their applicability for art-related image classification tasks. We perform extensive CNN fine-tuning experiments and consolidate in one place the results for five different art-related classification tasks on three large fine art datasets. Along with addressing the previously explored tasks of artist, genre, style and time period classification, we introduce a novel task of classifying artworks based on their association with a specific national artistic context. We present state-of-the-art classification results of the addressed tasks, signifying the impact of our method on computational analysis of art, as well as other image classification related research areas. Furthermore, in order to question transferability of deep representations across various source and target domains, we systematically compare the effects of domain-specific weight initialization by evaluating networks pre-trained for different tasks, varying from object and scene recognition to sentiment and memorability labelling. We show that fine-tuning networks pre-trained for scene recognition and sentiment prediction yields better results than fine-tuning networks pre-trained for object recognition. This novel outcome of our work suggests that the semantic correlation between different domains could be inherent in the CNN weights. Additionally, we address the practical applicability of our results by analysing different aspects of image similarity. We show that features derived from fine-tuned networks can be employed to retrieve images similar in either style or content, which can be used to enhance capabilities of search systems in different online art collections.
A Closer Look at Art Mediums: The MAMe Image Classification Dataset
ArXiv, 2020
Art is an expression of human creativity, skill and technology. An exceptionally rich source of visual content. In the context of AI image processing systems, artworks represent one of the most challenging domains conceivable: Properly perceiving art requires attention to detail, a huge generalization capacity, and recognizing both simple and complex visual patterns. To challenge the AI community, this work introduces a novel image classification task focused on museum art mediums, the MAMe dataset. Data is gathered from three different museums, and aggregated by art experts into 29 classes of medium (i.e. materials and techniques). For each class, MAMe provides a minimum of 850 images (700 for training) of high-resolution and variable shape. The combination of volume, resolution and shape allows MAMe to fill a void in current image classification challenges, empowering research in aspects so far overseen by the research community. After reviewing the singularity of MAMe in the cont...
Deep Learning Approaches to Art Style Recognition in
2017
Foreword Computation for the work described in this report was supported by the DeiC National HPC Centre, SDU. The authors will be referring to themselves in first person plural. Counselling and the original project proposal has been delivered by Manfred Jaeger.
Classification of basic artistic media based on a deep convolutional approach
The Visual Computer, 2019
Artistic media play an important role in recognizing and classifying artworks in many artwork classification works and public artwork databases. We employ deep CNN structure to recognize artistic media from artworks and to classify them into predetermined categories. For this purpose, we define basic artistic media as oilpaint brush, pastel, pencil and watercolor and build artwork image dataset by collecting artwork images from various websites. To build our classifier, we implement various recent deep CNN structures and compare their performances. Among them, we select DenseNet, which shows best performance for recognizing artistic media. Through the human baseline experiment, we show that the performance of our classifier is compatible with that of trained human. Furthermore, we also show that our classifier shows a similar recognition and classification pattern with human in terms of well-classifying media, ill-classifying media, confusing pair and confusing case. We also collect synthesized oilpaint artwork images from fourteen important oilpaint literatures and apply them to our classifier. Our classifier shows a meaningful performance, which will lead to an evaluation scheme for the artistic media simulation techniques of non-photorealistic rendering (NPR) society.
Classification of Fine Art Oil Paintings by Semantic Category
In this paper we explore supervised learning techniques that are able to classify fine-art paintings by the general subject matter of the painting (e.g., landscape, people, seascape and still life). Classifying art paintings by semantic category may pose unique challenges because art is subjective and highly interpretive. State-of-the-art feature extraction and encoding techniques used for object and scene recognition in photographic images are evaluated for their potential use for classifying art paintings. In this work we evaluate several types of features individually and also in combinations to reveal the benefit of complimentary information. Feature ranking techniques are implemented as a means for identifying the most important features between any two labels in the data set. In order to compute a final ranking of the most relevant features for all class labels, a metric is proposed using the principal components of the combined feature vector to prioritize the final ranking of the top 'k' features across all class label pairs. Classification algorithms used for evaluation include a soft margin linear SVM and L-2 regularized logistic regression. Experimental results show that several feature classes can be successfully used to classify art paintings with improved accuracy when multiple features are combined, ranked and prioritized to form a final feature vector.