Discerning the painter’s hand: machine learning on surface topography (original) (raw)
Related papers
PEARC '20: Practice and Experience in Advanced Research Computing, 2020
Art historians are interested in possible methods and visual criteria for determining the style and authorship of artworks. One approach, developed by Giovanni Morelli in the late nineteenth century, focused on abstracting, extracting and comparing details of recognizable human forms, although he never prescribed what exactly to look for. In this work, we asked what could a contemporary method like convolution networks contribute or reveal about such a humanistic method that is not fully determined, but that is also so clearly aligned with computation? Convolution networks have become very successful in object recognition because they learn general features to distinguish and classify large sets of objects. Thus, we wanted to explore what features are present in these networks that have some discriminatory power for distinguishing paintings. We input the digitized art into a large-scale convolutional network that was pre-trained for object recognition from naturalistic images. Because we do not have labels, we extracted activations from the network and ran K-means clustering. We contrasted and evaluated discriminatory power between shallow and deeper layers. We also compared predetermined features from standard computer vision techniques of edge detection. It turns out that the deep network individual feature maps are highly generic and do not easily map onto obvious authorship interpretations, but in the aggregate can have strong discriminating power that are intuitive. Although this does not directly test issues of attribution, the application can inform humanistic perspectives regarding what counts as features that make up visual elements of paintings.
Deep transfer learning for visual analysis and attribution of paintings by Raphael
Heritage Science, 2023
Visual analysis and authentication of artworks are challenging tasks central to art history and criticism. This preliminary study presents a computational tool for scholars examining and authenticating a restricted class of paintings, with a specific focus on the paintings of Raffaello Sanzio da Urbino, more popularly known as Raphael. We applied transfer learning to the ResNet50 deep neural network for feature extraction and used a support vector machine (SVM) binary classifier in support of authentication. Edge detection and analysis algorithms, considered to be crucial for capturing the essence of Raphael's artistic style, including the brushwork signatures, were also integrated and are used as an authentication tool. The machine learning approach we have developed demonstrates an accuracy of 98% in image-based classification tasks during validation using a test set of well known and authentic paintings by Raphael. Of course, a full authentication protocol relies on provenance, history, material studies, iconography, studies of a work's condition, and more. Our work, then, contributes to just a portion of a full authentication protocol. Our findings suggest that machine learning methods, properly employed by experts aware of context, may enhance and expand traditional visual analysis for problems in art authentication.