Akash Gupta, Niluthpol Mithun, Conrad Rudolph, and Amit K. Roy-Chowdhury, "Deep Learning Based Identity Verification in Renaissance Portraits," IEEE International Conference on Multimedia and Expo (ICME) 2018 [n.p.]. (original) (raw)
Related papers
In this work, we explore the feasibility of face recognition technologies for analyzing works of portraiture, and in the process provide a quantitative source of evidence to art historians in answering many of their ambiguities concerning identity of the subject in some portraits and in understanding artists' styles. Works of portrait art bear the mark of visual interpretation of the artist. Moreover, the number of samples available to model these effects are often limited. Based on an understanding of artistic conventions, we show how to learn and validate features that are robust in distinguishing subjects in portraits (sitters) and that are also capable of characterizing an individual artist's style. This can be used to learn a feature space called Portrait Feature Space (PFS) that is representative of quantitative measures of similarities between portrait pairs known to represent same/different sitters. Through statistical hypothesis tests we analyze uncertain portraits against known identities and explain the significance of the results from an art historian's perspective. Results are shown on our data consisting of over 270 portraits belonging largely to the Renaissance era.
FACES (Faces, Art, and Computerized Evaluation Systems) is a project that, after two years of research support from the National Endowment for the Humanities (NEH), has established proof of concept for the application of face recognition technology to works of portrait art. In the application of face recognition technology to photographed human faces, a number of difficulties are inherent in a real or perceived alteration of appearance of the face through variations in facial expression, age, angle of pose, and so on. With works of portrait art, not only do all these problems pertain, but these works also have their own additional challenges. Most notably, portrait art does not provide what might be called a photographic likeness but rather one that goes through a process of visual interpretation on the part of the artist. After establishing the initial parameters of the application of this technology, the main goal of FACES has been to test the ability of the FACES algorithm to restore lost identities to works of portrait art, something our research has shown is clearly feasible. Our work has also suggested a number of other potential applications, both using the FACES algorithm and employing basic concept of FACES in an altered form.For example an altered form of the technology used in FACES might also be used to study a wide range of other applications such as adherence or non-adherence to widely recognized artistic canons, formal or informal; the identification of variations in the practice of an individual artist (over time, with different subjects, with different genres, after exposure to external influences, and so on); probable bodies of work of anonymous artists; difference in larger bodies of works (art historical "big data"); and even to detect the change of masons in medieval building. FACES was conceived of by myself and I am Project Director and Principal Investigator. The FACES project is a collaboration of the humanities (art history) and the sciences (computer science). This article, of which I am the author, presents FACES from the point of view of the humanities, that is, how this technology generally works, what the parameters of its application to portrait art are at this time, what its advantages are, and so on. The computer science basis of the study has appeared in a leading computer science journal [Ramya Srinivasan, Conrad Rudolph, and Amit Roy-Chowdhury, "Computerized Face Recognition in Renaissance Portrait Art," Signal Processing Magazine 32:4 (July 2015) 85-94]. These two papers are meant to operate as a pair.
Renaissance portraits were depictions of some important royals of those times. Analysis of faces in these portraits can provide valuable dynastical information in addition to enriching personal details of the depicted sitter. Such studies can offer insights to the art-history community in understanding and linking personal histories. In particular, face recognition technologies can be useful for identifying subjects when there is ambiguity. However, portraits are subject to several complexities such as aesthetic sensibilities of the artist or social standing of the sitter. Thus, for robust automated face recognition, it becomes important to model the characteristics of the artist. In this paper, we focus on modeling the styles of artists by considering case studies involving Renaissance art-works. After a careful examination of artistic trends, we arrive at relevant features for analysis. From a set of instances known to match/not match, we learn distributions of match and non-match scores which we collectively refer to as the portrait feature space (PFS). Thereafter, using statistical permutation tests we learn which of the chosen features were emphasized in various works involving (a) same artist depicting same sitter, (b) same sitter but by different artists and (c) same artist but depicting different sitters. Finally, we show that the knowledge of these specific choices can provide valuable information regarding the sitter and/or artist. 1
We present a work that explores the feasibility of automated face recognition technologies for analyzing identities in works of portraiture, and in the process provide additional evidence to settle some long-standing questions in art history. Works of portrait art bear the mark of visual interpretation of the artist. Moreover, the number of samples available to model these effects is often limited. From a set of portraiture of the Renaissance and Baroque periods, where the identities of subjects are known, we derive appropriate features that are based on domain knowledge of artistic renderings, and learn and validate statistical models for the distribution of the match and non-match scores, which we refer to as portrait feature space (PFS). Thereafter, we use this PFS on a number of cases that have been "open questions" to art historians. They are usually in the form of validating two portraits as belonging to the same person. Using statistical hypothesis tests on the PFS, we provide quantitative measures of similarity for each of these questions. It is, to the best of our knowledge, the first study that applies automated face recognition technologies to the analysis of portraits of multiple subjects in various formspaintings, death masks, sculptures.
FACES , in: artibus et historiae no. 75 (XXXVIII), 2017
FACES (Faces, Art, and Computerized Evaluation Systems) is a project that, after two years of research support from the National Endowment for the Humanities (NEH), has established proof of concept for the application of face recognition technology to works of portrait art. In the application of face recognition technology to photographed human faces, a number of difficulties are inherent in a real or perceived alteration of appearance of the face through variations in facial expression, age, angle of pose, and so on. With works of portrait art, not only do all these problems pertain, but these works also have their own additional challenges. Most notably, portrait art does not provide what might be called a photographic likeness but rather one that goes through a process of visual interpretation on the part of the artist. After establishing the initial parameters of the application of this technology, the main goal of FACES has been to test the ability of the FACES algorithm to restore lost identities to works of portrait art, something our research has shown is clearly feasible. Our work has also suggested a number of other potential applications, both using the FACES algorithm and employing basic concept of FACES in an altered form. The use of the FACES algorithm should not be thought of as limited to facial recognition in the sense of identification alone. An altered form of the technology used in FACES might also be used to study a wide range of other applications such as adherence or non-adherence to widely recognized artistic canons, formal or informal; the identification of variations in the practice of an individual artist (over time, with different subjects, with different genres, after exposure to external influences, and so on); probable bodies of work of anonymous artists; difference in larger bodies of works (art historical ‘big data’); even to detect the change of masons in medieval building.
2022
Portraits of Roman emperors are traditionally recognised by their unique coiffure patterns, a method that runs the risk of ignoring portraits that do not cohere to the standardised image of the emperor. This article investigates whether it is possible to recognise and distinguish emperors using the facial features of their portraits. By using a technique called transfer learning, it utilises existing deep-learning facial recognition models, augmented with images of Roman imperial portraits, to provide a new empirical foothold in the debate of Roman emperor recognition. The results of the experiments demonstrate that by only a limited amount of training, such a so-called "pre-trained" model (i.e., InceptionResnet-V1) is able to correctly classify most images in the dataset of Roman emperors. As such, this article has made a first step towards applying facial recognition models to the study of ancient imperial portraiture.
2012
One of the enduring mysteries in the history of the Renaissance is the adult appearance of the archetypical "Renaissance Man," Leonardo da Vinci. His only acknowledged self-portrait is from an advanced age, and various candidate images of younger men are di cult to assess given the absence of documentary evidence. One clue about Leonardo's appearance comes from the remark of the contemporary historian, Vasari, that the sculpture of David by Leonardo's master, Andrea del Verrocchio, was based on the appearance of Leonardo when he was an apprentice. Taking a cue from this statement, we suggest that the more mature sculpture of St. Thomas, also by Verrocchio, might also have been a portrait of Leonardo. We tested the possibility Leonardo was the subject for Verrocchio's sculpture by a novel computational technique for the comparison of three-dimensional facial configurations. Based on quantitative measures of similarities, we also assess whether another pair of candidate two-dimensional images are plausibly attributable as being portraits of Leonardo as a young adult. Our results are consistent with the claim Leonardo is indeed the subject in these works, but we need comparisons with images in a larger corpora of candidate artworks before our results achieve statistical significance.