Prathmesh Madhu - Academia.edu (original) (raw)
Uploads
Papers by Prathmesh Madhu
Background: Interventional workflow analysis can help to increase the quality and efficiency of p... more Background: Interventional workflow analysis can help to increase the quality and efficiency of performed procedures, which are two important factors in the medical domain [ref:1]. A useful tool for analyzing medical workflow is video-based phase annotation of procedures, since the duration[for full text, please go to the a.m. URL]
In the field of Art History, images of artworks and their contexts are core to understanding the ... more In the field of Art History, images of artworks and their contexts are core to understanding the underlying semantic information. However, the highly complex and sophisticated representation of these artworks makes it difficult, even for the experts, to analyze the scene. From the computer vision perspective, the task of analyzing such artworks can be divided into sub-problems by taking a bottom-up approach. In this paper, we focus on the problem of recognizing the characters in Art History. From the iconography of AnnunciationoftheLordAnnunciation of the LordAnnunciationoftheLord (Figure 1), we consider the representation of the main protagonists, MaryMaryMary and GabrielGabrielGabriel, across different artworks and styles. We investigate and present the findings of training a character classifier on features extracted from their face images. The limitations of this method, and the inherent ambiguity in the representation of GabrielGabrielGabriel, motivated us to consider their bodies (a bigger context) to analyze in order to recognize the characters...
The understanding of iconography and visual narration in ancient imagery is one of the main foci ... more The understanding of iconography and visual narration in ancient imagery is one of the main foci in the field of Classical Archaeology, e.g. in Attic vase paintings of the fifth century B.C. In order to depict the situations and actions of a narrative as well as to characterise its protagonists, ancient Greek artists made use of a broad variety of often similar image elements [1]. The interaction and meaningful relationship of the protagonists is depicted with significant postures and gestures (schemata) in order to illustrate key aspects of the storyline [2, 3]. These schemes are not restricted to a certain iconography, so that visual links between different images occur. Being familiar with these relationships the ancient viewer could detect the specific narration and understand the meaning of the image. For example, the scheme of leading the bride in Attic vase paintings is characterised by a significant leading-gesture (χεῖρ' ἐπὶ καρπῷ – hand on wrist / hand on hand) that re...
Heritage Preservation, 2018
ArXiv, 2020
Human pose estimation (HPE) is a central part of understanding the visual narration and body move... more Human pose estimation (HPE) is a central part of understanding the visual narration and body movements of characters depicted in artwork collections, such as Greek vase paintings. Unfortunately, existing HPE methods do not generalise well across domains resulting in poorly recognized poses. Therefore, we propose a two step approach: (1) adapting a dataset of natural images of known person and pose annotations to the style of Greek vase paintings by means of image style-transfer. We introduce a perceptually-grounded style transfer training to enforce perceptual consistency. Then, we fine-tune the base model with this newly created dataset. We show that using style-transfer learning significantly improves the SOTA performance on unlabelled data by more than 6% mean average precision (mAP) as well as mean average recall (mAR). (2) To improve the already strong results further, we created a small dataset (ClassArch) consisting of ancient Greek vase paintings from the 6-5th century BCE w...
Proceedings of the 1st Workshop on Structuring and Understanding of Multimedia heritAge Contents - SUMAC '19, 2019
Figure 1: Art historical scene depicting the iconography called Annunciation of the Lord (left [1... more Figure 1: Art historical scene depicting the iconography called Annunciation of the Lord (left [10], right [32]). Mary and Gabriel are the main protagonists. We can clearly see the differences in the background, in the artistic style, in the foreground, in the objects, their properties, and the use of color.
ArXiv, 2020
Image compositions as a tool for analysis of artworks is of extreme significance for art historia... more Image compositions as a tool for analysis of artworks is of extreme significance for art historians. These compositions are useful in analyzing the interactions in an image to study artists and their artworks. Max Imdahl in his work called Ikonik, along with other prominent art historians of the 20th century, underlined the aesthetic and semantic importance of the structural composition of an image. Understanding underlying compositional structures within images is challenging and a time consuming task. Generating these structures automatically using computer vision techniques (1) can help art historians towards their sophisticated analysis by saving lot of time; providing an overview and access to huge image repositories and (2) also provide an important step towards an understanding of man made imagery by machines. In this work, we attempt to automate this process using the existing state of the art machine learning techniques, without involving any form of training. Our approach,...
One of the first steps to understanding images is by reading their captions. In the field of comp... more One of the first steps to understanding images is by reading their captions. In the field of computer vision and natural language processing, generating captions from images has been an important and a challenging problem. In this work, we aim at understanding the pioneering work in neural caption generation, Show-and-Tell [4], with respect to art historical data. Artworks are characterized by various artistic styles, attributes, and motives; along with great diversity of artists creating these artworks and the different periods in history when these were created. This makes it very challenging to build models that are agnostic to all such variations. In this work, we propose an art history based image captioning dataset of 4000 images across 9 iconographies (annunciation, adoration, baptism, still-life, nativity, virgin and child, rape, tower of babel and noli me tangere) along with a description for each image consisting of one or more paragraphs. Inspired by [4], we fine-tune the...
Scientific Reports, 2020
Exercise-induced pulmonary hemorrhage (EIPH) is a common condition in sport horses with negative ... more Exercise-induced pulmonary hemorrhage (EIPH) is a common condition in sport horses with negative impact on performance. Cytology of bronchoalveolar lavage fluid by use of a scoring system is considered the most sensitive diagnostic method. Macrophages are classified depending on the degree of cytoplasmic hemosiderin content. The current gold standard is manual grading, which is however monotonous and time-consuming. We evaluated state-of-the-art deep learning-based methods for single cell macrophage classification and compared them against the performance of nine cytology experts and evaluated inter- and intra-observer variability. Additionally, we evaluated object detection methods on a novel data set of 17 completely annotated cytology whole slide images (WSI) containing 78,047 hemosiderophages. Our deep learning-based approach reached a concordance of 0.85, partially exceeding human expert concordance (0.68 to 0.86, mean of 0.73, SD of 0.04). Intra-observer variability was high (...
Background: Interventional workflow analysis can help to increase the quality and efficiency of p... more Background: Interventional workflow analysis can help to increase the quality and efficiency of performed procedures, which are two important factors in the medical domain [ref:1]. A useful tool for analyzing medical workflow is video-based phase annotation of procedures, since the duration[for full text, please go to the a.m. URL]
In the field of Art History, images of artworks and their contexts are core to understanding the ... more In the field of Art History, images of artworks and their contexts are core to understanding the underlying semantic information. However, the highly complex and sophisticated representation of these artworks makes it difficult, even for the experts, to analyze the scene. From the computer vision perspective, the task of analyzing such artworks can be divided into sub-problems by taking a bottom-up approach. In this paper, we focus on the problem of recognizing the characters in Art History. From the iconography of AnnunciationoftheLordAnnunciation of the LordAnnunciationoftheLord (Figure 1), we consider the representation of the main protagonists, MaryMaryMary and GabrielGabrielGabriel, across different artworks and styles. We investigate and present the findings of training a character classifier on features extracted from their face images. The limitations of this method, and the inherent ambiguity in the representation of GabrielGabrielGabriel, motivated us to consider their bodies (a bigger context) to analyze in order to recognize the characters...
The understanding of iconography and visual narration in ancient imagery is one of the main foci ... more The understanding of iconography and visual narration in ancient imagery is one of the main foci in the field of Classical Archaeology, e.g. in Attic vase paintings of the fifth century B.C. In order to depict the situations and actions of a narrative as well as to characterise its protagonists, ancient Greek artists made use of a broad variety of often similar image elements [1]. The interaction and meaningful relationship of the protagonists is depicted with significant postures and gestures (schemata) in order to illustrate key aspects of the storyline [2, 3]. These schemes are not restricted to a certain iconography, so that visual links between different images occur. Being familiar with these relationships the ancient viewer could detect the specific narration and understand the meaning of the image. For example, the scheme of leading the bride in Attic vase paintings is characterised by a significant leading-gesture (χεῖρ' ἐπὶ καρπῷ – hand on wrist / hand on hand) that re...
Heritage Preservation, 2018
ArXiv, 2020
Human pose estimation (HPE) is a central part of understanding the visual narration and body move... more Human pose estimation (HPE) is a central part of understanding the visual narration and body movements of characters depicted in artwork collections, such as Greek vase paintings. Unfortunately, existing HPE methods do not generalise well across domains resulting in poorly recognized poses. Therefore, we propose a two step approach: (1) adapting a dataset of natural images of known person and pose annotations to the style of Greek vase paintings by means of image style-transfer. We introduce a perceptually-grounded style transfer training to enforce perceptual consistency. Then, we fine-tune the base model with this newly created dataset. We show that using style-transfer learning significantly improves the SOTA performance on unlabelled data by more than 6% mean average precision (mAP) as well as mean average recall (mAR). (2) To improve the already strong results further, we created a small dataset (ClassArch) consisting of ancient Greek vase paintings from the 6-5th century BCE w...
Proceedings of the 1st Workshop on Structuring and Understanding of Multimedia heritAge Contents - SUMAC '19, 2019
Figure 1: Art historical scene depicting the iconography called Annunciation of the Lord (left [1... more Figure 1: Art historical scene depicting the iconography called Annunciation of the Lord (left [10], right [32]). Mary and Gabriel are the main protagonists. We can clearly see the differences in the background, in the artistic style, in the foreground, in the objects, their properties, and the use of color.
ArXiv, 2020
Image compositions as a tool for analysis of artworks is of extreme significance for art historia... more Image compositions as a tool for analysis of artworks is of extreme significance for art historians. These compositions are useful in analyzing the interactions in an image to study artists and their artworks. Max Imdahl in his work called Ikonik, along with other prominent art historians of the 20th century, underlined the aesthetic and semantic importance of the structural composition of an image. Understanding underlying compositional structures within images is challenging and a time consuming task. Generating these structures automatically using computer vision techniques (1) can help art historians towards their sophisticated analysis by saving lot of time; providing an overview and access to huge image repositories and (2) also provide an important step towards an understanding of man made imagery by machines. In this work, we attempt to automate this process using the existing state of the art machine learning techniques, without involving any form of training. Our approach,...
One of the first steps to understanding images is by reading their captions. In the field of comp... more One of the first steps to understanding images is by reading their captions. In the field of computer vision and natural language processing, generating captions from images has been an important and a challenging problem. In this work, we aim at understanding the pioneering work in neural caption generation, Show-and-Tell [4], with respect to art historical data. Artworks are characterized by various artistic styles, attributes, and motives; along with great diversity of artists creating these artworks and the different periods in history when these were created. This makes it very challenging to build models that are agnostic to all such variations. In this work, we propose an art history based image captioning dataset of 4000 images across 9 iconographies (annunciation, adoration, baptism, still-life, nativity, virgin and child, rape, tower of babel and noli me tangere) along with a description for each image consisting of one or more paragraphs. Inspired by [4], we fine-tune the...
Scientific Reports, 2020
Exercise-induced pulmonary hemorrhage (EIPH) is a common condition in sport horses with negative ... more Exercise-induced pulmonary hemorrhage (EIPH) is a common condition in sport horses with negative impact on performance. Cytology of bronchoalveolar lavage fluid by use of a scoring system is considered the most sensitive diagnostic method. Macrophages are classified depending on the degree of cytoplasmic hemosiderin content. The current gold standard is manual grading, which is however monotonous and time-consuming. We evaluated state-of-the-art deep learning-based methods for single cell macrophage classification and compared them against the performance of nine cytology experts and evaluated inter- and intra-observer variability. Additionally, we evaluated object detection methods on a novel data set of 17 completely annotated cytology whole slide images (WSI) containing 78,047 hemosiderophages. Our deep learning-based approach reached a concordance of 0.85, partially exceeding human expert concordance (0.68 to 0.86, mean of 0.73, SD of 0.04). Intra-observer variability was high (...