Analyzing Character Representation in Media Content using Multimodal Foundation Model: Effectiveness and Trust (original) (raw)

Taka, Evdoxia ORCID: https://orcid.org/0000-0001-7011-3367, Bhattacharya, Debadyuti, Garde-Hansen, Joanne, Sharma, Sanjay and Guha, Tanaya ORCID: https://orcid.org/0000-0003-2167-4891(2025) Analyzing Character Representation in Media Content using Multimodal Foundation Model: Effectiveness and Trust. In: 27th International Conference on Multimodal Interaction (ICMI 2025), Canberra, Australia, 13-17 Oct 2025, pp. 466-474. ISBN 9798400714993(doi: 10.1145/3716553.3750785)

Abstract

Recent advances in AI has enabled automated analysis of complex media content at scale and generate actionable insights regarding character representation along such dimensions as gender and age. Past work focused on quantifying representation from audio/video/text using various ML models, but without having the audience in the loop. We ask, even if character distribution along demographic dimensions are available, how useful are they to the general public? Do they actually trust the numbers generated by AI models? Our work addresses these questions through a user study, while proposing a new AI-based character representation and visualization tool. Our tool based on the Contrastive Language Image Pretraining (CLIP) foundation model to analyze visual screen data to quantify character representation across dimensions of age and gender. We also designed effective visualizations suitable for presenting such analytics to lay audience. Next, we conducted a user study to seek empirical evidence on the usefulness and trustworthiness of the AI-generated results for carefully chosen movies presented in the form of our visualizations. We note that participants were able to understand the analytics from our visualization, and deemed the tool `overall useful'. Participants also indicated a need for more detailed visualizations to include more demographic categories and contextual information of the characters. Participants' trust in AI-based gender and age models is seen to be moderate to low, although they were not against the use of AI in this context.

University Staff: Request a correction | Enlighten Editors: Update this record

Deposit and Record Details