Andrea Ferrario | Swiss Federal Institute of Technology (ETH) (original) (raw)
Drafts by Andrea Ferrario
![Research paper thumbnail of Transparency as design publicity: explaining and justifying inscrutable algorithms [Final open access]](https://attachments.academia-assets.com/64733471/thumbnails/1.jpg)
](Ethics and Information Technology
In this paper we argue that transparency of machine learning algorithms, just as explanation, can... more In this paper we argue that transparency of machine learning algorithms, just as explanation, can be defined at different levels of abstraction. We criticize recent attempts to identify the explanation of black box algorithms with making their decisions (post-hoc) interpretable, focusing our discussion on counterfactual explanations. These approaches to explanation simplify the real nature of the black boxes and risk misleading the public about the normative features of a model. We propose a new form of algorithmic transparency, that consists in explaining algorithms as an intentional product, that serves a particular goal, or multiple goals (Daniel Dennet’s design stance) in a given domain of applicability, and that provides a measure of the extent to which such a goal is achieved, and evidence about the way that measure has been reached. We call such idea of algorithmic transparency “design publicity.” We argue that design publicity can be more easily linked with the justification of the use and of the design of the algorithm, and of each individual decision following from it. In comparison to post-hoc explanations of individual algorithmic decisions, design publicity meets a different demand (the demand for impersonal justification) of the explainee. Finally, we argue that when models that pursue justifiable goals (which may include fairness as avoidance of bias towards specific groups) to a justifiable degree are used consistently, the resulting decisions are all justified even if some of them are (unavoidably) based on incorrect predictions. For this argument, we rely on John Rawls’s idea of procedural justice applied to algorithms conceived as institutions.
List of publications by Andrea Ferrario
Philosophy & Technology, 2019
Real engines of the artificial intelligence (AI) revolution, machine learning (ML) models and alg... more Real engines of the artificial intelligence (AI) revolution, machine learning (ML) models and algorithms are embedded nowadays in many services and products around us. As a society, we argue it is now necessary to transition into a phronetic paradigm focused on the ethical dilemmas stemming from the conception and application of AIs to define actionable recommendations as well as normative solutions. However, both academic research and society-driven initiatives are still quite far from clearly defining a solid programme of study and intervention. In this contribution, we will focus on selected ethical investigations around AI by proposing an incremental model of trust that can be applied to both human-human and human-AI interactions. Starting with a quick overview of the existing accounts of trust, with special attention to Taddeo’s concept of “e-trust”, we will discuss all the components of the proposed model and the reasons to trust in human-AI-interactions in an example of relevance for business organizations. We end this contribution with an analysis of the epistemic and pragmatic reasons of trust in human-AI interactions and with a discussion of kinds of normativity in trustworthiness of AIs.
data ethics by Andrea Ferrario
Philosophy and Technology, 2019
Real engines of the artificial intelligence (AI) revolution, machine learning (ML) models, and al... more Real engines of the artificial intelligence (AI) revolution, machine learning (ML) models, and algorithms are embedded nowadays in many services and products around us. As a society, we argue it is now necessary to transition into a phronetic paradigm focused on the ethical dilemmas stemming from the conception and application of AIs to define actionable recommendations as well as normative solutions. However, both academic research and society-driven initiatives are still quite far from clearly defining a solid program of study and intervention. In this contribution, we will focus on selected ethical investigations around AI by proposing an incremental model of trust that can be applied to both human-human and human-AI interactions. Starting with a quick overview of the existing accounts of trust, with special attention to Taddeo’s concept of “e-trust,” we will discuss all the components of the proposed model and the reasons to trust in human-AI interactions in an example of relevance for business organizations. We end this contribution with an analysis of the epistemic and pragmatic reasons of trust in human-AI interactions and with a discussion of kinds of normativity in trustworthiness of AIs.
Keywords
Artificial intelligence (AI) Trust Trustworthiness E-trust
Papers by Andrea Ferrario
Synthese, May 23, 2023
Trust and monitoring are traditionally antithetical concepts. Describing trust as a property of a... more Trust and monitoring are traditionally antithetical concepts. Describing trust as a property of a relationship of reliance, we introduce a theory of trust and monitoring, which uses mathematical models based on two classes of functions, including q-exponentials, and relates the levels of trust to the costs of monitoring. As opposed to several accounts of trust that attempt to identify the special ingredient of reliance and trust relationships, our theory characterizes trust as a quantitative property of certain relations of reliance that can be quantified and expressed as a scalar quantity. Our theory is applicable to both human-human and human-artificial agent interactions, as it is agnostic with respect to the concrete realization of trustworthiness properties, and is compatible with many views differing on which properties contribute to trust and trustworthiness. Finally, as our mathematical models make the quantitative features of trust measurable, they provide empirical studies on trust with a rigorous methodology for its measurement.
Journal of Medical Internet Research, Apr 13, 2023
Background: Work stress places a heavy economic and disease burden on society. Recent technologic... more Background: Work stress places a heavy economic and disease burden on society. Recent technological advances include digital health interventions for helping employees prevent and manage their stress at work effectively. Although such digital solutions come with an array of ethical risks, especially if they involve biomedical big data, the incorporation of employees' values in their design and deployment has been widely overlooked. Objective: To bridge this gap, we used the value sensitive design (VSD) framework to identify relevant values concerning a digital stress management intervention (dSMI) at the workplace, assess how users comprehend these values, and derive specific requirements for an ethics-informed design of dSMIs. VSD is a theoretically grounded framework that front-loads ethics by accounting for values throughout the design process of a technology. Methods: We conducted a literature search to identify relevant values of dSMIs at the workplace. To understand how potential users comprehend these values and derive design requirements, we conducted a web-based study that contained closed and open questions with employees of a Swiss company, allowing both quantitative and qualitative analyses. Results: The values health and well-being, privacy, autonomy, accountability, and identity were identified through our literature search. Statistical analysis of 170 responses from the web-based study revealed that the intention to use and perceived usefulness of a dSMI were moderate to high. Employees' moderate to high health and well-being concerns included worries that a dSMI would not be effective or would even amplify their stress levels. Privacy concerns were also rated on the higher end of the score range, whereas concerns regarding autonomy, accountability, and identity were rated lower. Moreover, a personalized dSMI with a monitoring system involving a machine learning-based analysis of data led to significantly higher privacy (P=.009) and accountability concerns (P=.04) than a dSMI without a monitoring system. In addition, integrability, user-friendliness, and digital independence emerged as novel values from the qualitative analysis of 85 text responses. Conclusions: Although most surveyed employees were willing to use a dSMI at the workplace, there were considerable health and well-being concerns with regard to effectiveness and problem perpetuation. For a minority of employees who value digital independence, a nondigital offer might be more suitable. In terms of the type of dSMI, privacy and accountability concerns must be particularly well addressed if a machine learning-based monitoring component is included. To help mitigate these concerns, we propose specific requirements to support the VSD of a dSMI at the workplace. The results of this work and our research protocol will inform future research on VSD-based interventions and further advance the integration of ethics in digital health.
SSRN Electronic Journal
We discuss the concepts of algorithm, machine learning and artificial intelligence. We start by n... more We discuss the concepts of algorithm, machine learning and artificial intelligence. We start by noting that different definitions of algorithm may serve different epistemic purposes, and by highlighting the specificities of computer science algorithms together with a high-level overview of their different types. We continue by discussing machine learning as a discipline and commenting on its algorithms, with a focus on artificial neural networks due to their role in the deep learning revolution. We discuss the epistemic and ethical challenges arising from the widely spread use of machine learning algorithms in the applications. Then, we discuss the artifacts of the artificial intelligence domain, clarifying their relation with machine learning algorithms. Finally, we provide the reader with comments on the emergence of the “trustworthy AI” paradigm, highlighting some of the theoretical challenges affecting the discussions on trust in human-AI interactions.
SSRN Electronic Journal, 2021
We provide a philosophical explanation of the relation between artificial intelligence (AI) expla... more We provide a philosophical explanation of the relation between artificial intelligence (AI) explainability and trust in AI, providing a case for expressions, such as “explainability fosters trust in AI,” that commonly appear in the literature. This explanation considers the justification of the trustworthiness of an AI with the need to monitor it during its use. We discuss the latter by referencing an account of trust, called “trust as anti-monitoring,” that different authors contributed developing. We focus our analysis on the case of medical AI systems, noting that our proposal is compatible with internalist and externalist justifications of trustworthiness of medical AI and recent accounts of warranted contractual trust. We propose that “explainability fosters trust in AI” if and only if it fosters warranted paradigmatic trust in AI, i.e., trust in the presence of the justified belief that the AI is trustworthy, which, in turn, causally contributes to rely on the AI in the absence of monitoring. We argue that our proposed approach can intercept the complexity of the interactions between physicians and medical AI systems in clinical practice, as it can distinguish between cases where humans hold different beliefs on the trustworthiness of the medical AI and exercise varying degrees of monitoring on them. Finally, we discuss the main limitations of explainable AI methods and we argue that the case of “explainability fosters trust in AI” may be feasible only in a limited number of physician-medical AI interactions in clinical practice.
Background: Work stress places a heavy economic and disease burden on society. Recent technologic... more Background: Work stress places a heavy economic and disease burden on society. Recent technological advances include digital health interventions for helping employees prevent and manage their stress at work effectively. Although such digital solutions come with an array of ethical risks, especially if they involve biomedical big data, the incorporation of employees' values in their design and deployment has been widely overlooked. Objective: To bridge this gap, we used the value sensitive design (VSD) framework to identify relevant values concerning a digital stress management intervention (dSMI) at the workplace, assess how users comprehend these values, and derive specific requirements for an ethics-informed design of dSMIs. VSD is a theoretically grounded framework that front-loads ethics by accounting for values throughout the design process of a technology. Methods: We conducted a literature search to identify relevant values of dSMIs at the workplace. To understand how potential users comprehend these values and derive design requirements, we conducted a web-based study that contained closed and open questions with employees of a Swiss company, allowing both quantitative and qualitative analyses. Results: The values health and well-being, privacy, autonomy, accountability, and identity were identified through our literature search. Statistical analysis of 170 responses from the web-based study revealed that the intention to use and perceived usefulness of a dSMI were moderate to high. Employees' moderate to high health and well-being concerns included worries that a dSMI would not be effective or would even amplify their stress levels. Privacy concerns were also rated on the higher end of the score range, whereas concerns regarding autonomy, accountability, and identity were rated lower. Moreover, a personalized dSMI with a monitoring system involving a machine learning-based analysis of data led to significantly higher privacy (P=.009) and accountability concerns (P=.04) than a dSMI without a monitoring system. In addition, integrability, user-friendliness, and digital independence emerged as novel values from the qualitative analysis of 85 text responses. Conclusions: Although most surveyed employees were willing to use a dSMI at the workplace, there were considerable health and well-being concerns with regard to effectiveness and problem perpetuation. For a minority of employees who value digital independence, a nondigital offer might be more suitable. In terms of the type of dSMI, privacy and accountability concerns must be particularly well addressed if a machine learning-based monitoring component is included. To help mitigate these concerns, we propose specific requirements to support the VSD of a dSMI at the workplace. The results of this work and our research protocol will inform future research on VSD-based interventions and further advance the integration of ethics in digital health.
SSRN Electronic Journal, 2020
Trust and monitoring are traditionally antithetical concepts. Describing trust as a property of a... more Trust and monitoring are traditionally antithetical concepts. Describing trust as a property of a relationship of reliance, we introduce a theory of trust and monitoring, which uses mathematical models based on two classes of functions, including q-exponentials, and relates the levels of trust to the costs of monitoring. As opposed to several accounts of trust that attempt to identify the special ingredient of reliance and trust relationships, our theory characterizes trust as a quantitative property of certain relations of reliance that can be quantified and expressed as a scalar quantity. Our theory is applicable to both human-human and human-artificial agent interactions, as it is agnostic with respect to the concrete realization of trustworthiness properties, and is compatible with many views differing on which properties contribute to trust and trustworthiness. Finally, as our mathematical models make the quantitative features of trust measurable, they provide empirical studies on trust with a rigorous methodology for its measurement.
Psychoneuroendocrinology, 2020
Background: The high prevalence of office stress and its detrimental health consequences are of c... more Background: The high prevalence of office stress and its detrimental health consequences are of concern to individuals, employers and society at large. Laboratory studies investigating office stress have mostly relied on data from participants that were tested individually on abstract tasks. In this study, we examined the effect of psychosocial office stress and work interruptions on the psychobiological stress response in a realistic but controlled group office environment. We also explored the role of cognitive stress appraisal as an underlying mechanism mediating the relationship between work stressors and the stress response. Methods and Materials: Ninety participants (44 female; mean age 23.11 ± 3.80) were randomly assigned to either a control condition or one of two experimental conditions in which they were exposed to psychosocial stress with or without prior work interruptions in a realistic multi-participant laboratory setting. To induce psychosocial stress, we adapted the Trier Social Stress Test for Groups to an office environment. Throughout the experiment, we continuously monitored heart rate and heart rate variability. Participants repeatedly reported on their current mood, calmness, wakefulness and perceived stress and gave saliva samples to assess changes in salivary cortisol and salivary alpha-amylase. Additionally, cognitive appraisal of the psychosocial stress test was evaluated. Results: Our analyses revealed significant group differences for most outcomes during or immediately after the stress test (i.e., mood, calmness, perceived stress, salivary cortisol, heart rate, heart rate variability) and during recovery (i.e., salivary cortisol and heart rate). Interestingly, the condition that experienced work interruptions showed a higher increase of cortisol levels but appraised the stress test as less threatening than individuals that experienced only psychosocial stress. Exploratory mediation analyses revealed a blunted response in subjective measures of stress, which was partially explained by the differences in threat appraisal. Discussion: The results showed that experimentally induced work stress led to significant responses of subjective measures of stress, the hypothalamic-pituitary-adrenal axis and the autonomic nervous system. However, there appears to be a discrepancy between the psychological and biological responses to preceding work interruptions. Appraising psychosocial stress as less threatening but still as challenging could be an adaptive way of coping and reflect a state of engagement and eustress.
Letters in Mathematical Physics, 2010
The aim of this short note is to present a proof of the existence of an A∞-quasi-isomorphism betw... more The aim of this short note is to present a proof of the existence of an A∞-quasi-isomorphism between the A∞-S(V *)-∧(V)-bimodule K, introduced in [1], and the Koszul complex K(V) of S(V *), viewed as an A∞-S(V *)-∧(V)-bimodule, for V a finite-dimensional (complex or real) vector space.
Journal of Medical Ethics, 2021
In their article ‘Who is afraid of black box algorithms? On the epistemological and ethical basis... more In their article ‘Who is afraid of black box algorithms? On the epistemological and ethical basis of trust in medical AI’, Durán and Jongsma discuss the epistemic and ethical challenges raised by black box algorithms in medical practice. The opacity of black box algorithms is an obstacle to the trustworthiness of their outcomes. Moreover, the use of opaque algorithms is not normatively justified in medical practice. The authors introduce a formalism, called computational reliabilism, which allows generating justified beliefs on the algorithm reliability and trustworthy outcomes of artificial intelligence (AI) systems by means of epistemic warrants, called reliability indicators. However, they remark the need for reliability indicators specific to black box algorithms and that justified knowledge is not sufficient to justify normatively the actions of the physicians using medical AI systems. Therefore, Durán and Jongsma advocate for a more transparent design and implementation of bla...
2022 ACM Conference on Fairness, Accountability, and Transparency
We provide a philosophical explanation of the relation between artificial intelligence (AI) expla... more We provide a philosophical explanation of the relation between artificial intelligence (AI) explainability and trust in AI, providing a case for expressions, such as "explainability fosters trust in AI, " that commonly appear in the literature. This explanation relates the justification of the trustworthiness of an AI with the need to monitor it during its use. We discuss the latter by referencing an account of trust, called "trust as anti-monitoring," that different authors contributed developing. We focus our analysis on the case of medical AI systems, noting that our proposal is compatible with internalist and externalist justifications of trustworthiness of medical AI and recent accounts of warranted contractual trust. We propose that "explainability fosters trust in AI" if and only if it fosters justified and warranted paradigmatic trust in AI, i.e., trust in the presence of the justified belief that the AI is trustworthy, which, in turn, causally contributes to rely on the AI in the absence of monitoring. We argue that our proposed approach can intercept the complexity of the interactions between physicians and medical AI systems in clinical practice, as it can distinguish between cases where humans hold different beliefs on the trustworthiness of the medical AI and exercise varying degrees of monitoring on them. Finally, we apply our account to user's trust in AI, where, we argue, explainability does not contribute to trust. By contrast, when considering public trust in AI as used by a human, we argue, it is possible for explainability to contribute to trust. Our account can explain the apparent paradox that in order to trust AI, we must trust AI users not to trust AI completely. Summing up, we can explain how explainability contributes to justified trust in AI, without leaving a reliabilist framework, but only by redefining the trusted entity as an AI-user dyad. CCS CONCEPTS • Human-centered computing → HCI theory, concepts and models; • Applied computing → Sociology; • Social and professional topics → Computing / technology policy; • Computing methodologies → Artificial intelligence.
Building trust in AI-based systems is deemed critical for their adoption and appropriate use. Rec... more Building trust in AI-based systems is deemed critical for their adoption and appropriate use. Recent research has thus attempted to evaluate how various attributes of these systems affect user trust. However, limitations regarding the definition and measurement of trust in AI have hampered progress in the field, leading to results that are inconsistent or difficult to compare. In this work, we provide an overview of the main limitations in defining and measuring trust in AI. We focus on the attempt of giving trust in AI a numerical value and its utility in informing the design of real-world human-AI interactions. Taking a socio-technical system perspective on AI, we explore two distinct approaches to tackle these challenges. We provide actionable recommendations on how these approaches can be implemented in practice and inform the design of human-AI interactions. We thereby aim to provide a starting point for researchers and designers to re-evaluate the current focus on trust in AI, improving the alignment between what empirical research paradigms may offer and the expectations of real-world human-AI interactions.
2019 6th Swiss Conference on Data Science (SDS)
In this paper, we outline the structure and content of a code of ethics for companies engaged in ... more In this paper, we outline the structure and content of a code of ethics for companies engaged in data-based business, i.e. companies whose value propositions strongly depends on using data. The code provides an ethical reference for all people in the organization who are responsible for activities around data. It is primarily targeting private industry, but public organizations and administrations may also use it. A joint industry-academic initiative, involving specialists for ethics as well as for all relevant data-related issues, developed this code.
Commentary on the use of digital technologies to enhance Advance Directives and support informed ... more Commentary on the use of digital technologies to enhance Advance Directives and support informed decision-making. This, in turn, promotes the ultimate end of goal concordant care.
Journal of Medical Ethics, 2020
In his recent article ‘Limits of trust in medical AI,’ Hatherley argues that, if we believe that ... more In his recent article ‘Limits of trust in medical AI,’ Hatherley argues that, if we believe that the motivations that are usually recognised as relevant for interpersonal trust have to be applied to interactions between humans and medical artificial intelligence, then these systems do not appear to be the appropriate objects of trust. In this response, we argue that it is possible to discuss trust in medical artificial intelligence (AI), if one refrains from simply assuming that trust describes human–human interactions. To do so, we consider an account of trust that distinguishes trust from reliance in a way that is compatible with trusting non-human agents. In this account, to trust a medical AI is to rely on it with little monitoring and control of the elements that make it trustworthy. This attitude does not imply specific properties in the AI system that in fact only humans can have. This account of trust is applicable, in particular, to all cases where a physician relies on the...
JMIR Aging, 2022
Background Language use and social interactions have demonstrated a close relationship with cogni... more Background Language use and social interactions have demonstrated a close relationship with cognitive measures. It is important to improve the understanding of language use and behavioral indicators from social context to study the early prediction of cognitive decline among healthy populations of older adults. Objective This study aimed at predicting an important cognitive ability, working memory, of 98 healthy older adults participating in a 4-day-long naturalistic observation study. We used linguistic measures, part-of-speech (POS) tags, and social context information extracted from 7450 real-life audio recordings of their everyday conversations. Methods The methods in this study comprise (1) the generation of linguistic measures, representing idea density, vocabulary richness, and grammatical complexity, as well as POS tags with natural language processing (NLP) from the transcripts of real-life conversations and (2) the training of machine learning models to predict working mem...
Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, 2020
In this work, we present ALEEDSA: the first system for performing interactive machine learning wi... more In this work, we present ALEEDSA: the first system for performing interactive machine learning with augmented reality. The system is characterized by the following three distinctive features: First, immersion is used for visualizing machine learning models in terms of their outcomes. The outcomes can then be compared against domain knowledge (e.g., via counterfactual explanations) so that users can better understand the behavior of machine learning models. Second, interactivity with augmented reality along the complete machine learning pipeline fosters rapid modeling. Third, collaboration enables a multi-user setting, wherein machine learning engineers and domain experts can jointly discuss the behavior of machine learning models. The effectiveness of our proof-of-concept is demonstrated in an experimental study involving both students and business professionals. Altogether, ALEEDSA provides a more straightforward utilization of machine learning in organizational and educational practice.
Counterfactual explanations are a prominent example of post-hoc interpretability methods in the e... more Counterfactual explanations are a prominent example of post-hoc interpretability methods in the explainable Artificial Intelligence research domain. They provide individuals with alternative scenarios and a set of recommendations to achieve a sought-after machine learning model outcome. Recently, the literature has identified desiderata of counterfactual explanations, such as feasibility, actionability and sparsity that should support their applicability in real-world contexts. However, we show that the literature has neglected the problem of the time dependency of counterfactual explanations. We argue that, due to their time dependency and because of the provision of recommendations, even feasible, actionable and sparse counterfactual explanations may not be appropriate in real-world applications. This is due to the possible emergence of what we call "unfortunate counterfactual events." These events may occur due to the retraining of machine learning models whose outcomes...
Ethics and Information Technology
In this paper we argue that transparency of machine learning algorithms, just as explanation, can... more In this paper we argue that transparency of machine learning algorithms, just as explanation, can be defined at different levels of abstraction. We criticize recent attempts to identify the explanation of black box algorithms with making their decisions (post-hoc) interpretable, focusing our discussion on counterfactual explanations. These approaches to explanation simplify the real nature of the black boxes and risk misleading the public about the normative features of a model. We propose a new form of algorithmic transparency, that consists in explaining algorithms as an intentional product, that serves a particular goal, or multiple goals (Daniel Dennet’s design stance) in a given domain of applicability, and that provides a measure of the extent to which such a goal is achieved, and evidence about the way that measure has been reached. We call such idea of algorithmic transparency “design publicity.” We argue that design publicity can be more easily linked with the justification of the use and of the design of the algorithm, and of each individual decision following from it. In comparison to post-hoc explanations of individual algorithmic decisions, design publicity meets a different demand (the demand for impersonal justification) of the explainee. Finally, we argue that when models that pursue justifiable goals (which may include fairness as avoidance of bias towards specific groups) to a justifiable degree are used consistently, the resulting decisions are all justified even if some of them are (unavoidably) based on incorrect predictions. For this argument, we rely on John Rawls’s idea of procedural justice applied to algorithms conceived as institutions.
Philosophy & Technology, 2019
Real engines of the artificial intelligence (AI) revolution, machine learning (ML) models and alg... more Real engines of the artificial intelligence (AI) revolution, machine learning (ML) models and algorithms are embedded nowadays in many services and products around us. As a society, we argue it is now necessary to transition into a phronetic paradigm focused on the ethical dilemmas stemming from the conception and application of AIs to define actionable recommendations as well as normative solutions. However, both academic research and society-driven initiatives are still quite far from clearly defining a solid programme of study and intervention. In this contribution, we will focus on selected ethical investigations around AI by proposing an incremental model of trust that can be applied to both human-human and human-AI interactions. Starting with a quick overview of the existing accounts of trust, with special attention to Taddeo’s concept of “e-trust”, we will discuss all the components of the proposed model and the reasons to trust in human-AI-interactions in an example of relevance for business organizations. We end this contribution with an analysis of the epistemic and pragmatic reasons of trust in human-AI interactions and with a discussion of kinds of normativity in trustworthiness of AIs.
Philosophy and Technology, 2019
Real engines of the artificial intelligence (AI) revolution, machine learning (ML) models, and al... more Real engines of the artificial intelligence (AI) revolution, machine learning (ML) models, and algorithms are embedded nowadays in many services and products around us. As a society, we argue it is now necessary to transition into a phronetic paradigm focused on the ethical dilemmas stemming from the conception and application of AIs to define actionable recommendations as well as normative solutions. However, both academic research and society-driven initiatives are still quite far from clearly defining a solid program of study and intervention. In this contribution, we will focus on selected ethical investigations around AI by proposing an incremental model of trust that can be applied to both human-human and human-AI interactions. Starting with a quick overview of the existing accounts of trust, with special attention to Taddeo’s concept of “e-trust,” we will discuss all the components of the proposed model and the reasons to trust in human-AI interactions in an example of relevance for business organizations. We end this contribution with an analysis of the epistemic and pragmatic reasons of trust in human-AI interactions and with a discussion of kinds of normativity in trustworthiness of AIs.
Keywords
Artificial intelligence (AI) Trust Trustworthiness E-trust
Synthese, May 23, 2023
Trust and monitoring are traditionally antithetical concepts. Describing trust as a property of a... more Trust and monitoring are traditionally antithetical concepts. Describing trust as a property of a relationship of reliance, we introduce a theory of trust and monitoring, which uses mathematical models based on two classes of functions, including q-exponentials, and relates the levels of trust to the costs of monitoring. As opposed to several accounts of trust that attempt to identify the special ingredient of reliance and trust relationships, our theory characterizes trust as a quantitative property of certain relations of reliance that can be quantified and expressed as a scalar quantity. Our theory is applicable to both human-human and human-artificial agent interactions, as it is agnostic with respect to the concrete realization of trustworthiness properties, and is compatible with many views differing on which properties contribute to trust and trustworthiness. Finally, as our mathematical models make the quantitative features of trust measurable, they provide empirical studies on trust with a rigorous methodology for its measurement.
Journal of Medical Internet Research, Apr 13, 2023
Background: Work stress places a heavy economic and disease burden on society. Recent technologic... more Background: Work stress places a heavy economic and disease burden on society. Recent technological advances include digital health interventions for helping employees prevent and manage their stress at work effectively. Although such digital solutions come with an array of ethical risks, especially if they involve biomedical big data, the incorporation of employees' values in their design and deployment has been widely overlooked. Objective: To bridge this gap, we used the value sensitive design (VSD) framework to identify relevant values concerning a digital stress management intervention (dSMI) at the workplace, assess how users comprehend these values, and derive specific requirements for an ethics-informed design of dSMIs. VSD is a theoretically grounded framework that front-loads ethics by accounting for values throughout the design process of a technology. Methods: We conducted a literature search to identify relevant values of dSMIs at the workplace. To understand how potential users comprehend these values and derive design requirements, we conducted a web-based study that contained closed and open questions with employees of a Swiss company, allowing both quantitative and qualitative analyses. Results: The values health and well-being, privacy, autonomy, accountability, and identity were identified through our literature search. Statistical analysis of 170 responses from the web-based study revealed that the intention to use and perceived usefulness of a dSMI were moderate to high. Employees' moderate to high health and well-being concerns included worries that a dSMI would not be effective or would even amplify their stress levels. Privacy concerns were also rated on the higher end of the score range, whereas concerns regarding autonomy, accountability, and identity were rated lower. Moreover, a personalized dSMI with a monitoring system involving a machine learning-based analysis of data led to significantly higher privacy (P=.009) and accountability concerns (P=.04) than a dSMI without a monitoring system. In addition, integrability, user-friendliness, and digital independence emerged as novel values from the qualitative analysis of 85 text responses. Conclusions: Although most surveyed employees were willing to use a dSMI at the workplace, there were considerable health and well-being concerns with regard to effectiveness and problem perpetuation. For a minority of employees who value digital independence, a nondigital offer might be more suitable. In terms of the type of dSMI, privacy and accountability concerns must be particularly well addressed if a machine learning-based monitoring component is included. To help mitigate these concerns, we propose specific requirements to support the VSD of a dSMI at the workplace. The results of this work and our research protocol will inform future research on VSD-based interventions and further advance the integration of ethics in digital health.
SSRN Electronic Journal
We discuss the concepts of algorithm, machine learning and artificial intelligence. We start by n... more We discuss the concepts of algorithm, machine learning and artificial intelligence. We start by noting that different definitions of algorithm may serve different epistemic purposes, and by highlighting the specificities of computer science algorithms together with a high-level overview of their different types. We continue by discussing machine learning as a discipline and commenting on its algorithms, with a focus on artificial neural networks due to their role in the deep learning revolution. We discuss the epistemic and ethical challenges arising from the widely spread use of machine learning algorithms in the applications. Then, we discuss the artifacts of the artificial intelligence domain, clarifying their relation with machine learning algorithms. Finally, we provide the reader with comments on the emergence of the “trustworthy AI” paradigm, highlighting some of the theoretical challenges affecting the discussions on trust in human-AI interactions.
SSRN Electronic Journal, 2021
We provide a philosophical explanation of the relation between artificial intelligence (AI) expla... more We provide a philosophical explanation of the relation between artificial intelligence (AI) explainability and trust in AI, providing a case for expressions, such as “explainability fosters trust in AI,” that commonly appear in the literature. This explanation considers the justification of the trustworthiness of an AI with the need to monitor it during its use. We discuss the latter by referencing an account of trust, called “trust as anti-monitoring,” that different authors contributed developing. We focus our analysis on the case of medical AI systems, noting that our proposal is compatible with internalist and externalist justifications of trustworthiness of medical AI and recent accounts of warranted contractual trust. We propose that “explainability fosters trust in AI” if and only if it fosters warranted paradigmatic trust in AI, i.e., trust in the presence of the justified belief that the AI is trustworthy, which, in turn, causally contributes to rely on the AI in the absence of monitoring. We argue that our proposed approach can intercept the complexity of the interactions between physicians and medical AI systems in clinical practice, as it can distinguish between cases where humans hold different beliefs on the trustworthiness of the medical AI and exercise varying degrees of monitoring on them. Finally, we discuss the main limitations of explainable AI methods and we argue that the case of “explainability fosters trust in AI” may be feasible only in a limited number of physician-medical AI interactions in clinical practice.
Background: Work stress places a heavy economic and disease burden on society. Recent technologic... more Background: Work stress places a heavy economic and disease burden on society. Recent technological advances include digital health interventions for helping employees prevent and manage their stress at work effectively. Although such digital solutions come with an array of ethical risks, especially if they involve biomedical big data, the incorporation of employees' values in their design and deployment has been widely overlooked. Objective: To bridge this gap, we used the value sensitive design (VSD) framework to identify relevant values concerning a digital stress management intervention (dSMI) at the workplace, assess how users comprehend these values, and derive specific requirements for an ethics-informed design of dSMIs. VSD is a theoretically grounded framework that front-loads ethics by accounting for values throughout the design process of a technology. Methods: We conducted a literature search to identify relevant values of dSMIs at the workplace. To understand how potential users comprehend these values and derive design requirements, we conducted a web-based study that contained closed and open questions with employees of a Swiss company, allowing both quantitative and qualitative analyses. Results: The values health and well-being, privacy, autonomy, accountability, and identity were identified through our literature search. Statistical analysis of 170 responses from the web-based study revealed that the intention to use and perceived usefulness of a dSMI were moderate to high. Employees' moderate to high health and well-being concerns included worries that a dSMI would not be effective or would even amplify their stress levels. Privacy concerns were also rated on the higher end of the score range, whereas concerns regarding autonomy, accountability, and identity were rated lower. Moreover, a personalized dSMI with a monitoring system involving a machine learning-based analysis of data led to significantly higher privacy (P=.009) and accountability concerns (P=.04) than a dSMI without a monitoring system. In addition, integrability, user-friendliness, and digital independence emerged as novel values from the qualitative analysis of 85 text responses. Conclusions: Although most surveyed employees were willing to use a dSMI at the workplace, there were considerable health and well-being concerns with regard to effectiveness and problem perpetuation. For a minority of employees who value digital independence, a nondigital offer might be more suitable. In terms of the type of dSMI, privacy and accountability concerns must be particularly well addressed if a machine learning-based monitoring component is included. To help mitigate these concerns, we propose specific requirements to support the VSD of a dSMI at the workplace. The results of this work and our research protocol will inform future research on VSD-based interventions and further advance the integration of ethics in digital health.
SSRN Electronic Journal, 2020
Trust and monitoring are traditionally antithetical concepts. Describing trust as a property of a... more Trust and monitoring are traditionally antithetical concepts. Describing trust as a property of a relationship of reliance, we introduce a theory of trust and monitoring, which uses mathematical models based on two classes of functions, including q-exponentials, and relates the levels of trust to the costs of monitoring. As opposed to several accounts of trust that attempt to identify the special ingredient of reliance and trust relationships, our theory characterizes trust as a quantitative property of certain relations of reliance that can be quantified and expressed as a scalar quantity. Our theory is applicable to both human-human and human-artificial agent interactions, as it is agnostic with respect to the concrete realization of trustworthiness properties, and is compatible with many views differing on which properties contribute to trust and trustworthiness. Finally, as our mathematical models make the quantitative features of trust measurable, they provide empirical studies on trust with a rigorous methodology for its measurement.
Psychoneuroendocrinology, 2020
Background: The high prevalence of office stress and its detrimental health consequences are of c... more Background: The high prevalence of office stress and its detrimental health consequences are of concern to individuals, employers and society at large. Laboratory studies investigating office stress have mostly relied on data from participants that were tested individually on abstract tasks. In this study, we examined the effect of psychosocial office stress and work interruptions on the psychobiological stress response in a realistic but controlled group office environment. We also explored the role of cognitive stress appraisal as an underlying mechanism mediating the relationship between work stressors and the stress response. Methods and Materials: Ninety participants (44 female; mean age 23.11 ± 3.80) were randomly assigned to either a control condition or one of two experimental conditions in which they were exposed to psychosocial stress with or without prior work interruptions in a realistic multi-participant laboratory setting. To induce psychosocial stress, we adapted the Trier Social Stress Test for Groups to an office environment. Throughout the experiment, we continuously monitored heart rate and heart rate variability. Participants repeatedly reported on their current mood, calmness, wakefulness and perceived stress and gave saliva samples to assess changes in salivary cortisol and salivary alpha-amylase. Additionally, cognitive appraisal of the psychosocial stress test was evaluated. Results: Our analyses revealed significant group differences for most outcomes during or immediately after the stress test (i.e., mood, calmness, perceived stress, salivary cortisol, heart rate, heart rate variability) and during recovery (i.e., salivary cortisol and heart rate). Interestingly, the condition that experienced work interruptions showed a higher increase of cortisol levels but appraised the stress test as less threatening than individuals that experienced only psychosocial stress. Exploratory mediation analyses revealed a blunted response in subjective measures of stress, which was partially explained by the differences in threat appraisal. Discussion: The results showed that experimentally induced work stress led to significant responses of subjective measures of stress, the hypothalamic-pituitary-adrenal axis and the autonomic nervous system. However, there appears to be a discrepancy between the psychological and biological responses to preceding work interruptions. Appraising psychosocial stress as less threatening but still as challenging could be an adaptive way of coping and reflect a state of engagement and eustress.
Letters in Mathematical Physics, 2010
The aim of this short note is to present a proof of the existence of an A∞-quasi-isomorphism betw... more The aim of this short note is to present a proof of the existence of an A∞-quasi-isomorphism between the A∞-S(V *)-∧(V)-bimodule K, introduced in [1], and the Koszul complex K(V) of S(V *), viewed as an A∞-S(V *)-∧(V)-bimodule, for V a finite-dimensional (complex or real) vector space.
Journal of Medical Ethics, 2021
In their article ‘Who is afraid of black box algorithms? On the epistemological and ethical basis... more In their article ‘Who is afraid of black box algorithms? On the epistemological and ethical basis of trust in medical AI’, Durán and Jongsma discuss the epistemic and ethical challenges raised by black box algorithms in medical practice. The opacity of black box algorithms is an obstacle to the trustworthiness of their outcomes. Moreover, the use of opaque algorithms is not normatively justified in medical practice. The authors introduce a formalism, called computational reliabilism, which allows generating justified beliefs on the algorithm reliability and trustworthy outcomes of artificial intelligence (AI) systems by means of epistemic warrants, called reliability indicators. However, they remark the need for reliability indicators specific to black box algorithms and that justified knowledge is not sufficient to justify normatively the actions of the physicians using medical AI systems. Therefore, Durán and Jongsma advocate for a more transparent design and implementation of bla...
2022 ACM Conference on Fairness, Accountability, and Transparency
We provide a philosophical explanation of the relation between artificial intelligence (AI) expla... more We provide a philosophical explanation of the relation between artificial intelligence (AI) explainability and trust in AI, providing a case for expressions, such as "explainability fosters trust in AI, " that commonly appear in the literature. This explanation relates the justification of the trustworthiness of an AI with the need to monitor it during its use. We discuss the latter by referencing an account of trust, called "trust as anti-monitoring," that different authors contributed developing. We focus our analysis on the case of medical AI systems, noting that our proposal is compatible with internalist and externalist justifications of trustworthiness of medical AI and recent accounts of warranted contractual trust. We propose that "explainability fosters trust in AI" if and only if it fosters justified and warranted paradigmatic trust in AI, i.e., trust in the presence of the justified belief that the AI is trustworthy, which, in turn, causally contributes to rely on the AI in the absence of monitoring. We argue that our proposed approach can intercept the complexity of the interactions between physicians and medical AI systems in clinical practice, as it can distinguish between cases where humans hold different beliefs on the trustworthiness of the medical AI and exercise varying degrees of monitoring on them. Finally, we apply our account to user's trust in AI, where, we argue, explainability does not contribute to trust. By contrast, when considering public trust in AI as used by a human, we argue, it is possible for explainability to contribute to trust. Our account can explain the apparent paradox that in order to trust AI, we must trust AI users not to trust AI completely. Summing up, we can explain how explainability contributes to justified trust in AI, without leaving a reliabilist framework, but only by redefining the trusted entity as an AI-user dyad. CCS CONCEPTS • Human-centered computing → HCI theory, concepts and models; • Applied computing → Sociology; • Social and professional topics → Computing / technology policy; • Computing methodologies → Artificial intelligence.
Building trust in AI-based systems is deemed critical for their adoption and appropriate use. Rec... more Building trust in AI-based systems is deemed critical for their adoption and appropriate use. Recent research has thus attempted to evaluate how various attributes of these systems affect user trust. However, limitations regarding the definition and measurement of trust in AI have hampered progress in the field, leading to results that are inconsistent or difficult to compare. In this work, we provide an overview of the main limitations in defining and measuring trust in AI. We focus on the attempt of giving trust in AI a numerical value and its utility in informing the design of real-world human-AI interactions. Taking a socio-technical system perspective on AI, we explore two distinct approaches to tackle these challenges. We provide actionable recommendations on how these approaches can be implemented in practice and inform the design of human-AI interactions. We thereby aim to provide a starting point for researchers and designers to re-evaluate the current focus on trust in AI, improving the alignment between what empirical research paradigms may offer and the expectations of real-world human-AI interactions.
2019 6th Swiss Conference on Data Science (SDS)
In this paper, we outline the structure and content of a code of ethics for companies engaged in ... more In this paper, we outline the structure and content of a code of ethics for companies engaged in data-based business, i.e. companies whose value propositions strongly depends on using data. The code provides an ethical reference for all people in the organization who are responsible for activities around data. It is primarily targeting private industry, but public organizations and administrations may also use it. A joint industry-academic initiative, involving specialists for ethics as well as for all relevant data-related issues, developed this code.
Commentary on the use of digital technologies to enhance Advance Directives and support informed ... more Commentary on the use of digital technologies to enhance Advance Directives and support informed decision-making. This, in turn, promotes the ultimate end of goal concordant care.
Journal of Medical Ethics, 2020
In his recent article ‘Limits of trust in medical AI,’ Hatherley argues that, if we believe that ... more In his recent article ‘Limits of trust in medical AI,’ Hatherley argues that, if we believe that the motivations that are usually recognised as relevant for interpersonal trust have to be applied to interactions between humans and medical artificial intelligence, then these systems do not appear to be the appropriate objects of trust. In this response, we argue that it is possible to discuss trust in medical artificial intelligence (AI), if one refrains from simply assuming that trust describes human–human interactions. To do so, we consider an account of trust that distinguishes trust from reliance in a way that is compatible with trusting non-human agents. In this account, to trust a medical AI is to rely on it with little monitoring and control of the elements that make it trustworthy. This attitude does not imply specific properties in the AI system that in fact only humans can have. This account of trust is applicable, in particular, to all cases where a physician relies on the...
JMIR Aging, 2022
Background Language use and social interactions have demonstrated a close relationship with cogni... more Background Language use and social interactions have demonstrated a close relationship with cognitive measures. It is important to improve the understanding of language use and behavioral indicators from social context to study the early prediction of cognitive decline among healthy populations of older adults. Objective This study aimed at predicting an important cognitive ability, working memory, of 98 healthy older adults participating in a 4-day-long naturalistic observation study. We used linguistic measures, part-of-speech (POS) tags, and social context information extracted from 7450 real-life audio recordings of their everyday conversations. Methods The methods in this study comprise (1) the generation of linguistic measures, representing idea density, vocabulary richness, and grammatical complexity, as well as POS tags with natural language processing (NLP) from the transcripts of real-life conversations and (2) the training of machine learning models to predict working mem...
Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, 2020
In this work, we present ALEEDSA: the first system for performing interactive machine learning wi... more In this work, we present ALEEDSA: the first system for performing interactive machine learning with augmented reality. The system is characterized by the following three distinctive features: First, immersion is used for visualizing machine learning models in terms of their outcomes. The outcomes can then be compared against domain knowledge (e.g., via counterfactual explanations) so that users can better understand the behavior of machine learning models. Second, interactivity with augmented reality along the complete machine learning pipeline fosters rapid modeling. Third, collaboration enables a multi-user setting, wherein machine learning engineers and domain experts can jointly discuss the behavior of machine learning models. The effectiveness of our proof-of-concept is demonstrated in an experimental study involving both students and business professionals. Altogether, ALEEDSA provides a more straightforward utilization of machine learning in organizational and educational practice.
Counterfactual explanations are a prominent example of post-hoc interpretability methods in the e... more Counterfactual explanations are a prominent example of post-hoc interpretability methods in the explainable Artificial Intelligence research domain. They provide individuals with alternative scenarios and a set of recommendations to achieve a sought-after machine learning model outcome. Recently, the literature has identified desiderata of counterfactual explanations, such as feasibility, actionability and sparsity that should support their applicability in real-world contexts. However, we show that the literature has neglected the problem of the time dependency of counterfactual explanations. We argue that, due to their time dependency and because of the provision of recommendations, even feasible, actionable and sparse counterfactual explanations may not be appropriate in real-world applications. This is due to the possible emergence of what we call "unfortunate counterfactual events." These events may occur due to the retraining of machine learning models whose outcomes...
Journal of Medical Ethics, 2021
Artificial intelligence (AI) systems are increasingly being used in healthcare, thanks to the hig... more Artificial intelligence (AI) systems are increasingly being used in healthcare, thanks to the high level of performance that these systems have proven to deliver. So far, clinical applications have focused on diagnosis and on prediction of outcomes. It is less clear in what way AI can or should support complex clinical decisions that crucially depend on patient preferences. In this paper, we focus on the ethical questions arising from the design, development and deployment of AI systems to support decision-making around cardiopulmonary resuscitation and the determination of a patient’s Do Not Attempt to Resuscitate status (also known as code status). The COVID-19 pandemic has made us keenly aware of the difficulties physicians encounter when they have to act quickly in stressful situations without knowing what their patient would have wanted. We discuss the results of an interview study conducted with healthcare professionals in a university hospital aimed at understanding the statu...
SSRN Electronic Journal, 2020
In this tutorial we introduce three approaches to preprocess text data with Natural Language Proc... more In this tutorial we introduce three approaches to preprocess text data with Natural Language Processing (NLP) and perform text document classification using machine learning. The first approach is based on 'bag-of-' models, the second one on word embeddings, while the third one introduces the two most popular Recurrent Neural Networks (RNNs), i.e. the Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) architectures. We apply all approaches on a case study where we classify movie reviews using Python and Tensorflow 2.0. The results of the case study show that extreme gradient boosting algorithms outperform adaptive boosting and random forests on bag-of-words and word embedding models, as well as LSTM and GRU RNNs, but at a steep computational cost. Finally, we provide the reader with comments on NLP applications for the insurance industry.
Philosophy & Technology, 2019
Real engines of the artificial intelligence (AI) revolution, machine learning (ML) models, and al... more Real engines of the artificial intelligence (AI) revolution, machine learning (ML) models, and algorithms are embedded nowadays in many services and products around us. As a society, we argue it is now necessary to transition into a phronetic paradigm focused on the ethical dilemmas stemming from the conception and application of AIs to define actionable recommendations as well as normative solutions. However, both academic research and society-driven initiatives are still quite far from clearly defining a solid program of study and intervention. In this contribution, we will focus on selected ethical investigations around AI by proposing an incremental model of trust that can be applied to both human-human and human-AI interactions. Starting with a quick overview of the existing accounts of trust, with special attention to Taddeo’s concept of “e-trust,” we will discuss all the components of the proposed model and the reasons to trust in human-AI interactions in an example of releva...