Explainabilitys gain is optimalitys loss? How explanations bias decision-making

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Citations (Scopus)

Abstract

Decisions in organizations are about evaluating alternatives and choosing the one that would best serve organizational goals. To the extent that the evaluation of alternatives could be formulated as a predictive task with appropriate metrics, machine learning algorithms are increasingly being used to improve the efficiency of the process. Explanations help to facilitate communication between the algorithm and the human decision-maker, making it easier for the latter to interpret and make decisions on the basis of predictions by the former. Feature-based explanations' semantics of causal models, however, induce leakage from the decision-maker's prior beliefs. Our findings from a field experiment demonstrate empirically how this leads to confirmation bias and disparate impact on the decision-maker's confidence in the predictions. Such differences can lead to sub-optimal and biased decision outcomes.

Original languageEnglish
Title of host publicationAIES 2022 - Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society
PublisherAssociation for Computing Machinery, Inc
Pages778-787
Number of pages10
ISBN (Electronic)9781450392471
DOIs
Publication statusPublished - 26 Jul 2022
Event5th AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society, AIES 2022 - Oxford, United Kingdom
Duration: 1 Aug 20223 Aug 2022

Publication series

NameAIES 2022 - Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society

Conference

Conference5th AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society, AIES 2022
Country/TerritoryUnited Kingdom
CityOxford
Period1/08/223/08/22

Keywords

  • bias
  • confirmation bias
  • decision-making
  • explanations
  • human causal reasoning
  • human-Algorithm communication
  • semantics of explanations

Fingerprint

Dive into the research topics of 'Explainabilitys gain is optimalitys loss? How explanations bias decision-making'. Together they form a unique fingerprint.

Cite this