SyntaxShap: Syntax-aware Explainability Method for Text Generation (original) (raw)
Related papers
Challenges and Opportunities in Text Generation Explainability
arXiv (Cornell University), 2024
The necessity for interpretability in natural language processing (NLP) has risen alongside the growing prominence of large language models. Among the myriad tasks within NLP, text generation stands out as a primary objective of autoregressive models. The NLP community has begun to take a keen interest in gaining a deeper understanding of text generation, leading to the development of modelagnostic explainable artificial intelligence (xAI) methods tailored to this task. The design and evaluation of explainability methods are non-trivial since they depend on many factors involved in the text generation process, e.g., the autoregressive model and its stochastic nature. This paper outlines 17 challenges categorized into three groups that arise during the development and assessment of attribution-based explainability methods. These challenges encompass issues concerning tokenization, defining explanation similarity, determining token importance and prediction change metrics, the level of human intervention required, and the creation of suitable test datasets. The paper illustrates how these challenges can be intertwined, showcasing new opportunities for the community. These include developing probabilistic word-level explainability methods and engaging humans in the explainability pipeline, from the data design to the final evaluation, to draw robust conclusions on xAI methods.
Evaluating and Explaining Natural Language Generation with GenX
Proceedings of the Second Workshop on Data Science with Human in the Loop: Language Advances
Current methods for evaluation of natural language generation models focus on measuring text quality but fail to probe the model creativity, i.e., its ability to generate novel but coherent text sequences not seen in the training corpus. We present the GenX tool which is designed to enable interactive exploration and explanation of natural language generation outputs with a focus on the detection of memorization. We demonstrate the tool on two domainconditioned generation use cases-phishing emails and ACL abstracts.
A Study of Automatic Metrics for the Evaluation of Natural Language Explanations
Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, 2021
As transparency becomes key for robotics and AI, it will be necessary to evaluate the methods through which transparency is provided, including automatically generated natural language (NL) explanations. Here, we explore parallels between the generation of such explanations and the much-studied field of evaluation of Natural Language Generation (NLG). Specifically, we investigate which of the NLG evaluation measures map well to explanations. We present the ExBAN corpus: a crowd-sourced corpus of NL explanations for Bayesian Networks. We run correlations comparing human subjective ratings with NLG automatic measures. We find that embedding-based automatic NLG evaluation methods, such as BERTScore and BLEURT, have a higher correlation with human ratings, compared to word-overlap metrics, such as BLEU and ROUGE. This work has implications for Explainable AI and transparent robotic and autonomous systems.
Automatic Generation of Natural Language Explanations
Proceedings of the 23rd International Conference on Intelligent User Interfaces Companion, 2018
An important task for recommender system is to generate explanations according to a user's preferences. Most of the current methods for explainable recommendations use structured sentences to provide descriptions along with the recommendations they produce. However, those methods have neglected the review-oriented way of writing a text, even though it is known that these reviews have a strong influence over user's decision. In this paper, we propose a method for the automatic generation of natural language explanations, for predicting how a user would write about an item, based on user ratings from different items' features. We design a character-level recurrent neural network (RNN) model, which generates an item's review explanations using longshort term memories (LSTM). The model generates text reviews given a combination of the review and ratings score that express opinions about different factors or aspects of an item. Our network is trained on a sub-sample from the large real-world dataset BeerAdvocate. Our empirical evaluation using natural language processing metrics shows the generated text's quality is close to a real user written review, identifying negation, misspellings, and domain specific vocabulary.
Explainable AI for NLP: Decoding Black Box
International Journal of Computer Trends and Technology, 2022
Recent advancements in machine learning have sparked greater interest in previously understudied topics. As machine learning improves, experts are being pushed to understand and trace how algorithms get their results, how models think, and why the end outcome. It is also difficult to communicate the outcome to end customers and internal stakeholders such as sales and customer service without explaining the outcomes in simple language, especially using visualization. In specialized domains like law and medicine, it becomes vital to understand the machine learning output.
Xplique: A Deep Learning Explainability Toolbox
Cornell University - arXiv, 2022
Today's most advanced machine-learning models are hardly scrutable. The key challenge for explainability methods is to help assisting researchers in opening up these black boxes-by revealing the strategy that led to a given decision, by characterizing their internal states or by studying the underlying data representation. To address this challenge, we have developed Xplique: a software library for explainability which includes representative explainability methods as well as associated evaluation metrics. It interfaces with one of the most popular learning libraries: Tensorflow as well as other libraries including PyTorch, scikit-learn and Theano. The code is licensed under the MIT license and is freely available at github.com/deel-ai/xplique.
I don't understand! Evaluation Methods for Natural Language Explanations
2021
Explainability of intelligent systems is key for future adoption. While much work is ongoing with regards to developing methods of explaining complex opaque systems, there is little current work on evaluating how effective these explanations are, in particular with respect to the user’s understanding. Natural language (NL) explanations can be seen as an intuitive channel between humans and artificial intelligence systems, in particular for enhancing transparency. This paper presents existing work on how evaluation methods from the field of Natural Language Generation (NLG) can be mapped onto NL explanations. Also, we present a preliminary investigation into the relationship between linguistic features and human evaluation, using a dataset of NL explanations derived from Bayesian Networks.
Proceedings of the 2nd Workshop on Evaluation and Comparison of NLP Systems, 2021
This paper describes the UMD submission to the Explainable Quality Estimation Shared Task at the Eval4NLP 2021 Workshop on "Evaluation & Comparison of NLP Systems". We participated in the word-level and sentencelevel MT Quality Estimation (QE) constrained tasks for all language pairs: Estonian-English, Romanian-English, German-Chinese, and Russian-German. Our approach combines the predictions of a word-level explainer model on top of a sentence-level QE model and a sequence labeler trained on synthetic data. These models are based on pre-trained multilingual language models and do not require any word-level annotations for training, making them well suited to zero-shot settings. Our best performing system improves over the best baseline across all metrics and language pairs, with an average gain of 0.1 in AUC, Average Precision, and Recall at Top-K score.
Deep Distilling: automated code generation using explainable deep learning
arXiv (Cornell University), 2021
Human reasoning can distill principles from observed patterns and generalize them to explain and solve novel problems. The most powerful artificial intelligence systems lack explainability and symbolic reasoning ability, and have therefore not achieved supremacy in domains requiring human understanding, such as science or common sense reasoning. Here we introduce deep distilling, a machine learning method that learns patterns from data using explainable deep learning and then condenses it into concise, executable computer code. The code, which can contain loops, nested logical statements, and useful intermediate variables, is equivalent to the neural network but is generally orders of magnitude more compact and human-comprehensible. On a diverse set of problems involving arithmetic, computer vision, and optimization, we show that deep distilling generates concise code that generalizes out-of-distribution to solve problems ordersof-magnitude larger and more complex than the training data. For problems with a known ground-truth rule set, deep distilling discovers the rule set exactly with scalable guarantees. For problems that are ambiguous or computationally intractable, the distilled rules are similar to existing human-derived algorithms and perform at par or better. Our approach demonstrates that unassisted machine intelligence can build generalizable and intuitive rules explaining patterns in large datasets that would otherwise overwhelm human reasoning. Human reasoning is able to successfully discover principles from the observed world and systematize them into rule-based knowledge frameworks such as scientific theories, mathematical equations, medical treatment protocols, heuristics, chemical synthesis pathways, and computer algorithms. The automation of this reasoning process is the long-term goal of artificial intelligence (AI) and machine learning. However, there is a widely acknowledged trade-off between models that can be explained intuitively (e.g. decision trees) and models that have wide scope and high predictive accuracy (e.g. neural networks) [1, 2]. Explainability is important for legal and ethical reasons [3], for making AI systems more amenable to modification and rational design, and for providing the guarantees and predictability needed for high-stakes applications such as medical diagnosis or autonomous motor vehicle driving. More fundamentally, human comprehensibility of automated reasoning is necessary to transition from computer-aided to computer-directed scientific discovery.
Knowledge-Intensive Language Understanding for Explainable AI
IEEE Internet Computing
AI systems have seen significant adoption in various domains. At the same time, further adoption in some domains is hindered by inability to fully trust an AI system that it will not harm a human. Besides the concerns for fairness, privacy, transparency, and explainability are key to developing trusts in AI systems. As stated in describing trustworthy AI (https://www.ibm.com/watson/trustworthy-ai) "Trust comes through understanding. How AI-led decisions are made and what determining factors were included are crucial to understand." The subarea of explaining AI systems has come to be known as XAI. Multiple aspects of an AI system can be explained; these include biases that the data might have, lack of data points in a particular region of the example space, fairness of gathering the data, feature importances, etc. However, besides these, it is critical to have humancentered explanations that are directly related to decisionmaking similar to how a domain expert makes decisions based on "domain knowledge," that also include well-established, peer-validated explicit guidelines. To understand and validate an AI system's outcomes (such as classification, recommendations, predictions), that lead to developing trust in the AI system, it is necessary to involve explicit domain knowledge that humans understand and use. Contemporary XAI methods are yet addressed explanations that enable decision-making similar to an expert. Figure one shows the stages of adoption of an AI system into the real world. Can inclusion of explicit knowledge help XAI provide human-understandable explanations and enable decision making? METHODS FOR EXPLAINABLE AI: OPENING THE BLACK BOX The availability of vast amounts of data and the advent of deep neural network models have accelerated the adoption of AI systems in the real world, owing to their significant success in natural language processing, computer vision, and