Feature Fusion with Hand-crafted and Transfer Learning Embeddings for Cause-Effect Relation Extraction (original) (raw)
Related papers
Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue, 2018
In this paper we have proposed a linguistically informed recursive neural network architecture for automatic extraction of cause-effect relations from text. These relations can be expressed in arbitrarily complex ways. The architecture uses word level embeddings and other linguistic features to detect causal events and their effects mentioned within a sentence. The extracted events and their relations are used to build a causal-graph after clustering and appropriate generalization, which is then used for predictive purposes. We have evaluated the performance of the proposed extraction model with respect to two baseline systems,one a rule-based classifier, and the other a conditional random field (CRF) based supervised model. We have also compared our results with related work reported in the past by other authors on SEMEVAL data set, and found that the proposed bidirectional LSTM model enhanced with an additional linguistic layer performs better. We have also worked extensively on creating new annotated datasets from publicly available data, which we are willing to share with the community.
A Joint Model for Detecting Causal Sentences and Cause-Effect Relations from Text
IOS Press eBooks, 2022
Text documents are rich repositories of causal knowledge. While journal publications typically contain analytical explanations of observations on the basis of scientific experiments conducted by researchers, analyst reports, News articles or even consumer generated text contain not only viewpoints of authors, but often contain causal explanations for those viewpoints. As interest in data science shifts towards understanding causality rather than mere correlations, there is also a surging interest in extracting causal constructs from text to provide augmented information for better decision making. Causality extraction from text is viewed as a relation extraction problem which requires identification of causal sentences as well as detection of cause and effect clauses separately. In this paper, we present a joint model for causal sentence classification and extraction of cause and effect clauses, using a sequence-labeling architecture cascaded with fine-tuned Bidirectional Encoder Representations from Transformers (BERT) language model. The cause and effect clauses are further processed to identify named entities and build a causal graph using domain constraints. We have done multiple experiments to assess the generalizability of the model. It is observed that when fine-tuned with sentences from a mixed corpus, and further trained to solve both the tasks correctly, the model learns the nuances of expressing causality independent of the domain. The proposed model has been evaluated against multiple state-of-the-art models proposed in literature and found to outperform them all.
A survey on extraction of causal relations from natural language text
Knowledge and Information Systems, 2022
As an essential component of human cognition, cause–effect relations appear frequently in text, and curating cause–effect relations from text helps in building causal networks for predictive tasks. Existing causality extraction techniques include knowledge-based, statistical machine learning (ML)-based, and deep learning-based approaches. Each method has its advantages and weaknesses. For example, knowledge-based methods are understandable but require extensive manual domain knowledge and have poor cross-domain applicability. Statistical machine learning methods are more automated because of natural language processing (NLP) toolkits. However, feature engineering is labor-intensive, and toolkits may lead to error propagation. In the past few years, deep learning techniques attract substantial attention from NLP researchers because of its powerful representation learning ability and the rapid increase in computational resources. Their limitations include high computational costs and ...
Automatic Extraction of Cause-Effect Relations in Natural Language Text
2013
The discovery of causal relations from text has been studied adopting various approaches based on rules or Machine Learning (ML) techniques. The approach proposed joins both rules and ML methods to combine the advantage of each one. In particular, our approach first identifies a set of plausible cause-effect pairs through a set of logical rules based on dependencies between words then it uses Bayesian inference to reduce the number of pairs produced by ambiguous patterns. The SemEval-2010 task 8 dataset challenge has been used to evaluate our model. The results demonstrate the ability of the rules for the relation extraction and the improvements made by the filtering process.
Automatic Extraction of Causal Relations from Natural Language Texts: A Comprehensive Survey
Automatic extraction of cause-effect relationships from natural language texts is a challenging open problem in Artificial Intelligence. Most of the early attempts at its solution used manually constructed linguistic and syntactic rules on small and domain-specific data sets. However, with the advent of big data, the availability of affordable computing power and the recent popularization of machine learning, the paradigm to tackle this problem has slowly shifted. Machines are now expected to learn generic causal extraction rules from labelled data with minimal supervision, in a domain independent-manner. In this paper, we provide a comprehensive survey of causal relation extraction techniques from both paradigms, and analyse their relative strengths and weaknesses, with recommendations for future work.
Causality Mining in Natural Languages Using Machine and Deep Learning Techniques: A Survey
Applied Sciences, 2021
The era of big textual corpora and machine learning technologies have paved the way for researchers in numerous data mining fields. Among them, causality mining (CM) from textual data has become a significant area of concern and has more attention from researchers. Causality (cause-effect relations) serves as an essential category of relationships, which plays a significant role in question answering, future events predication, discourse comprehension, decision making, future scenario generation, medical text mining, behavior prediction, and textual prediction entailment. While, decades of development techniques for CM are still prone to performance enhancement, especially for ambiguous and implicitly expressed causalities. The ineffectiveness of the early attempts is mainly due to small, ambiguous, heterogeneous, and domain-specific datasets constructed by manually linguistic and syntactic rules. Many researchers have deployed shallow machine learning (ML) and deep learning (DL) te...
2020
Causality can be understood as the relationship between two events such that the occurrence of one event results in the occurrence of the other event either directly or indirectly. This paper aims to identify whether the given sentences have a causality effect present in them and to classify the cause and effect words/phrases if present. The approach used for classification uses deep learning algorithms and the annotation task uses machine translation. These models are applied to the dataset provided by CEREX@FIRE2020. The best result for the causality identification part was obtained from BiLSTM with an F1 score of 0.60 and for the second task of annotation as cause and effect, NMT with Bahdanau attention mechanism with an F1 score of 0.44.
Causal BERT: Language Models for Causality Detection Between Events Expressed in Text
Lecture Notes in Networks and Systems, 2021
Causality understanding between events is a critical natural language processing task that is helpful in many areas, including health care, business risk management and finance. On close examination, one can find a huge amount of textual content both in the form of formal documents or in content arising from social media like twitter, dedicated to communicating and exploring various types of causality in the real world. Recognizing these "Cause-Effect" relationships between natural language events continues to remain a challenge simply because it is often expressed implicitly. Implicit causality is hard to detect through most of the techniques employed in literature and can also, at times be perceived as ambiguous or vague. Also, although well known datasets do exist for this problem, the examples in them are limited in the range and complexity of the causal relationships they depict especially when related to implicit relationships. Most of the contemporary methods are either based on lexico-semantic pattern matching or are feature-driven supervised methods. Therefore, as expected these methods are more geared towards handling explicit causal relationships leading to limited coverage for implicit relationships and are hard to generalize. In this paper, we investigate the language model's capabilities for causal association among events expressed in natural language text using sentence context combined with event information, and by leveraging masked event context with in-domain and out-of-domain data distribution. Our proposed methods achieve the state-of-art performance in three different data distributions and can be leveraged for extraction of a causal diagram and/or building a chain of events from unstructured text.
Knowledge-Augmented Language Models for Cause-Effect Relation Classification
Proceedings of the First Workshop on Commonsense Representation and Reasoning (CSRR 2022)
Previous studies have shown the efficacy of knowledge augmentation methods in pretrained language models. However, these methods behave differently across domains and downstream tasks. In this work, we investigate the augmentation of pretrained language models with knowledge graph data in the causeeffect relation classification and commonsense causal reasoning tasks. After automatically verbalizing triples in ATOMIC 20 20 , a wide coverage commonsense reasoning knowledge graph, we continually pretrain BERT and evaluate the resulting model on cause-effect pair classification and answering commonsense causal reasoning questions. Our results show that a continually pretrained language model augmented with commonsense reasoning knowledge outperforms our baselines on two commonsense causal reasoning benchmarks, COPA and BCOPA-CE, and a Temporal and Causal Reasoning (TCR) dataset, without additional improvement in model architecture or using quality-enhanced data for fine-tuning.