CNN- and LSTM-based claim classification in online user comments (original) (raw)
Related papers
Using Convolutional Neural Network in Cross-Domain Argumentation Mining Framework
Lecture Notes in Computer Science, 2019
Argument Mining has become a remarkable research area in computational argumentation and Natural Language Processing fields. Despite its importance, most of the current proposals are restricted to a text type (e.g., Essays, web comments) on a specific domain and fall behind expectations when applied to cross-domain data. This paper presents a new framework for Argumentation Mining to detect argumentative segments and their components automatically using Convolutional Neural Network (CNN). We focus on both (1) argumentative sentence detection and (2) argument components detection tasks. Based on different corpora, we investigate the performance of CNN on both in-domain level and cross-domain level. The investigation shows challenging results in comparison with classic machine learning models.
2018
We predict which claim in a political debate should be prioritized for fact-checking. A particular challenge is, given a debate, how to produce a ranked list of its sentences based on their worthiness for fact checking. We develop a Recurrent Neural Network (RNN) model that learns a sentence embedding, which is then used to predict the checkworthiness of a sentence. Our sentence embedding encodes both semantic and syntactic dependencies using pretrained word2vec word embeddings as well as part-of-speech tagging and syntactic dependency parsing. This results in a multi-representation of each word, which we use as input to a RNN with GRU memory units; the output from each word is aggregated using attention, followed by a fully connected layer, from which the output is predicted using a sigmoid function. The overall performance of our techniques is successful, achieving the overall second best performing run (MAP: 0.1152) in the competition, as well as the highest overall performance (...
We develop a novel unified representation for the argumentation mining task facilitating the extracting from text and the labelling of the non-argumentative units and argumentation components-premises, claims, and major claims-and the argumentative relations-premise to claim or premise in a support or attack relation, and claim to major-claim in a for or against relation-in an end-to-end machine learning pipeline. This tightly integrated representation combines the component and relation identification subproblems and enables a unitary solution for detecting argumentation structures. This new representation together with a new deep learning architecture composed of a mixed embedding method, a multi-head attention layer, two biLSTM layers, and a final linear layer obtain state-of-the-art accuracy on the Persuasive Essays dataset. An augmentation of the corpus (Paragraph version) by including copies of major claims has further increased the performance.
2021
In this paper we investigate the efficacy of using contextual embeddings from multilingual BERT and German BERT in identifying fact-claiming comments in German on social media. Additionally, we examine the impact of formulating the classification problem as a multi-task learning problem, where the model identifies toxicity and engagement of the comment in addition to identifying whether it is fact-claiming. We provide a thorough comparison of the two BERT based models compared with a logistic regression baseline and show that German BERT features trained using a multi-task objective achieves the best F1 score on the test set. This work was done as part of a submission to GermEval 2021 shared task on the identification of fact-claiming comments.
Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, 2021
The conceptualization of a claim lies at the core of argument mining. The segregation of claims is complex, owing to the divergence in textual syntax and context across different distributions. Another pressing issue is the unavailability of labeled unstructured text for experimentation. In this paper, we propose LESA, a framework which aims at advancing headfirst into expunging the former issue by assembling a source-independent generalized model that captures syntactic features through part-of-speech and dependency embeddings, as well as contextual features through a fine-tuned language model. We resolve the latter issue by annotating a Twitter dataset which aims at providing a testing ground on a large unstructured dataset. Experimental results show that LESA improves upon the state-of-the-art performance across six benchmark claim datasets by an average of 3 claim-F1 points for in-domain experiments and by 2 claim-F1 points for general-domain experiments. On our dataset too, LESA outperforms existing baselines by 1 claim-F1 point on the in-domain experiments and 2 claim-F1 points on the general-domain experiments. We also release comprehensive data annotation guidelines compiled during the annotation phase (which was missing in the current literature).
2022
We develop a novel unified representation for the argumentation mining task facilitating the extracting from text and the labelling of the non-argumentative units and argumentation components-premises, claims, and major claims-and the argumentative relationspremise to claim or premise in a support or attack relation, and claim to major-claim in a for or against relation-in an end-to-end machine learning pipeline. This tightly integrated representation combines the component and relation identification sub-problems and enables a unitary solution for detecting argumentation structures. This new representation together with a new deep learning architecture composed of a mixed embedding method, a multi-head attention layer, two biLSTM layers, and a final linear layer obtain state-of-the-art accuracy on the Persuasive Essays dataset. Also, we have introduced a decoupled solution to identify the entities and relations first, and on top of that a second model is used to detect distance between the detected related components. An augmentation of the corpus (paragraph version) by including copies of major claims has further increased the performance.
Procedia Computer Science, 2017
We propose a novel approach in a dataset of argumentation relations. This task is intended to analyze the presence of a support relation between two sentences. To be able to identify relations between two sentences or arguments, one is obliged to understand the nuance brought by both sentences. Our models are modification of siamese network architectures, in which we replace the feature extractor into Long Short Term Memory and implement cosine distance as the energy function. Our models take a pair of sentences as their input and try to identify whether there is a support relation between those two sentences or not.The primary motivation of this research is to prove that a high degree of similarity between two sentences correlates to sentences supporting each other. This work will focus more on the modification of siamese network and the implementation of attention mechanism. Due to the difference in dataset setting, we cannot arbitrarily compare our results with the prior research results. Therefore, this work will not highlight the comparison between deep learning and traditional machine learning algorithm per se, but it will be more of an exploratory research. Our models are able to outperform the baseline score of accuracy with a margin of 17.33% (67.33%). By surpassing the baseline performance,we believe that our work can be a stepping stone for deep learning implementation in argumentation mining field.
Cornell University - arXiv, 2020
In this digital age of news consumption, a news reader has the ability to react, express and share opinions with others in a highly interactive and fast manner. As a consequence, fake news has made its way into our daily life because of very limited capacity to verify news on the Internet by large companies as well as individuals. In this paper, we focus on solving two problems which are part of the factchecking ecosystem that can help to automate fact-checking of claims in an ever increasing stream of content on social media. For the first problem, claim check-worthiness prediction, we explore the fusion of syntactic features and deep transformer Bidirectional Encoder Representations from Transformers (BERT) embeddings, to classify check-worthiness of a tweet, i.e. whether it includes a claim or not. We conduct a detailed feature analysis and present our best performing models for English and Arabic tweets. For the second problem, claim retrieval, we explore the pre-trained embeddings from a Siamese network transformer model (sentence-transformers) specifically trained for semantic textual similarity, and perform KD-search to retrieve verified claims with respect to a query tweet.
Lecture Notes in Computer Science, 2018
We present an overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims, with focus on Task 1: Check-Worthiness. The task asks to predict which claims in a political debate should be prioritized for fact-checking. In particular, given a debate or a political speech, the goal was to produce a ranked list of its sentences based on their worthiness for fact checking. We offered the task in both English and Arabic, based on debates from the 2016 US Presidential Campaign, as well as on some speeches during and after the campaign. A total of 30 teams registered to participate in the Lab and seven teams actually submitted systems for Task 1. The most successful approaches used by the participants relied on recurrent and multi-layer neural networks, as well as on combinations of distributional representations, on matchings claims' vocabulary against lexicons, and on measures of syntactic dependency. The best systems achieved mean average precision of 0.18 and 0.15 on the English and on the Arabic test datasets, respectively. This leaves large room for further improvement, and thus we release all datasets and the scoring scripts, which should enable further research in check-worthiness estimation.