Extracting Opinion Targets Using Attention-Based Neural Model (original) (raw)

Attention Mechanism Architecture for Arabic Sentiment Analysis

ACM Transactions on Asian and Low-Resource Language Information Processing

This article tackles the problem of sentiment analysis in the Arabic language where a new deep learning model has been put forward. The proposed model uses a hybrid bidirectional gated recurrent unit (BiGRU) and bidirectional long short-term memory (BiLSTM) additive-attention model where the Bidirectional GRU/LSTM reads the individual sentence input from left to right and vice versa, enabling the capture of the contextual information. However, the model is trained on two types of embeddings: FastText and local learnable embeddings. The BiLSTM and BiGRU architectures are put into competition to identify the best hyperparameter set for the model. The developed model has been tested on three large-scale commonly employed Arabic sentiment dataset: large-scale Arabic Book Reviews Dataset (ABRD), Hotel Arabic-Reviews Dataset (HARD), and Books Reviews in the Arabic Dataset (BRAD). The testing results demonstrate that our model outperforms both the baseline models and the state-of-the-art m...

Aspect-Context Level Information Extraction via Transformer Based Interactive Attention Mechanism for Sentiment Classification

IEEE Access

Aspect-context sentiment classification aims to classify the sentiments about an aspect that corresponds to its context. Typically, machine learning models considers the aspect and context separately. They do not execute the aspect and context in parallel. To model the contexts and aspects separately, most of the methods with attention mechanisms typically employ the Long Short Term Memory network approach. Attention mechanisms, on the other hand, take this into account and compute the parallel sequencing of the aspects-context. The interactive attention mechanism extracts features of a specific aspect regarding its context in the sequence, which means aspects are considered when generating context sequence representations. However, when determining the relationship between words in a sentence, the interactive attention mechanism does not consider semantic dependency information. Moreover, the attention mechanisms did not capture the polysemous words. Normally conventional embedding models, such as GloVe word vectors, have been used. In this study, transformers are embedded into the attention mechanism approaches to overcome the semantic relationship problem. For this reason, the BERT pre-train language model is used to capture the relationship among the words in a sentence. The interactive attention mechanism is then applied to the model's distribution of that word. The final sequence-to-sequence representation in terms of context and aspect is used into general machine learning classifiers for aspect-level sentiment classification. The proposed model was evaluated on the two datasets, i.e., Restaurant and Laptop review. The proposed approach has state-of-the-art results with all attention mechanisms and attained significantly better performance than the existing ones. INDEX TERMS Aspect-context level sentiment classification, transformer based interactive attention mechanism, BERT, BERT interactive attention representation, aspect-context feature extracted data, machine learning classifiers. I. INTRODUCTION Opinions and reviews by gazillion of people are expressed on products, offers, services, etc., [1] via online platforms such as social networks, blogs, wikis, and discussion chatbots. With the arising of social networks, specifically social media, public reviews are constantly generated about something (like a particular political situation, a public person, a product, or a movie). These reviews are generated worldwide and The associate editor coordinating the review of this manuscript and approving it for publication was Rongbo Zhu. are easily accessible via the web. The public follows the opinions on social media, blogs, tweets, YouTube, and many companies evaluate product reviews, preferences, or client satisfaction via these sources. So naturally, the extraction of these types of information, i.e., expressed opinions, has to be significant for marketing, businesses, professionals, and researchers [2], [3]. Society has long been involved in public opinion. Sentiment Analysis (SA) [4], [5], also called Opinion Mining, is a great challenge [6], [7], [8] and it is the most active area for Research in Natural Language Processing (NLP) and

Manifesto of Deep Learning Architecture for Aspect Level Sentiment Analysis to extract customer criticism

ICST transactions on scalable information systems, 2024

Sentiment analysis, a critical task in natural language processing, aims to automatically identify and classify the sentiment expressed in textual data. Aspect-level sentiment analysis focuses on determining sentiment at a more granular level, targeting specific aspects or features within a piece of text. In this paper, we explore various techniques for sentiment analysis, including traditional machine learning approaches and state-of-the-art deep learning models. Additionally, deep learning techniques has been utilized to identifying and extracting specific aspects from text, addressing aspect-level ambiguity, and capturing nuanced sentiments for each aspect. These datasets are valuable for conducting aspect-level sentiment analysis. In this article, we explore a language model based on pre-trained deep neural networks. This model can analyze sequences of text to classify sentiments as positive, negative, or neutral without explicit human labeling. To evaluate these models, data from Twitter's US airlines sentiment database was utilized. Experiments on this dataset reveal that the BERT, RoBERTA and DistilBERT model outperforms than the ML based model in accuracy and is more efficient in terms of training time. Notably, our findings showcase significant advancements over previous state-of-the-art methods that rely on supervised feature learning, bridging existing gaps in sentiment analysis methodologies. Our findings shed light on the advancements and challenges in sentiment analysis, offering insights for future research directions and practical applications in areas such as customer feedback analysis, social media monitoring, and opinion mining.

MULTI-LAYER ATTENTION APPROACH FOR ASPECT BASED SENTIMENT ANALYSIS

Based on the aspect-level sentiment analysis is typical of fine-grained emotional classification that assigns sentiment polarity for each of the aspectsin a review. For better handle the emotion classificationtask,this paper put forward a newmodel which apply Long Short-Term Memory network combine multiple attention with aspect context.Where multiple attention mechanism (i.e., location attention, content attention and class attention) refers to takes the factors of context location, content semantics and class balancing into consideration.Therefore, the proposed model can adaptively integrate location and semantic information between the aspect targets and their contexts into sentimental features, and overcome the model data variance introduced by the imbalanced training dataset. In addition, the aspect context is encoded on both sides of the aspect target, so as to enhance the ability of the model to capture semantic information. The Multi-Attention mechanism (MATT) and Aspect Context (AC) allow our model to perform betterwhen facingreviewswith more complicated structures.The result of this experiment indicate that the accuracy of the new model is up to 80.6% and 75.1% for two datasets in SemEval-2014 Task4 respectively, While the accuracy of the data set on twitter 71.1%, and 81.6% for the Chinese automotive-domain dataset. Compared with some previous models for sentiment analysis, our model shows a higher accuracy

Attention and Lexicon Regularized LSTM for Aspect-based Sentiment Analysis

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop

Attention based deep learning systems have been demonstrated to be the state of the art approach for aspect-level sentiment analysis, however, end-to-end deep neural networks lack flexibility as one can not easily adjust the network to fix an obvious problem, especially when more training data is not available: e.g. when it always predicts positive when seeing the word disappointed. Meanwhile, it is less stressed that attention mechanism is likely to "over-focus" on particular parts of a sentence, while ignoring positions which provide key information for judging the polarity. In this paper, we describe a simple yet effective approach to leverage lexicon information so that the model becomes more flexible and robust. We also explore the effect of regularizing attention vectors to allow the network to have a broader "focus" on different parts of the sentence. The experimental results demonstrate the effectiveness of our approach.

Deep Learning for Aspect-Based Sentiment Analysis on Indonesian Hotels Reviews

Kinetik : game technology, information system, computer network, computing, electronics, and control, 2021

Tourism is one of the fastest-growing industries. Many travelers book hotels and share their experiences using travel e-commerce sites. To improve the quality of products and services, we can take advantage by analyzing their reviews. We can see the good and the bad thing reviews in every aspect of the hotel. However, research to analyze sentiment in every aspect using Indonesian hotel reviews is still relatively new. In this work, we propose to create an Aspect-based Sentiment Analysis (ABSA) using Indonesian hotel reviews to solve the problem. This research consists of four steps: collecting data, preprocessing, aspect classification, and sentiment classification. Our classification process compares with eight deep learning methods (RNN, LSTM, GRU, BiLSTM, Attention BiLSTM, CNN, CNN-LSTM, and CNN-BiLSTM). In aspect classification, we have six classes of aspects which are harga (price), hotel, kamar (room), lokasi (location), pelayanan (service), and restoran (restaurant). In sentiment analysis, we compared two scenarios to classify sentiments as positive or negative. The first one is to classify sentiment in all aspects, and the second one is to classify sentiment in every aspect. The results showed that LSTM achieved the best model for aspect classification with an accuracy value of 0.926. For sentiment classification, our experiments showed that classify sentiment in every aspect achieved a better result than classify sentiment in all aspects. The result showed that the CNN model gets an average accuracy score of 0.904.

Attention-based aspect sentiment classification using enhanced learning through cnn-Bilstm networks

Knowledge-Based Systems, 2022

Deep neural networks (DNN) techniques for aspect-based sentiment classification have been widely studied. The success of these methods depends largely on training data which are often inadequate because of the rigor involved in manually tagging large collection of opinionated texts. Attempts have been made to transfer knowledge from document-level to aspect-level sentiment task. However, the success of this approach is also dependent on the model because aspect sentiment data like other type of texts contain complex semantic features. In this paper, we present an attention-based deep learning technique which jointly learns on document and aspect-level sentiment data and which also transfers learning from the document-level data to aspect-level sentiment classification. It basically consists of a convolutional layer and a bidirectional long short-term memory (BiLSTM) layer. The first variant of our technique uses convolutional neural network (CNN) to extract high-level semantic features. The output of the feature extraction is then fed into the BiLSTM layer which captures the contextual feature representation of the texts. The second variant applies the BiLSTM layer directly on the input data. In both variants, the output hidden representation is passed to an output layer using softmax activation function for sentiment polarity classification. We evaluate our model on four standard benchmark datasets which shows the effectiveness of our approach with improvements over baselines. We also conduct ablation studies to show the effect of the different document-level weights on the learning techniques.

A Multi-Layer Dual Attention Deep Learning Model with Refined Word Embeddings for Aspect-Based Sentiment Analysis

IEEE Access

Although the sentiment analysis domain has been deeply studied in last few years, the analysis of social media content is still a challenging task due to exponential growth of the multimedia content. Natural language ambiguities and indirect sentiments within the social media text has made it hard to classify. Aspect based sentiment analysis creates a need to develop explicit extraction techniques using syntactic parsers to exploit the relationship between aspect and sentiment terms. Along-with the extraction approaches, word embeddings are generated through Word2Vec models for the continuous low-dimensional vector representation of text that fails to capture the significant sentiment information. This paper presents a coextraction model with refined word embeddings to exploit the dependency structures without using syntactic parsers. For this purpose, a deep learning based multilayer dual attention model is proposed to exploit the indirect relation between the aspect and opinion terms. In addition, word embeddings are refined by providing distinct vector representations to dissimilar sentiments unlike Word2Vec model. For this we employed a sentiment refinement technique for pre-trained word embedding model to overcome the problem of similar vector representations of opposite sentiments. Performance of the proposed model is evaluated on three benchmark datasets of SemEval Challenge 2014 and 2015. Experimental results indicate the effectiveness of our model as compared to existing state-of-the-art models for aspect-based sentiment analysis.

Toward a multitask aspect-based sentiment analysis model using deep learning

IAES International Journal of Artificial Intelligence (IJ-AI)

Sentiment analysis or opinion mining is used to understand the community’s opinions on a particular product. This is a system of selection and classification of opinions on sentences or documents. At a more detailed level, aspect-based sentiment analysis makes an effort to extract and categorize sentiments on aspects of entities in opinion text. In this paper, we propose a novel supervised learning approach using deep learning techniques for a multitasking aspect-based opinion mining system that supports four main subtasks: extract opinion target, classify aspect, classify entity (category) and estimate opinion polarity (positive, neutral, negative) on each extracted aspect of the entity. We have used a part-of-speech (POS) layer to define the words’ morphological features integrated with GloVe word embedding in the previous layer and fed to the convolutional neural network_bidirectional long-short term memory (CNN_BiLSTM) stacked construction to improve the model’s accuracy in the ...

A Memory-Driven Neural Attention Model for Aspect-Based Sentiment Classification

Journal of Web Engineering, 2022

Sentiment analysis techniques are becoming more and more important as the number of reviews on the World Wide Web keeps increasing. Aspect-based sentiment analysis (ABSA) entails the automatic analysis of sentiments at the highly fine-grained aspect level. One of the challenges of ABSA is to identify the correct sentiment expressed towards every aspect in a sentence. In this paper, a neural attention model is discussed and three extensions are proposed to this model. First, the strengths and weaknesses of the highly successful CABASC model are discussed, and three shortcomings are identified: the aspect-representation is poor, the current attention mechanism can be extended for dealing with polysemy in natural language, and the design of the aspect-specific sentence representation is upheld by a weak construction. We propose the Extended CABASC (E-CABASC) model, which aims to solve all three of these problems. The model incorporates a context-aware aspect representation, a multi-dimensional attention mechanism, and an aspect-specific sentence representation. The main contribution of this work is that it is shown that attention models can be improved upon using some relatively simple extensions, such as fusion gates and multi-dimensional attention, which can be implemented in many state-of-the-art models. Additionally, an analysis of the parameters and attention weights is provided.