Generating Topic-Oriented Summaries Using Neural Attention (original) (raw)
Related papers
Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond
Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, 2016
In this work, we model abstractive text summarization using Attentional Encoder-Decoder Recurrent Neural Networks, and show that they achieve state-of-the-art performance on two different corpora. We propose several novel models that address critical problems in summarization that are not adequately modeled by the basic architecture, such as modeling keywords , capturing the hierarchy of sentence-toword structure, and emitting words that are rare or unseen at training time. Our work shows that many of our proposed models contribute to further improvement in performance. We also propose a new dataset consisting of multi-sentence summaries, and establish performance benchmarks for further research.
Abstractive Sentence Summarization with Attentive Recurrent Neural Networks
Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
ive Sentence Summarization generates a shorter version of a given sentence while attempting to preserve its meaning. We introduce a conditional recurrent neural network (RNN) which generates a summary of an input sentence. The conditioning is provided by a novel convolutional attention-based encoder which ensures that the decoder focuses on the appropriate input words at each step of generation. Our model relies only on learned features and is easy to train in an end-to-end fashion on large data sets. Our experiments show that the model significantly outperforms the recently proposed state-of-the-art method on the Gigaword corpus while performing competitively on the DUC-2004 shared task.
Neural Attention Model for Abstractive Text Summarization Using Linguistic Feature Space
IEEE Access
Summarization generates a brief and concise summary which portrays the main idea of the source text. There are two forms of summarization: abstractive and extractive. Extractive summarization chooses important sentences from the text to form a summary whereas abstractive summarization paraphrase using advanced and nearer-to human explanation by adding novel words or phrases. For a human annotator, producing summary of a document is time consuming and expensive because it requires going through the long document and composing a short summary. An automatic feature-rich model for text summarization is proposed that can reduce the amount of labor and produce a quick summary by using both extractive and abstractive approach. A feature-rich extractor highlights the important sentences in the text and linguistic characteristics are used to enhance results. The extracted summary is then fed to an abstracter to further provide information using features such as named entity tags, part of speech tags and term weights. Furthermore, a loss function is introduced to normalize the inconsistency between word-level and sentencelevel attentions. The proposed two-staged network achieved a ROUGE score of 37.76% on the benchmark CNN/DailyMail dataset, outperforming the earlier work. Human evaluation is also conducted to measure the comprehensiveness, conciseness and informativeness of the generated summary.
A Neural Attention Model for Sentence Summarization
2015
Summarization based on text extraction is inherently limited, but generation-style ab-stractive methods have proven challeng-ing to build. In this work, we propose a fully data-driven approach to abstrac-tive sentence summarization. Our method utilizes a local attention-based model that generates each word of the summary con-ditioned on the input sentence. While the model is structurally simple, it can eas-ily be trained end-to-end and scales to a large amount of training data. The model shows significant performance gains on the DUC-2004 shared task compared with several strong baselines. 1
2017
We present SummaRuNNer, a Recurrent Neural Network (RNN) based sequence model for extractive summarization of documents and show that it achieves performance better than or comparable to state-of-the-art. Our model has the additional advantage of being very interpretable, since it allows visualization of its predictions broken up by abstract features such as information content, salience and novelty. Another novel contribution of our work is abstractive training of our extractive model that can train on human generated reference summaries alone, eliminating the need for sentence-level extractive labels.
Improving Neural Abstractive Text Summarization with Prior Knowledge (Position Paper)
International Conference of the Italian Association for Artificial Intelligence, 2016
ive text summarization is a complex task whose goal is to generate a concise version of a text without necessarily reusing the sentences from the original source, but still preserving the meaning and the key contents. In this position paper we address this issue by modeling the problem as a sequence to sequence learning and exploiting Recurrent Neural Networks (RNN). Moreover, we discuss the idea of combining RNNs and probabilistic models in a unified way in order to incorporate prior knowledge, such as linguistic features. We believe that this approach can obtain better performance than the state-of-the-art models for generating well-formed summaries.
Assessing the Efficacy of LSTM, Transformer, and RNN Architectures in Text Summarization
International Conference on Applied Engineering and Natural Sciences
The need for efficient and effective techniques for automatic text summarization has become increasingly critical with the exponential growth of textual data in different domains. Summarizing long texts into short summaries facilitates a quick understanding of the key information contained in the documents. In this paper, we evaluate various architectures for automatic text summarization using the TEDx dataset, a valuable resource consisting of a large collection of TED talks with rich and informative speech transcripts. Our research focuses on evaluating the performance of Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), Recurrent Neural Network (RNN) and Transformer architectures for automatic text summarization. We measure the accuracy of each model by comparing the generated summaries with human-written summaries. The findings show that the Transformer model achieves the highest accuracy, followed closely by the GRU model. However, LSTM, RNN exhibit relatively lower ac...
Summary Generation Using Deep Learning
IEEE EXPLORE, 2021
Summarization is the process of converting any given long text into a lossy compressed version while preserving the overall essence and meaning of the original given text. A problem in natural language processing is the task of summarizing a voluminous text. This research, lays emphasis on abstractive summarization of the text, a process which generates a coherent, short and meaningful text by learning the context from given input source text. This work is a preliminary attempt to generate abstractive summary. The proposed work uses the Attentional Encoder and Decoder based Sequence to Sequence Recurrent Neural Network model. The proposed research's primary objective is to get insights on the working of the mentioned Sequence architecture and the intricacies of different subcomponents of the encoders and the decoders architecture working together so as to give a particular end result. The scope of this paper covers an experimentation of performing text summarization process using Encoder and Decoder based Sequential Learning Recurrent Neural Network design. Satisfactory summary results are obtained after the model is experimented and analyzed on the Amazon Fine Food reviews.
SHEG: summarization and headline generation of news articles using deep learning
Neural Computing and Applications, 2020
The human attention span is continuously decreasing, and the amount of time a person wants to spend on reading is declining at an alarming rate. Therefore, it is imperative to provide a quick glance of important news by generating a concise summary of the prominent news article, along with the most intuitive headline in line with the summary. When humans produce summaries of documents, they not only extract phrases and concatenate them but also produce new grammatical phrases or sentences that coincide with each other and capture the most significant information of the original article. Humans have an incredible ability to create abstractions; however, automatic summarization is a challenging problem. This paper aims to develop an end-to-end methodology that can generate brief summaries and crisp headlines that can capture the attention of readers and convey a significant amount of relevant information. In this paper, we propose a novel methodology known as SHEG, which is designed as a hybrid model. It works by integrating both extractive and abstractive mechanisms using a pipelined approach to produce a concise summary, which is then used for headline generation. Experiments were performed on publicly available datasets, viz. CNN/Daily Mail, Gigaword, and NEWS-ROOM. The results obtained validate our approach and demonstrate that the proposed SHEG model is effectively producing a concise summary as well as a captivating and fitting headline.
NUTS: Network for Unsupervised Telegraphic Summarization
2018
Extractive summarization methods operate by ranking and selecting the sentences which best encapsulate the theme of a given document. They do not fare well in domains like fictional narratives where there is no central theme and core information is not encapsulated by a small set of sentences. For the purpose of reducing the size of the document while conveying the idea expressed by each sentence, we need more sentence specific methods. Telegraphic summarization, which selects short segments across several sentences, is better suited for such domains. Telegraphic summarization captures the plot better by retaining shorter versions of each sentence while not really concerning itself with grammatically linking these segments. In this paper, we propose an unsupervised deep learning network (NUTS) to generate telegraphic summaries. We use multiple encoderdecoder networks and learn to drop portions of the text that are inferable from the chosen segments. The model is agnostic to both sen...