Neural Language Generation: Formulation, Methods, and Evaluation (original) (raw)
Related papers
A Systematic Literature Review on Text Generation Using Deep Neural Network Models
IEEE Access
In recent years, significant progress has been made in text generation. The latest text generation models are revolutionizing the domain by generating human-like text. It has gained wide popularity recently in many domains like news, social networks, movie scriptwriting, and poetry composition, to name a few. The application of text generation in various fields has resulted in a lot of interest from the scientific community in this area. To the best of our knowledge, there is a lack of extensive review and an up-to-date body of knowledge of text generation deep learning models. Therefore, this survey aims to bring together all the relevant work in a systematic mapping study highlighting key contributions from various researchers over the years, focusing on the past, present, and future trends. In this work, we have identified 90 primary studies from 2015 to 2021 employing the PRISMA framework. We also identified research gaps that are further needed to be explored by the research community. In the end, we provide some future directions for researchers and guidelines for practitioners based on the findings of this review.
Review on Natural Language Generation
International journal of health sciences
Natural Language Generation is a field evolved from a computational linguistic, the discipline concerned with understanding written and spoken words and building artifacts that usually process and produce language. The emphasis of NLG is on computer systems that can produce understandable texts in human languages. It is one of the fastest growing applications of Artificial Intelligence as it articulately communicates ideas from data at remarkable scale and accuracy. NLG includes variety of application areas such as Healthcare, Finance, Human Resources, Legal, Marketing, Sales, Operations, Strategy, and Supply Chain. The field of NLG has changed drastically in last few years with the emergence of successful deep learning methods. This paper focuses on deep learning techniques and methods used for Natural Language Generation by reviewing some of the recent work done in this direction, mainly in the field of healthcare. In recent times the need for automatic medical report generation h...
The Survey: Text Generation Models in Deep Learning
Journal of King Saud University - Computer and Information Sciences
This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Journal of Artificial Intelligence Research
Developing artificial learning systems that can understand and generate natural language has been one of the long-standing goals of artificial intelligence. Recent decades have witnessed an impressive progress on both of these problems, giving rise to a new family of approaches. Especially, the advances in deep learning over the past couple of years have led to neural approaches to natural language generation (NLG). These methods combine generative language learning techniques with neural-networks based frameworks. With a wide range of applications in natural language processing, neural NLG (NNLG) is a new and fast growing field of research. In this state-of-the-art report, we investigate the recent developments and applications of NNLG in its full extent from a multidimensional view, covering critical perspectives such as multimodality, multilinguality, controllability and learning strategies. We summarize the fundamental building blocks of NNLG approaches from these aspects and pr...
Learning Natural Language Generation from Scratch
2021
This paper introduces TRUncated ReinForcement Learning for Language (TrufLL), an original approach to train conditional language models from scratch by only using reinforcement learning (RL). As RL methods unsuccessfully scale to large action spaces, we dynamically truncate the vocabulary space using a generic language model. TrufLL thus enables to train a language agent by solely interacting with its environment without any task-specific prior knowledge; it is only guided with a task-agnostic language model. Interestingly, this approach avoids the dependency to labelled datasets and inherently reduces pretrained policy flaws such as language or exposure biases. We evaluate TrufLL on two visual question generation tasks, for which we report positive results over performance and language metrics, which we then corroborate with a human evaluation. To our knowledge, it is the first approach that successfully learns a language generation policy (almost) from scratch.
Toward Natural Language Generation by Humans
Intelligent Narrative Technologies, 2015
Natural language generation (NLG) has been featured in at most a handful of shipped games and interactive stories. This is certainly due to it being a very specialized practice, but another contributing factor is that the state of the art today, in terms of content quality, is simply inadequate. The major benefits of NLG are its alleviation of authorial burden and the capability it gives to a system of generating state-bespoke content, but we believe we can have these benefits without actually employing a full NLG pipeline. In this paper, we present the preliminary design of Expressionist, an in-development mixed-initiative authoring tool that instantiates an authoring scheme residing somewhere between conventional NLG and conventional human content authoring. In this scheme, a human author plays the part of an NLG module in that she starts from a set of deep representations constructed for the game or story domain and proceeds to specify dialogic content that may express those representations. Rather than authoring static dialogue, the author defines a probabilistic context-free grammar that yields templated dialogue. This allows a human author to still harness a computer's generativity, but in a capacity in which it can be trusted: operating over probabilities and treelike control structures. Additional features of Expressionist's design include arbitrary markup and realtime feedback showing currently valid derivations.
Natural Language Generation The State of the Art and the Concept of Ph
2020
Computational systems use natural language for communication with humans more often in the last years. This work summarises state-of-the-art approaches in the field of generative models, especially in the text domain. It offers a complex study of specific problems known from this domain and related ones like adversarial training, reinforcement learning, artificial neural networks, etc. It also addresses the usage of these models in the context of non-generative approaches and the possibility of combining both. This work was supported by Grant No. SGS-2019-018 Processing of heterogeneous data and its specialized applications. Copies of this report are available on http://www.kiv.zcu.cz/en/research/publications/ or by surface mail on request sent to the following address: University of West Bohemia Department of Computer Science and Engineering Univerzitní 8 30614 Plzeň Czech Republic Copyright c ○ 2020 University of West Bohemia, Czech Republic
This paper surveys the current state of the art in Natural Language Generation (nlg), defined as the task of generating text or speech from non-linguistic input. A survey of nlg is timely in view of the changes that the field has undergone over the past decade or so, especially in relation to new (usually data-driven) methods, as well as new applications of nlg technology. This survey therefore aims to (a) give an up-to-date synthesis of research on the core tasks in nlg and the architectures adopted in which such tasks are organised; (b) highlight a number of relatively recent research topics that have arisen partly as a result of growing synergies between nlg and other areas of artificial intelligence; (c) draw attention to the challenges in nlg evaluation, relating them to similar challenges faced in other areas of nlp, with an emphasis on different evaluation methods and the relationships between them.
Controlled Text Generation with Natural Language Instructions
arXiv (Cornell University), 2023
Large language models can be prompted to produce fluent output for a wide range of tasks without being specifically trained to do so. Nevertheless, it is notoriously difficult to control their generation in such a way that it satisfies userspecified constraints. In this paper, we present IN-STRUCTCTG, a simple controlled text generation framework that incorporates different constraints by verbalizing them as natural language instructions. We annotate natural texts through a combination of off-the-shelf NLP tools and simple heuristics with the linguistic and extra-linguistic constraints they satisfy. Then, we verbalize the constraints into natural language instructions to form weakly supervised training data, i.e., we prepend the natural language verbalizations of the constraints in front of their corresponding natural language sentences. Next, we fine-tune a pretrained language model on the augmented corpus. Compared to existing methods, INSTRUCTCTG is more flexible in terms of the types of constraints it allows the practitioner to use. It also does not require any modification of the decoding procedure. Finally, INSTRUCTCTG allows the model to adapt to new constraints without retraining through the use of in-context learning.