Towards evaluation in natural language generation (original) (raw)
Related papers
Evaluation in the context of natural language generation
Computer Speech & Language, 1998
What role should evaluation play in the development of natural language generation () techniques and systems? In this paper we describe what is involved in natural language generation, and survey how evaluation has figured in work in this area to date. We comment on the issues raised by this existing work and on how the problems of evaluation are different from the problems of evaluating work in natural language understanding. The paper is concluded by suggesting a way forward by looking more closely at the component problems that are addressed in natural language generation research; a particular text generation application is examined and the issues that are raised in assessing its performance on a variety of dimensions are looked at.
This paper surveys the current state of the art in Natural Language Generation (nlg), defined as the task of generating text or speech from non-linguistic input. A survey of nlg is timely in view of the changes that the field has undergone over the past decade or so, especially in relation to new (usually data-driven) methods, as well as new applications of nlg technology. This survey therefore aims to (a) give an up-to-date synthesis of research on the core tasks in nlg and the architectures adopted in which such tasks are organised; (b) highlight a number of relatively recent research topics that have arisen partly as a result of growing synergies between nlg and other areas of artificial intelligence; (c) draw attention to the challenges in nlg evaluation, relating them to similar challenges faced in other areas of nlp, with an emphasis on different evaluation methods and the relationships between them.
A repository of data and evaluation resources for natural language generation
2012
Abstract Starting in 2007, the field of natural language generation (NLG) has organised shared-task evaluation events every year, under the Generation Challenges umbrella. In the course of these shared tasks, a wealth of data has been created, along with associated task definitions and evaluation regimes. In other contexts too, sharable NLG data is now being created.
An Introduction to Natural Language Generation
2003
Topic 1 NLG Overview Course Objectives • to give a broad overview of the field of nlg • to show the state of the art in nlg • to give an overview of the more prominent nlg systems and approaches • to highlight the current major issues in nlg research
Building applied natural language generation systems
1997
In this article, we g i v e a n o verview of Natural Language Generation (nlg) from an applied system-building perspective. The article includes a discussion of when nlg techniques should be used suggestions for carrying out requirements analyses and a description of the basic nlg tasks of content determination, discourse planning, sentence aggregation, lexicalization, referring expression generation, and linguistic realisation. Throughout, the emphasis is on established techniques that can be used to build simple but practical working systems now. We also provide pointers to techniques in the literature that are appropriate for more complicated scenarios.
Proceedings of the 12th European Workshop on Natural Language Generation (ENLG 2009)}
Proceedings of the 12th European Workshop on Natural Language Generation (ENLG 2009)}, 2009
@Book{ENLG:2009, editor = {Emiel Krahmer and Mari\"{e}t Theune}, title = {Proceedings of the 12th European Workshop on Natural Language Generation (ENLG 2009)}, month = {March}, year = {2009}, address = {Athens, Greece}, publisher = {Association for Computational Linguistics}, url = {http://www.aclweb.org/anthology/W09-06} } @InProceedings{reiter-EtAl:2009:ENLG, author = {Reiter, Ehud and Turner, Ross and Alm, Norman and Black, Rolf and Dempster, Martin and Waller, Annalu}, title = {Using {NLG} to Help Language-Impaired Users Tell ...
Natural Language Generation in Artificial Intelligence and Computational Linguistics
The Kluwer International Series in Engineering and Computer Science, 1991
A number of collections of papers from the field of natural language generation (NLG) have been published over the last few years: Kempen (1987), Zock and Sabah (1988), Dale, Mellish, and Zock (1990), and now the present volume. All have in common that they are derived in one way or another from workshops on the subject, and should therefore make available new and often exploratory research in a timely fashion. If such a book is to be more than a conference proceedings, it has to do a little more too, of course; it should present the research in more detail than a conference proceedings would, there should be greater cohesion amongst the papers, and it should be produced to an appropriate standard. The present book, like its predecessors, succeeds on some counts but fails on others. The papers in the book are organized into three strands, described in turn below: text planning, lexical choice, and grammatical resources. The balance between these is rather skewed, however: the first section contains eight papers, and the second and third contain only three papers each.
Natural Language Generation: Scope, Applications and Approaches
Natural Language Generation is a subfield of com- putational linguistic that is concerned with the computer systems which can produce understandable texts in some human lan- guages. The system uses machine understandable logical form as input and produces syntactically and semantically valid sentences in natural language. The different stages of NLG include Content selection, Lexical selection, Sentence structuring and Discourse planning. The applications of NLG include text summarization, machine translation and question answering. The effectiveness of the NLG depends on the efficiency of internal knowledge representation. An ontology based Knowledge representation will improve the output text quality. This work also discusses the scope of applying Karaka relations in language modeling for NLG.
Review on Natural Language Generation
International journal of health sciences
Natural Language Generation is a field evolved from a computational linguistic, the discipline concerned with understanding written and spoken words and building artifacts that usually process and produce language. The emphasis of NLG is on computer systems that can produce understandable texts in human languages. It is one of the fastest growing applications of Artificial Intelligence as it articulately communicates ideas from data at remarkable scale and accuracy. NLG includes variety of application areas such as Healthcare, Finance, Human Resources, Legal, Marketing, Sales, Operations, Strategy, and Supply Chain. The field of NLG has changed drastically in last few years with the emergence of successful deep learning methods. This paper focuses on deep learning techniques and methods used for Natural Language Generation by reviewing some of the recent work done in this direction, mainly in the field of healthcare. In recent times the need for automatic medical report generation h...