Rhetorically Controlled Encoder-Decoder for Modern Chinese Poetry Generation (original) (raw)

A Multi-Modal Chinese Poetry Generation Model

2018 International Joint Conference on Neural Networks (IJCNN)

Recent studies in sequence-to-sequence learning demonstrate that RNN encoder-decoder structure can successfully generate Chinese poetry. However, existing methods can only generate poetry with a given first line or user's intent theme. In this paper, we proposed a three-stage multi-modal Chinese poetry generation approach. Given a picture, the first line, the title and the other lines of the poem are successively generated in three stages. According to the characteristics of Chinese poems, we propose a hierarchy-attention seq2seq model which can effectively capture character, phrase, and sentence information between contexts and improve the symmetry delivered in poems. In addition, the Latent Dirichlet allocation (LDA) model is utilized for title generation and improve the relevance of the whole poem and the title. Compared with strong baseline, the experimental results demonstrate the effectiveness of our approach, using machine evaluations as well as human judgments.

End-to-end style-conditioned poetry generation: What does it take to learn from examples alone?

Proceedings of the 5th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature (LaTeCH-CLfL 2021), 2021

In this work, we design an end-to-end model for poetry generation based on conditioned recurrent neural network (RNN) language models whose goal is to learn stylistic features (poem length, sentiment, alliteration, and rhyming) from examples alone. We show this model successfully learns the ‘meaning' of length and sentiment, as we can control it to generate longer or shorter as well as more positive or more negative poems. However, the model does not grasp sound phenomena like alliteration and rhyming, but instead exploits low-level statistical cues. Possible reasons include the size of the training data, the relatively low frequency and difficulty of these sublexical phenomena as well as model biases. We show that more recent GPT-2 models also have problems learning sublexical phenomena such as rhyming from examples alone.

Haiku Generation A Transformer Based Approach With Lots Of Control

Haiku generation has attracted the interest of the NLP community for decades. After researching different ways of imbuing language models with a poetic sense, we introduce Haikoo, a transformer-based model that outperforms previous state-of-the-art neural networkbased haiku poetry generators. Haikoo consists mainly of two pieces: a GPT-2 model, which we fine-tuned on haiku poems, and a Plug and Play Language Models, which we employ to control the generated results to a further extent than the classic prompt approach. We found that GPT-2 learns successfully to generalize many of the qualities of haiku poetry while retaining enough flexibility to compose poems on entities never seen on the training data. PPLM on the other hand acts as a rudder, moving the generated output towards a specific concept or word. Haikoo helps to search the space of haiku poems satisfying poetic constraints, producing results that make sense to most readers, and that range from lyrical to hilarious.

SP-GPT2: Semantics Improvement in Vietnamese Poetry Generation

2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA), 2021

Automatic text generation has garnered growing attention in recent years as an essential step towards computer creativity. Generative Pre-training Transformer 2 (GPT2) is one of the state-of-the-art approaches that have excellent successes. In this paper, we took the first step to investigate the power of GPT2 in traditional Vietnamese poetry generation. In the earlier time, our experiment with base GPT2 was quite good at generating the poem in the proper template. Though it can learn the patterns, including rhyme and tone rules, from the training data, like almost all other text generation approaches, the poems generated still has a topic drift and semantic inconsistency. To improve the cohesion within the poems, we proposed a new model SP-GPT2 (semantic poem GPT2) which was built on the top GPT2 model and an additional loss to constrain context throughout the entire poem. For better evaluation, we examined the methods by both automatic quantitative evaluation and human evaluation. Both automatic and human evaluation demonstrated that our approach can generate poems that have better cohesion without losing the quality due to additional loss. At the same time, we are the pioneers of this topic. We released the first computational scoring module for poems generated in the template containing the style's rule dictionary. Additionally, we are the first to publish a Luc-Bat dataset, including 87609 Luc Bat poems, which is equivalent to about 2.6 million sentences, combined with about 83579 poems in other styles was also published for further exploration.

Poem Generation using Transformers and Doc2Vec Embeddings

2020 International Joint Conference on Neural Networks (IJCNN)

Poems are sequences of words that express ideas and emotions in an imaginative style, some following strict literary syntax or form. They are artistic expressions, and their generation requires in-depth knowledge and mastery of language. As such, poem generation is considered a very challenging task in Natural Language Processing and has been attracting research interest in the recent decade. We propose a method of generating poems using transformers, coupled with doc2vec embeddings in order to assess the automatically generated poems. In this method, we first train a transformer and a doc2vec model using a poem dataset. Then the trained transformer takes an input text and produces several generated poems. To have an objective basis for assessing the generated poems, we present a preliminary attempt at measuring the quality of a machine-generated poem by computing the cosine similarity score, referenced to the trained doc2vec model. This score is used as a basis for choosing the final output poem. The results show that this method ensures good cohesion between the machine-generated poem and the given input text. We then also explore the implication of the transformer training to the doc2vec embeddings of its output poems, which are shown to be more similar to poems (documents) in the train set as training progresses. Finally, we demonstrate how transformers can learn some poetry styles by exposing them to poems of specific poets.

Full FACE Poetry Generation

Proceedings of the …, 2012

We describe a corpus-based poetry generation system which uses templates to construct poems according to given constraints on rhyme, meter, stress, sentiment, word frequency and word similarity. Moreover, the software constructs a mood for the day by analysing newspaper articles; uses this to determine both an article to base a poem on and a template for the poem; creates an aesthetic based on relevance to the article, lyricism, sentiment and flamboyancy; searches for an instantiation of the template which maximises the aesthetic; and provides a commentary for the whole process to add value to the creative act. We describe the processes behind this approach, present some experimental results which helped in fine tuning, and provide some illustrative poems and commentaries. We argue that this is the first poetry system which generates examples, forms concepts, invents aesthetics and frames its work, and so can be assessed favourably with respect to the FACE model for comparing creative systems.

Computational modelling of poetry generation

Poems are particular well-crafted formulations of certain messages that satisfy constraints on their form as well on their content. From an engineering point of view, the ability to optimize the combination of form and content is a very desireable feature to be able to model. The present paper reviews a number of efforts at modelling poetry generation computationally, and it brings together insights from these efforts into a computational model that allows integration of a number of AI technologies combined together according to control structures compatible with observed human behaviour. The relation of this computational model with existing cognitive models of the writing task, and with models of computational creativity is discussed.

Syllable Neural Language Models for English Poem Generation

2021

Automatic Poem Generation is an ambitious Natural Language Generation (NLG) problem. Models have to replicate a poem’s structure, rhyme and meter, while producing creative and emotional verses. The lack of abundant poetic corpora, especially for archaic poetry, is a serious limitation for the development of strong poem generators. In this paper, we propose a syllable neural language model for the English language, focusing on the generation of verses with the style of a target author: William Wordsworth. To alleviate the problem of limited available data, we exploit transfer learning. Furthermore, we bias the generation of verses according to a combination of different scoring functions based on meter, style and grammar in order to select lines more compliant with the author’s characteristics. The results of both quantitative and human evaluations show the effectiveness of our approach. In particular, human judges struggle to recognize real verses from the generated ones.

PoeTryMe: Towards Meaningful Poetry Generation

PoeTryMe is a poetry generation platform under development that intends to help the automatic generation of meaningful poetry according to a given semantics. It has a versatile architecture that provides a high level of customisation where the user can define features that go from the base semantics and sentence templates to the generation strategy and the poem configuration. A prototype using PoeTryMe was implemented to generate Portuguese poetry. The results are interesting but there is still a long way for improvement, so we devise ideas for future work.