Does Collaborative Human-LM Dialogue Generation Help Information Extraction from Human Dialogues? (original) (raw)
Abstract:The capabilities of pretrained language models have opened opportunities to explore new application areas, but applications involving human-human interaction are limited by the fact that most data is protected from public release for privacy reasons. Problem-solving human dialogues in real applications can be much more complex than existing Wizard-of-Oz collections, preventing successful domain transfer. To support information extraction (IE) for a private call center dataset, we introduce a human-in-the-loop dialogue generation framework capable of synthesizing realistic dialogues. In IE experiments with auto insurance call center dialogues, we observe 25\% relative improvement in F_1F_1F_1 after augmenting a small set of real human conversations with synthetic data. We release code and our synthetic dataset to illustrate the complexity of real-world call center conversations and encourage development of complex dialogue datasets that are more representative of natural data.
Submission history
From: Bo-Ru Lu [view email]
[v1] Thu, 13 Jul 2023 20:02:50 UTC (7,592 KB)
[v2] Tue, 20 Feb 2024 06:12:39 UTC (7,588 KB)