Using NLG to help language-impaired users tell stories and participate in social dialogues (original) (raw)
Related papers
Providing intelligent language feedback for augmentative communication users
Lecture Notes in Computer Science
People with severe speech and motor impairments SSMI can often use augmentative communication devices to help them communicate. While these devices can provide speech synthesis or text output, the rate of communication is typically very slow. Consequently, augmentative communication users often develop telegraphic patterns of language usage. A natural language processing technique termed compansion compression-expansion has been developed that expands unin ected content w ords i.e., compressed or telegraphic utterances into syntactically and semantically well-formed sentences. While originally designed as a rate enhancement technique, compansion may also be viewed as a potential tool to support English literacy for augmentative communication users. Accurate grammatical feedback from ill-formed inputs might b e v ery bene cial in the learning process. However, the problems of dealing with inherently ambiguous errors and multiple corrections are not trivial. This paper proposes the addition of an adaptive user language model as a way to address some of these diculties. It also discusses a possible implementation strategy using grammatical mal-rules for a prototype application that uses the compansion technique.
ArXiv, 2021
Augmentative and alternative communication (AAC) is a practical means of communication for people with language disabilities. In this study, we propose PICTALKY, which is an AI-based AAC system that helps children with language developmental disabilities to improve their communication skills and language comprehension abilities. PICTALKY can process both text and pictograms more accurately by connecting a series of neural-based NLP modules. Moreover, we perform quantitative and qualitative analyses on the essential features of PICTALKY. It is expected that those suffering from language problems will be able to express their intentions or desires more easily and improve their quality of life by using this service. We have made the models freely available alongside a demonstration of the Web interface 1. Furthermore, we implemented robotics AAC for the first time by applying PICTALKY to the NAO robot.
Some interface issues in developing intelligent communication aids for people with disabilities
Proceedings of the 2nd international conference on Intelligent user interfaces - IUI '97, 1997
Augmentative and Alternative Communication (AAC) is the field of study concerned with providing devices and techniques to augment the communicative ability of a person whose disability makes it difficult to speak in an understandable fashion. For several years, we have been applying natural language processing techniques to the field of AAC in order to develop intelligent communication aids that attempt to provide linguistically "correct" output while speeding communication rate. In this paper we describe some of the interface issues that must be considered when developing such a device. We focus on a project aimed at a group of users who have cognitive impairments that affect their linguistic ability. A prototype system is under development which will hopefully not only prove to be an effective communication aid, but may provide some language intervention benefits for this population.
The Application of Natural Language Processing to Augmentative and Alternative Communication
Assistive Technology, 2012
Significant progress has been made in the application of natural language processing (NLP) to augmentative and alternative communication (AAC), particularly in the areas of interface design and word prediction. This article will survey the current state-of-the-science of NLP in AAC and discuss its future applications for the development of next generation of AAC technology.
Using NLG and sensors to support personal narrative for children with complex communication needs
2010
We are building a tool that helps children with Complex Communication Needs 1 (CCN) to create stories about their day at school. The tool uses Natural Language Generation (NLG) technology to create a draft story based on sensor data of the child's activities, which the child can edit. This work is still in its early stages, but we believe it has great potential to support interactive personal narrative which is not well supported by current Augmentative and Alternative Communication (AAC) tools. 1 The term Complex Communication Needs (CCN) describes individuals who, due to motor, language, cognitive, and/or sensory perceptual impairments (e.g., as a result of cerebral palsy), do not develop speech and language skills as expected. This heterogeneous group typically experiences restricted access to the environment, limited interactions with their communication partners, and few opportunities for communication (Light and Drager 2007).
A flexible approach to natural language generation for disabled children
Proceedings of the 21st International Conference on computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop on - COLING ACL '06, 2006
Natural Language Generation (NLG) is a way to automatically realize a correct expression in response to a communicative goal. This technology is mainly explored in the fields of machine translation, report generation, dialog system etc. In this paper we have explored the NLG technique for another novel applicationassisting disabled children to take part in conversation. The limited physical ability and mental maturity of our intended users made the NLG approach different from others. We have taken a flexible approach where main emphasis is given on flexibility and usability of the system. The evaluation results show this technique can increase the communication rate of users during a conversation.