Caroline Lyon - Academia.edu (original) (raw)
Papers by Caroline Lyon
Topics in Cognitive Science, 2014
This article presents results from a multidisciplinary research project on the integration and tr... more This article presents results from a multidisciplinary research project on the integration and transfer of language knowledge into robots as an empirical paradigm for the study of language development in both humans and humanoid robots. Within the framework of human linguistic and cognitive development, we focus on how three central types of learning interact and co-develop: individual learning about one's own embodiment and the environment, social learning (learning from others), and learning of linguistic capability. Our primary concern is how these capabilities can scaffold each other's development in a continuous feedback cycle as their interactions yield increasingly sophisticated competencies in the…
The pattern matching capabilities of neural networks can be used to locate syntactic constituents... more The pattern matching capabilities of neural networks can be used to locate syntactic constituents of natural language. This paper describes a fully automated hybrid system, using neural nets operating within a grammatic framework. It addresses the representation of language for connectionist processing, and describes methods of constraining the problem size. The function of the network is briefly explained, and results are given.
The pattern matching capabilities of neural networks can be mobilised for an automated, natural l... more The pattern matching capabilities of neural networks can be mobilised for an automated, natural language, partial parser. First, language complexity is addressed by decomposing the problem into more tractable subtasks. Second, a representation is devised that enables effective, single layer networks to be used to map a pre-defined grammatic framework onto actual sentences. This paper examines data representation, network architecture and learning algorithms appropriate for linguistic data with their characteristic distributions. Users can access a working prototype via telnet on which they can try their own text.
This paper reports experiments on a corpus of news articles from the Financial Times, comparing d... more This paper reports experiments on a corpus of news articles from the Financial Times, comparing different text similarity models. First the Ferret system using a method based solely on lexical similarities is used, then methods based on semantic similarities are investigated. Different feature string selection criteria are used, for instance with and without synonyms obtained from WordNet, or with noun phrases extracted for comparison. The results indicate that synonyms rather than lexical strings are important for finding similar texts. Hypernyms and noun phrases also contribute to the identification of text similarity, though they are not better than synonyms. However, precision is a problem for the semantic similarity methods because too many irrelevant texts are retrieved.
![Research paper thumbnail of Using single layer networks for discrete, sequential data: an example from natural language processing. [extended version]](https://attachments.academia-assets.com/72298341/thumbnails/1.jpg)
](Natural Language Processing (NLP) is concerned with processing ordinary, unrestricted text. This ... more Natural Language Processing (NLP) is concerned with processing ordinary, unrestricted text. This work takes a new approach to a traditional NLP task, using neural computing methods. A parser which has been successfully implemented is described. It is a hybrid system, in which neural processors operate within a rule based framework. The neural processing components belong to the class of Generalized Single Layer Networks (GSLN). In general, supervised, feed-forward networks need more than one layer to process data. However, in some cases data can be pre-processed with a nonlinear transformation, and then presented in a linearly separable form for subsequent processing by a single layer net. Such networks offer advantages of functional transparency and operational speed. For our parser, the initial stage of processing maps linguistic data onto a higher order representation, which can then be analysed by a single layer network. This transformation is supported by information theoretic analysis. Three different algorithms for the neural component were investigated. Single layer nets can be trained by finding weight adjustments based on (a) factors proportional to the input, as in the Perceptron, (b)factors proportional to the existing weights, and (c) an error minimization method. In our experiments generalization ability varies little; method (b) is used for a prototype parser. This is available via telnet.
The theoretical background to the automated detection of plagiarism and collusion is investigated... more The theoretical background to the automated detection of plagiarism and collusion is investigated in this paper. We examine the underlying concepts, and see how features of language can be exploited to produce an effective system. Independently written texts have markedly different characteristics to those that include passages that have been fully or partially copied, and they can be effectively identified. The paper describes the implementation of the Ferret plagiarism and collusion detector, and its use in the University of Hertfordshire and other institutions. The difference between human and machine analysis is examined, and we conclude that an approach using machine processing is likely to be necessary in many situations.
Using the word count facility in Microsoft Word, or the Unix command wc, any string of characters... more Using the word count facility in Microsoft Word, or the Unix command wc, any string of characters separated by white space, is considered a word. In Ferret there must be a letter in the string for it to count as a word. Numerals and standalone punctuation marks are ignored. Thus the word count of a file in Ferret may not exactly match the Word or Unix measure.
Lecture Notes in Computer Science, 2007
The followings are our motivation. Rules of natural languages such as usage, grammar, and vocabul... more The followings are our motivation. Rules of natural languages such as usage, grammar, and vocabulary change diachronically dependent upon the social situations of the language community. This workshop focused on those language phenomena concerning language changes and evolution, that is, emergence, pidginization, and creolization, from the viewpoints of social, evolutionary, computational linguistics. Thus, we expected that the workshop would contribute to the joint discussion among those who share this common interest. Relevant themes were stated as Language change, Language emergence, Language acquisition, Second language acquisition, Multi-agent model of communication, Lingua Franca, Pidgin and creole, and other computer simulation concerning language dynamics. Thus far, the similar topics have been included in EVOLANG (International Conference on the Evolution of Language), though we intended to concentrate more on linguistic aspects of human and/or artificial agents' communication. At the workshop, J. R. Hurford of University of Edinburgh, UK, was invited to give a talk. Besides, fourteen technical papers were accepted to present, the topics of which were diverse including many branches of linguistic communication. In this volume, we will show the selected papers among them.
This paper describes a HRI case study which demonstrates how a humanoid robot can use simple heur... more This paper describes a HRI case study which demonstrates how a humanoid robot can use simple heuristics to acquire and use vocabulary in the context of being shown a series of shapes presented to it by a human and how the interaction style of the human changes as the robot learns and expresses its learning through speech. The case study is based on findings on how adults use childdirected speech when socially interacting with infants. The results indicate that humans are generally willing to engage with a robot in a similar manner to their engagement with a human infant and use similar styles of interaction varying as the shared understanding between them becomes more apparent. The case study also demonstrates that a rudimentary form of shared intentional reference can sufficiently bias the learning procedure. As a result, the robot associates humantaught lexical items for a series of presented shapes with its own sensorimotor experience, and is able to utter these words, acquired from the particular tutor, appropriately in an interactive, embodied context exhibiting apparent reference and discrimination.
Characteristics of natural language can be illuminated through the application of well known tool... more Characteristics of natural language can be illuminated through the application of well known tools in Information Theory. This paper shows how some of these characteristics can be exploited in the development of automated speech and language processing applications. The explicit representation of discontinuities in a temporal sequence of sounds, such as pauses in speech, can be utilized to improve the transmission of information. Arguments based on comparative entropy measures are used. The approach taken is rst to examine certain observed phenomena in speech, and suggest how their exploitation could have conferred an advantage as human language evolved. Similar arguments can then be applied to the development of more e cient automated processors. One consequence of this analysis is to show that there seems a certain inevitability to the evolution of structured language. These theories have a practical application in a current project in broadcasting technology: this is the semi-automated production of real time subtitles for live TV programmes.
This paper presents a statistical method for fingerprinting text. In a large collection of indepe... more This paper presents a statistical method for fingerprinting text. In a large collection of independently written documents each text is associated with a fingerprint which should be different from all the others. If fingerprints are too close then it is suspected that passages of copied or similar text occur in two documents. Our method exploits the characteristic distribution of word trigrams, and measures to determine similarity are based on set theoretic principles. The system was developed using a corpus of broadcast news reports and has been successfully used to detect plagiarism in students' work. It can find small sections that are similar as well as those that are identical. The method is very simple and effective, but seems not to have been used before
The complexity of parsing English sentences can be reduced by decomposing the problem into three ... more The complexity of parsing English sentences can be reduced by decomposing the problem into three subtasks. Declarative sentences can almost always be segmented into three concatenated sections: pre-subject-subject-predicate. Other constituents, such as clauses, phrases, noun groups, are contained within these segments, but do not normally cross the boundaries between them. Though a constituent in one section may have dependent links to elements in other sections, such as agreement between the head of the subject and the main verb, once the three sections have been located, they can then be partially processed separately, in parallel. An information theoretic analysis is used to support this approach. If sentences are represented as sequences of part-of-speech tags, then modelling them with the tripartite segmentation reduces the entropy. This indicates that some of the structure of the sentence has been captured. The tripartite segmentation can be produced automatically, using the ALPINE parser, which is then described. This is a hybrid processor in which neural networks operate within a rule based framework. It has been developed using corpora from technical manuals. Performance on unseen data from the manuals on which the processor was trained are over 90%. On data from other technical manuals performance is over 85%.
This thesis investigates how the pattern matching capabilities of neural networks can be used to ... more This thesis investigates how the pattern matching capabilities of neural networks can be used to detect dominant syntactic features in natural language. The critical issue addressed is reconciling a natural language representation with the capabilities of the network. In order to demonstrate that this is a valid approach a particular task is undertaken. This is to decompose sentences into their constituent parts of subject and predicate, then to nd the head of the subject, in a fully automated process. The language used comes from three domains: children's reading primers, a computer science text book and technical manuals. Unrestricted text is accepted. A data driven approach is supported by a skeletal grammatic framework. This produces a hybrid system in which rule based elements are integrated with connectionist methods for learning grammar from examples. To learn e ectively from examples negative as well as positive information should be used, and neural networks can model this. Feed forward networks with supervised learning are used, and di erent single layer and multi-layer networks are investigated. Single layer, higher order networks give the best results, and the reasons for this are explored. Concepts underlying the partitioning of the vocabulary into part-of-speech tag classes have been examined, and customised tagsets for speci c processing stages developed. We examine the connected issue of decomposing the parsing task into manageable sub-tasks. The importance of utilising both positive and negative information has been addressed, and methods of generating and modelling examples developed. The generation process leads to a problem of computational tractability, which is e ectively resolved using prohibitive rules as local constraints. I am very grateful to Peter Pym, Technical Publications Manager of Perkins Engines Ltd., for the use of text from their manuals as raw material for processing. I thank James Monaghan for opening up linguistic vistas. I also express my gratitude to Laurence Dixon for his support, and for some illuminating insights. My thanks also go to John Stobo for his helpful and perceptive comments. I am grateful to Rod Adams for his kind support. Finally, I thank Mary Buchanan and Audrey Mayes for help and support of every kind, and I particularly thank Mary for reading and discussing this thesis.
Topics in Cognitive Science, 2014
This article presents results from a multidisciplinary research project on the integration and tr... more This article presents results from a multidisciplinary research project on the integration and transfer of language knowledge into robots as an empirical paradigm for the study of language development in both humans and humanoid robots. Within the framework of human linguistic and cognitive development, we focus on how three central types of learning interact and co-develop: individual learning about one's own embodiment and the environment, social learning (learning from others), and learning of linguistic capability. Our primary concern is how these capabilities can scaffold each other's development in a continuous feedback cycle as their interactions yield increasingly sophisticated competencies in the…
The pattern matching capabilities of neural networks can be used to locate syntactic constituents... more The pattern matching capabilities of neural networks can be used to locate syntactic constituents of natural language. This paper describes a fully automated hybrid system, using neural nets operating within a grammatic framework. It addresses the representation of language for connectionist processing, and describes methods of constraining the problem size. The function of the network is briefly explained, and results are given.
The pattern matching capabilities of neural networks can be mobilised for an automated, natural l... more The pattern matching capabilities of neural networks can be mobilised for an automated, natural language, partial parser. First, language complexity is addressed by decomposing the problem into more tractable subtasks. Second, a representation is devised that enables effective, single layer networks to be used to map a pre-defined grammatic framework onto actual sentences. This paper examines data representation, network architecture and learning algorithms appropriate for linguistic data with their characteristic distributions. Users can access a working prototype via telnet on which they can try their own text.
This paper reports experiments on a corpus of news articles from the Financial Times, comparing d... more This paper reports experiments on a corpus of news articles from the Financial Times, comparing different text similarity models. First the Ferret system using a method based solely on lexical similarities is used, then methods based on semantic similarities are investigated. Different feature string selection criteria are used, for instance with and without synonyms obtained from WordNet, or with noun phrases extracted for comparison. The results indicate that synonyms rather than lexical strings are important for finding similar texts. Hypernyms and noun phrases also contribute to the identification of text similarity, though they are not better than synonyms. However, precision is a problem for the semantic similarity methods because too many irrelevant texts are retrieved.
Natural Language Processing (NLP) is concerned with processing ordinary, unrestricted text. This ... more Natural Language Processing (NLP) is concerned with processing ordinary, unrestricted text. This work takes a new approach to a traditional NLP task, using neural computing methods. A parser which has been successfully implemented is described. It is a hybrid system, in which neural processors operate within a rule based framework. The neural processing components belong to the class of Generalized Single Layer Networks (GSLN). In general, supervised, feed-forward networks need more than one layer to process data. However, in some cases data can be pre-processed with a nonlinear transformation, and then presented in a linearly separable form for subsequent processing by a single layer net. Such networks offer advantages of functional transparency and operational speed. For our parser, the initial stage of processing maps linguistic data onto a higher order representation, which can then be analysed by a single layer network. This transformation is supported by information theoretic analysis. Three different algorithms for the neural component were investigated. Single layer nets can be trained by finding weight adjustments based on (a) factors proportional to the input, as in the Perceptron, (b)factors proportional to the existing weights, and (c) an error minimization method. In our experiments generalization ability varies little; method (b) is used for a prototype parser. This is available via telnet.
The theoretical background to the automated detection of plagiarism and collusion is investigated... more The theoretical background to the automated detection of plagiarism and collusion is investigated in this paper. We examine the underlying concepts, and see how features of language can be exploited to produce an effective system. Independently written texts have markedly different characteristics to those that include passages that have been fully or partially copied, and they can be effectively identified. The paper describes the implementation of the Ferret plagiarism and collusion detector, and its use in the University of Hertfordshire and other institutions. The difference between human and machine analysis is examined, and we conclude that an approach using machine processing is likely to be necessary in many situations.
Using the word count facility in Microsoft Word, or the Unix command wc, any string of characters... more Using the word count facility in Microsoft Word, or the Unix command wc, any string of characters separated by white space, is considered a word. In Ferret there must be a letter in the string for it to count as a word. Numerals and standalone punctuation marks are ignored. Thus the word count of a file in Ferret may not exactly match the Word or Unix measure.
Lecture Notes in Computer Science, 2007
The followings are our motivation. Rules of natural languages such as usage, grammar, and vocabul... more The followings are our motivation. Rules of natural languages such as usage, grammar, and vocabulary change diachronically dependent upon the social situations of the language community. This workshop focused on those language phenomena concerning language changes and evolution, that is, emergence, pidginization, and creolization, from the viewpoints of social, evolutionary, computational linguistics. Thus, we expected that the workshop would contribute to the joint discussion among those who share this common interest. Relevant themes were stated as Language change, Language emergence, Language acquisition, Second language acquisition, Multi-agent model of communication, Lingua Franca, Pidgin and creole, and other computer simulation concerning language dynamics. Thus far, the similar topics have been included in EVOLANG (International Conference on the Evolution of Language), though we intended to concentrate more on linguistic aspects of human and/or artificial agents' communication. At the workshop, J. R. Hurford of University of Edinburgh, UK, was invited to give a talk. Besides, fourteen technical papers were accepted to present, the topics of which were diverse including many branches of linguistic communication. In this volume, we will show the selected papers among them.
This paper describes a HRI case study which demonstrates how a humanoid robot can use simple heur... more This paper describes a HRI case study which demonstrates how a humanoid robot can use simple heuristics to acquire and use vocabulary in the context of being shown a series of shapes presented to it by a human and how the interaction style of the human changes as the robot learns and expresses its learning through speech. The case study is based on findings on how adults use childdirected speech when socially interacting with infants. The results indicate that humans are generally willing to engage with a robot in a similar manner to their engagement with a human infant and use similar styles of interaction varying as the shared understanding between them becomes more apparent. The case study also demonstrates that a rudimentary form of shared intentional reference can sufficiently bias the learning procedure. As a result, the robot associates humantaught lexical items for a series of presented shapes with its own sensorimotor experience, and is able to utter these words, acquired from the particular tutor, appropriately in an interactive, embodied context exhibiting apparent reference and discrimination.
Characteristics of natural language can be illuminated through the application of well known tool... more Characteristics of natural language can be illuminated through the application of well known tools in Information Theory. This paper shows how some of these characteristics can be exploited in the development of automated speech and language processing applications. The explicit representation of discontinuities in a temporal sequence of sounds, such as pauses in speech, can be utilized to improve the transmission of information. Arguments based on comparative entropy measures are used. The approach taken is rst to examine certain observed phenomena in speech, and suggest how their exploitation could have conferred an advantage as human language evolved. Similar arguments can then be applied to the development of more e cient automated processors. One consequence of this analysis is to show that there seems a certain inevitability to the evolution of structured language. These theories have a practical application in a current project in broadcasting technology: this is the semi-automated production of real time subtitles for live TV programmes.
This paper presents a statistical method for fingerprinting text. In a large collection of indepe... more This paper presents a statistical method for fingerprinting text. In a large collection of independently written documents each text is associated with a fingerprint which should be different from all the others. If fingerprints are too close then it is suspected that passages of copied or similar text occur in two documents. Our method exploits the characteristic distribution of word trigrams, and measures to determine similarity are based on set theoretic principles. The system was developed using a corpus of broadcast news reports and has been successfully used to detect plagiarism in students' work. It can find small sections that are similar as well as those that are identical. The method is very simple and effective, but seems not to have been used before
The complexity of parsing English sentences can be reduced by decomposing the problem into three ... more The complexity of parsing English sentences can be reduced by decomposing the problem into three subtasks. Declarative sentences can almost always be segmented into three concatenated sections: pre-subject-subject-predicate. Other constituents, such as clauses, phrases, noun groups, are contained within these segments, but do not normally cross the boundaries between them. Though a constituent in one section may have dependent links to elements in other sections, such as agreement between the head of the subject and the main verb, once the three sections have been located, they can then be partially processed separately, in parallel. An information theoretic analysis is used to support this approach. If sentences are represented as sequences of part-of-speech tags, then modelling them with the tripartite segmentation reduces the entropy. This indicates that some of the structure of the sentence has been captured. The tripartite segmentation can be produced automatically, using the ALPINE parser, which is then described. This is a hybrid processor in which neural networks operate within a rule based framework. It has been developed using corpora from technical manuals. Performance on unseen data from the manuals on which the processor was trained are over 90%. On data from other technical manuals performance is over 85%.
This thesis investigates how the pattern matching capabilities of neural networks can be used to ... more This thesis investigates how the pattern matching capabilities of neural networks can be used to detect dominant syntactic features in natural language. The critical issue addressed is reconciling a natural language representation with the capabilities of the network. In order to demonstrate that this is a valid approach a particular task is undertaken. This is to decompose sentences into their constituent parts of subject and predicate, then to nd the head of the subject, in a fully automated process. The language used comes from three domains: children's reading primers, a computer science text book and technical manuals. Unrestricted text is accepted. A data driven approach is supported by a skeletal grammatic framework. This produces a hybrid system in which rule based elements are integrated with connectionist methods for learning grammar from examples. To learn e ectively from examples negative as well as positive information should be used, and neural networks can model this. Feed forward networks with supervised learning are used, and di erent single layer and multi-layer networks are investigated. Single layer, higher order networks give the best results, and the reasons for this are explored. Concepts underlying the partitioning of the vocabulary into part-of-speech tag classes have been examined, and customised tagsets for speci c processing stages developed. We examine the connected issue of decomposing the parsing task into manageable sub-tasks. The importance of utilising both positive and negative information has been addressed, and methods of generating and modelling examples developed. The generation process leads to a problem of computational tractability, which is e ectively resolved using prohibitive rules as local constraints. I am very grateful to Peter Pym, Technical Publications Manager of Perkins Engines Ltd., for the use of text from their manuals as raw material for processing. I thank James Monaghan for opening up linguistic vistas. I also express my gratitude to Laurence Dixon for his support, and for some illuminating insights. My thanks also go to John Stobo for his helpful and perceptive comments. I am grateful to Rod Adams for his kind support. Finally, I thank Mary Buchanan and Audrey Mayes for help and support of every kind, and I particularly thank Mary for reading and discussing this thesis.