Use of Weighted Finite State Transducers in Part of Speech Tagging (original) (raw)

Part-of-speech tagging using parallel weighted finite-state transducers

Advances in Natural Language Processing, 2010

We use parallel weighted finite-state transducers to implement a part-of-speech tagger, which obtains state-of-the-art accuracy when used to tag the Europarl corpora for Finnish, Swedish and English. Our system consists of a weighted lexicon and a guesser combined with a bigram model factored into two weighted transducers. We use both lemmas and tag sequences in the bigram model, which guarantees reliable bigram estimates.

Deterministic Part-of-Speech Tagging with Finite-State Transducers

Computational Linguistics, 1995

Stochastic approaches to natural language processing have often been preferred to rule-based approaches because of their robustness and their automatic training capabilities. This was the case for part-of-speech tagging until Brill showed how state-of-the-art part-of-speech tagging can be achieved with a rule-based tagger by inferring rules from a training corpus. However, current implementations of the rule-based tagger run more slowly

Finite-State Transducers in Language and Speech Processing

Finite-state machines have been used in various domains of natural language processing. We consider here the use of a type of transducer that supports very efficient programs: sequential transducers. We recall classical theorems and give new ones characterizing sequential string-tostring transducers. Transducers that output weights also play an important role in language and speech processing. We give a specific study of string-to-weight transducers, including algorithms for determinizing and minimizing these transducers very efficiently, and characterizations of the transducers admitting determinization and the corresponding algorithms. Some applications of these algorithms in speech recognition are described and illustrated.

Weighted finite-state transducers in speech recognition

Computer Speech & Language, 2002

We survey the use of weighted finite-state transducers (WFSTs) in speech recognition. We show that WFSTs provide a common and natural representation for HMM models, context-dependency, pronunciation dictionaries, grammars, and alternative recognition outputs. Furthermore, general transducer operations combine these representations flexibly and efficiently. Weighted determinization and minimization algorithms optimize their time and space requirements, and a weight pushing algorithm distributes the weights along the paths of a weighted transducer optimally for speech recognition.

Combining Statistical Models for POS Tagging using Finite-State Calculus

2011

We introduce a framework for POS tagging which can incorporate a variety of different information sources such as statistical models and hand-written rules. The information sources are compiled into a set of weighted finite-state transducers and tagging is accomplished using weighted finite-state algorithms. Our aim is to develop a fast and flexible way for trying out different tagger designs and combining them into hybrid systems. We test the applicability of the framework by constructing HMM taggers with augmented lexical models for English and Finnish. We compare our taggers with two existing statistical taggers TnT and Hunpos and find that we achieve superior accuracy.

Language model combination and adaptation usingweighted finite state transducers

2010

In speech recognition systems language model (LMs) are often constructed by training and combining multiple n-gram models. They can be either used to represent different genres or tasks found in diverse text sources, or capture stochastic properties of different linguistic symbol sequences, for example, syllables and words. Unsupervised LM adaptation may also be used to further improve robustness to varying styles or tasks. When using these techniques, extensive software changes are often required. In this paper an alternative and more general approach based on weighted finite state transducers (WFSTs) is investigated for LM combination and adaptation. As it is entirely based on well-defined WFST operations, minimum change to decoding tools is needed. A wide range of LM combination configurations can be flexibly supported. An efficient on-the-fly WFST decoding algorithm is also proposed. Significant error rate gains of 7.3% relative were obtained on a state-of-the-art broadcast audio recognition task using a history dependently adapted multi-level LM modelling both syllable and word sequences.

Two parsing algorithms by means of finite state transducers

Proceedings of the 15th conference on Computational linguistics -, 1994

We present a new apl)roach , ilhlstrated by two algo-rithms> for parsing not only Finite SI.ate (:Iranlnlars but also Context Free Grainlnars and their extension, by means of finite state machines. '/'he basis is the computation of a flxed point of a linite-state function, i.e. a finite-state transducer. Using these techniques, we have built a program that parses French sentences with a gramnlar of more than 200>000 lexical rules with a typical response time of less than a second. The tirst algorithm computes a fixed point of a non-deterluinistic tinite-state transducer and the second coniplites a lixed point of a deterministic bidirectiollal device called a bimachine. These two algoril;hms point out a new connection between the theory of parsing and the theory of representation of rational transduetions.

A Layered, Transducer-Based Model for Speech and Language Processing

2008

This paper presents a transducer-based model for speech and language processing. The proposed model consists of a series of layers of interconnected transducers. At the lower layer there is a Finite-State Transducer (FST) containing the lexicon of the system. At the next layer another FST represents the language model. Then a word-to-POS transducer is used to provide a link between the graphemic form of the word and the part-of-speech tag associated with it. The upper layer is composed of a transducer, which utilises the POS information of the previous layer to form syntactic structures based on context-free grammatical rules. Transition probabilities are also considered thus forming Weighted Finite-State Transducers (WFSTs). Keeping these probabilities outside the grammatical information allows their independent composition. The grammatical component can be composed from existing carefully built lexicons, language models and syntactic rules, while the probabilities can be derived automatically from corpora afterwards and be applied as additional information to the existing structure. A set of on-line algorithms for rapid update of automata and transducers has been developed to support this approach.