A combined memory-based semantic role labeler of English (original) (raw)

Dependency parsing and semantic role labeling as a single task

Ethical Theory and Moral Practice, 2009

We present a comparison between two systems for establishing syntactic and semantic dependencies: one that performs dependency parsing and semantic role labeling as a single task, and another that performs the two tasks in isolation. The systems are based on local memorybased classifiers predicting syntactic and semantic dependency relations between pairs of words. In a second global phase, the systems perform a deterministic ranking procedure in which the output of the local classifiers is combined per sentence into a dependency graph and semantic role labeling assignments for all predicates. The comparison shows that in the learning phase a joint approach produces better-scoring classifiers, while after the ranking phase the isolated approach produces the most accurate syntactic dependencies, while the joint approach yields the most accurate semantic role assignments.

Memory-based semantic role labeling: Optimizing features, algorithm, and output

Proceedings of the CoNLL-2004, 2004

In this paper we interpret the semantic role labeling problem as a classification task, and apply memory-based learning to it in an approach similar to Buchholz et al.(1999) and Buchholz (2002) for grammatical relation labeling. We apply feature selection and algorithm parameter optimization strategies to our learner. In addition, we investigate the effect of two innovations:(i) the use of sequences of classes as classification output, combined with a simple voting mechanism, and (ii) the use of iterative classifier stacking which takes as ...

The necessity of syntactic parsing for semantic role labeling

International Joint Conference on …, 2005

We provide an experimental study of the role of syntactic parsing in semantic role labeling. Our conclusions demonstrate that syntactic parse information is clearly most relevant in the very first stage -the pruning stage. In addition, the quality of the pruning stage cannot be determined solely based on its recall and precision. Instead it depends on the characteristics of the output candidates that make downstream problems easier or harder. Motivated by this observation, we suggest an effective and simple approach of combining different semantic role labeling systems through joint inference, which significantly improves the performance.

The effect of syntactic representation on semantic role labeling

Proceedings of the 22nd International Conference on Computational Linguistics - COLING '08, 2008

Almost all automatic semantic role labeling (SRL) systems rely on a preliminary parsing step that derives a syntactic structure from the sentence being analyzed. This makes the choice of syntactic representation an essential design decision. In this paper, we study the influence of syntactic representation on the performance of SRL systems. Specifically, we compare constituent-based and dependencybased representations for SRL of English in the FrameNet paradigm. Contrary to previous claims, our results demonstrate that the systems based on dependencies perform roughly as well as those based on constituents: For the argument classification task, dependencybased systems perform slightly higher on average, while the opposite holds for the argument identification task. This is remarkable because dependency parsers are still in their infancy while constituent parsing is more mature. Furthermore, the results show that dependency-based semantic role classifiers rely less on lexicalized features, which makes them more robust to domain changes and makes them learn more efficiently with respect to the amount of training data.

A simple generative pipeline approach to dependency parsing and semantic role labeling

Proceedings of the Thirteenth Conference on Computational Natural Language Learning Shared Task - CoNLL '09, 2009

We describe our CoNLL 2009 Shared Task system in the present paper. The system includes three cascaded components: a generative dependency parser, a classifier for syntactic dependency labels and a semantic classifier. The experimental results show that the labeled macro F1 scores of our system on the joint task range from 43.50% (Chinese) to 57.95% (Czech), with an average of 51.07%.

DeSRL: A Linear-Time Semantic Role Labeling System

Proceedings of the Twelfth Conference on Computational Natural Language Learning, 2008

This paper describes the DeSRL system, a joined effort of Yahoo! Research Barcelona and Università di Pisa for the CoNLL-2008 Shared Task (Surdeanu et al., 2008). The system is characterized by an efficient pipeline of linear complexity components, each carrying out a different sub-task. Classifier errors and ambiguities are addressed with several strategies: revision models, voting, and reranking. The system participated in the closed challenge ranking third in the complete problem evaluation with the following scores: 82.06 labeled macro F1 for the overall task, 86.6 labeled attachment for syntactic dependencies, and 77.5 labeled F1 for semantic dependencies.

Semantic role labeling using different syntactic views

Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics - ACL '05, 2005

Semantic role labeling is the process of annotating the predicate-argument structure in text with semantic labels. In this paper we present a state-of-the-art baseline semantic role labeling system based on Support Vector Machine classifiers. We show improvements on this system by: i) adding new features including features extracted from dependency parses, ii) performing feature selection and calibration and iii) combining parses obtained from semantic parsers trained using different syntactic views. Error analysis of the baseline system showed that approximately half of the argument identification errors resulted from parse errors in which there was no syntactic constituent that aligned with the correct argument. In order to address this problem, we combined semantic parses from a Minipar syntactic parse and from a chunked syntactic representation with our original baseline system which was based on Charniak parses. All of the reported techniques resulted in performance improvements. *

Dependency-based semantic role labeling of PropBank

Proceedings of the Conference on Empirical Methods in Natural Language Processing - EMNLP '08, 2008

We present a PropBank semantic role labeling system for English that is integrated with a dependency parser. To tackle the problem of joint syntactic-semantic analysis, the system relies on a syntactic and a semantic subcomponent. The syntactic model is a projective parser using pseudo-projective transformations, and the semantic model uses global inference mechanisms on top of a pipeline of classifiers. The complete syntactic-semantic output is selected from a candidate pool generated by the subsystems. We evaluate the system on the CoNLL-2005 test sets using segment-based and dependency-based metrics. Using the segment-based CoNLL-2005 metric, our system achieves a near state-of-the-art F1 figure of 77.97 on the WSJ+Brown test set, or 78.84 if punctuation is treated consistently. Using a dependency-based metric, the F1 figure of our system is 84.29 on the test set from CoNLL-2008. Our system is the first dependency-based semantic role labeler for PropBank that rivals constituent-based systems in terms of performance.

Semantic Role Labeling by Tagging Syntactic Chunks

2004

In this paper, we present a semantic role la- beler (or chunker) that groups syntactic chunks (i.e. base phrases) into the arguments of a pred- icate. This is accomplished by casting the se- mantic labeling as the classification of syntactic chunks (e.g. NP-chunk, PP-chunk) into one of several classes such as the beginning of an ar- gument (B-ARG), inside an

Mixing and blending syntactic and semantic dependencies

Proceedings of the Twelfth Conference on Computational Natural Language Learning - CoNLL '08, 2008

Our system for the CoNLL 2008 shared task uses a set of individual parsers, a set of stand-alone semantic role labellers, and a joint system for parsing and semantic role labelling, all blended together. The system achieved a macro averaged labelled F 1score of 79.79 (WSJ 80.92, Brown 70.49) for the overall task. The labelled attachment score for syntactic dependencies was 86.63 (WSJ 87.36, Brown 80.77) and the labelled F 1-score for semantic dependencies was 72.94 (WSJ 74.47, Brown 60.18).