Generalization of Semantic Roles in Automatic Semantic Role Labeling (original) (raw)

A comparative study on generalization of semantic roles in FrameNet

Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1 - ACL-IJCNLP '09, 2009

A number of studies have presented machine-learning approaches to semantic role labeling with availability of corpora such as FrameNet and PropBank. These corpora define the semantic roles of predicates for each frame independently. Thus, it is crucial for the machine-learning approach to generalize semantic roles across different frames, and to increase the size of training instances. This paper explores several criteria for generalizing semantic roles in FrameNet: role hierarchy, human-understandable descriptors of roles, semantic types of filler phrases, and mappings from FrameNet roles to thematic roles of VerbNet. We also propose feature functions that naturally combine and weight these criteria, based on the training data. The experimental result of the role classification shows 19.16% and 7.42% improvements in error reduction rate and macro-averaged F1 score, respectively. We also provide in-depth analyses of the proposed criteria.

Semantic role labeling via FrameNet, VerbNet and PropBank

Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the ACL - ACL '06, 2006

This article describes a robust semantic parser that uses a broad knowledge base created by interconnecting three major resources: FrameNet, VerbNet and PropBank. The FrameNet corpus contains the examples annotated with semantic roles whereas the VerbNet lexicon provides the knowledge about the syntactic behavior of the verbs. We connect VerbNet and FrameNet by mapping the FrameNet frames to the VerbNet Intersective Levin classes. The PropBank corpus, which is tightly connected to the VerbNet lexicon, is used to increase the verb coverage and also to test the effectiveness of our approach. The results indicate that our model is an interesting step towards the design of more robust semantic parsers.

Calibrating features for semantic role labeling

2004

This paper takes a critical look at the features used in the semantic role tagging literature and show that the information in the input, generally a syntactic parse tree, has yet to be fully exploited. We propose an additional set of features and our experiments show that these features lead to fairly significant improvements in the tasks we performed. We further show that different features are needed for different subtasks. Finally, we show that by using a Maximum Entropy classifier and fewer features, we achieved results comparable with the best previously reported results obtained with SVM models. We believe this is a clear indication that developing features that capture the right kind of information is crucial to advancing the stateof-the-art in semantic analysis.

Introduction to the CoNLL-2004 shared task: Semantic role labeling

Proceedings of CoNLL, 2004

In this paper we describe the CoNLL-2004 shared task: semantic role labeling. We introduce the specification and goal of the task, describe the data sets and evaluation methods, and present a general overview of the systems that have contributed to the task, providing comparative description.

Introduction to the CoNLL-2005 shared task: semantic role labeling

Proceedings of the Ninth Conference on Computational Natural Language Learning, 2005

In this paper we describe the CoNLL-2005 shared task on Semantic Role Labeling. We introduce the specification and goals of the task, describe the data sets and evaluation methods, and present a general overview of the 19 systems that have contributed to the task, providing a comparative description and results.

A FrameNet-based semantic role labeler for Swedish

Proceedings of the COLING/ACL on Main conference poster sessions -, 2006

We present a FrameNet-based semantic role labeling system for Swedish text. As training data for the system, we used an annotated corpus that we produced by transferring FrameNet annotation from the English side to the Swedish side in a parallel corpus. In addition, we describe two frame element bracketing algorithms that are suitable when no robust constituent parsers are available. We evaluated the system on a part of the FrameNet example corpus that we translated manually, and obtained an accuracy score of 0.75 on the classification of presegmented frame elements, and precision and recall scores of 0.67 and 0.47 for the complete task.

Structured Learning for Semantic Role Labeling

Lecture Notes in Computer Science, 2011

The use of complex grammatical features in statistical language learning assumes the availability of large scale training data and good quality parsers, especially for languages different from English. In this paper, we show how good quality FrameNet Semantic Role Labeling systems can be obtained without relying on full syntactic parsing, by backing off to surface grammatical representations and structured learning. In line with this approach, the ioB Annotation Based Engine for srL (BABEL) has been implemented as a flexible system for Semantic Role Labeling based on a Structured Support Vector Machine learning framework. While the underlying learning paradigm allows employing BABEL when no syntactic parser is available, its accuracy is in line with state-of-the-art systems for English. BABEL is among the best performing Semantic Role Labeling systems also for Italian, as recently evaluated in the role labeling task of the Frame Labeling over Italian Texts at the Evalita 2011 competition. Moreover, the same learning framework is applied to effectively acquire surface grammatical information, achieving state-of-the-art results also with respect to the Part-of-speech tagging task of the Evalita 2009 competition. Finally, BABEL can POS tag more than 1,500 word per second while the SRL module can process about 35 sentences per second, thus making its use straightforward in Web scale applications.

The effect of syntactic representation on semantic role labeling

Proceedings of the 22nd International Conference on Computational Linguistics - COLING '08, 2008

Almost all automatic semantic role labeling (SRL) systems rely on a preliminary parsing step that derives a syntactic structure from the sentence being analyzed. This makes the choice of syntactic representation an essential design decision. In this paper, we study the influence of syntactic representation on the performance of SRL systems. Specifically, we compare constituent-based and dependencybased representations for SRL of English in the FrameNet paradigm. Contrary to previous claims, our results demonstrate that the systems based on dependencies perform roughly as well as those based on constituents: For the argument classification task, dependencybased systems perform slightly higher on average, while the opposite holds for the argument identification task. This is remarkable because dependency parsers are still in their infancy while constituent parsing is more mature. Furthermore, the results show that dependency-based semantic role classifiers rely less on lexicalized features, which makes them more robust to domain changes and makes them learn more efficiently with respect to the amount of training data.

DeSRL: A Linear-Time Semantic Role Labeling System

Proceedings of the Twelfth Conference on Computational Natural Language Learning, 2008

This paper describes the DeSRL system, a joined effort of Yahoo! Research Barcelona and Università di Pisa for the CoNLL-2008 Shared Task (Surdeanu et al., 2008). The system is characterized by an efficient pipeline of linear complexity components, each carrying out a different sub-task. Classifier errors and ambiguities are addressed with several strategies: revision models, voting, and reranking. The system participated in the closed challenge ranking third in the complete problem evaluation with the following scores: 82.06 labeled macro F1 for the overall task, 86.6 labeled attachment for syntactic dependencies, and 77.5 labeled F1 for semantic dependencies.

Semantic Role Labeling: An Introduction to the Special Issue

Computational Linguistics, 2008

Semantic role labeling, the computational identification and labeling of arguments in text, has become a leading task in computational linguistics today. Although the issues for this task have been studied for decades, the availability of large resources and the development of statistical machine learning methods have heightened the amount of effort in this field. This special issue presents selected and representative work in the field. This overview describes linguistic background of the problem, the movement from linguistic theories to computational practice, the major resources that are being used, an overview of steps taken in computational systems, and a description of the key issues and results in semantic role labeling (as revealed in several international evaluations). We assess weaknesses in semantic role labeling and identify important challenges facing the field. Overall, the opportunities and the potential for useful further research in semantic role labeling are considerable.