Optimizing Differentiable Relaxations of Coreference Evaluation Metrics (original) (raw)
Related papers
Deep Reinforcement Learning for Mention-Ranking Coreference Models
Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2016
Coreference resolution systems are typically trained with heuristic loss functions that require careful tuning. In this paper we instead apply reinforcement learning to directly optimize a neural mention-ranking model for coreference evaluation metrics. We experiment with two approaches: the REINFORCE policy gradient algorithm and a rewardrescaled max-margin objective. We find the latter to be more effective, resulting in significant improvements over the current state-ofthe-art on the English and Chinese portions of the CoNLL 2012 Shared Task.
Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2017
An assential aspect of structured prediction is the evaluation of an output structure against the gold standard. Especially in the loss-augmented setting, the need of finding the max-violating constraint has severely limited the expressivity of effective loss functions. In this paper, we trade off exact computation for enabling the use of more complex loss functions for coreference resolution (CR). Most noteworthily, we show that such functions can be (i) automatically learned also from controversial but commonly accepted CR measures, e.g., MELA, and (ii) successfully used in learning algorithms. The accurate model comparison on the standard CoNLL-2012 setting shows the benefit of more expressive loss for Arabic and English data.
Incorporating contextual cues in trainable models for coreference resolution
We propose a method that incorporates various novel contextual cues into a ma-chine learning for resolving coreference. Distinct characteristics of our model are (i) incorporating more linguistic fea-tures capturing contextual information that is more sophisticated than what is offered in Centering Theory, and (ii) a tournament model for selecting a ref-erent. Our experiments show that this model significantly outperforms earlier machine learning approaches, such as Soon et al. (2001).
Maximum Metric Score Training for Coreference Resolution
A large body of prior research on coreference resolution recasts the problem as a two-class classification problem. However, standard supervised machine learning algorithms that minimize classification errors on the training instances do not always lead to maximizing the F-measure of the chosen evaluation metric for coreference resolution. In this paper, we propose a novel approach comprising the use of instance weighting and beam search to maximize the evaluation metric score on the training corpus during training. Experimental results show that this approach achieves significant improvement over the state-of-the-art. We report results on standard benchmark corpora (two MUC corpora and three ACE corpora), when evaluated using the link-based MUC metric and the mention-based B-CUBED metric.
Towards Harnessing Memory Networks for Coreference Resolution
Proceedings of the 2nd Workshop on Representation Learning for NLP, 2017
Coreference resolution task demands comprehending a discourse, especially for anaphoric mentions which require semantic information for resolving antecedents. We investigate into how memory networks can be helpful for coreference resolution when posed as question answering problem. The comprehension capability of memory networks assists coreference resolution, particularly for the mentions those require semantic and context information. We experiment memory networks for coreference resolution, with 4 synthetic datasets generated for coreference resolution with varying difficulty levels. Our system's performance is compared with a traditional coreference resolution system to show why memory networks can be promising for coreference resolution.
Coreference Resolution Using Competition Learning Approach
2003
In this paper we propose a competition learning approach to coreference resolution. Traditionally, supervised machine learning approaches adopt the singlecandidate model. Nevertheless the preference relationship between the antecedent candidates cannot be determined accurately in this model. By contrast, our approach adopts a twin-candidate learning model. Such a model can present the competition criterion for antecedent candidates reliably, and ensure that the most preferred candidate is selected. Furthermore, our approach applies a candidate filter to reduce the computational cost and data noises during training and resolution. The experimental results on MUC-6 and MUC-7 data set show that our approach can outperform those based on the singlecandidate model.
Specialized models and ranking for coreference resolution
Proceedings of the Conference on Empirical …, 2008
This paper investigates two strategies for improving coreference resolution: (1) training separate models that specialize in particular types of mentions (e.g., pronouns versus proper nouns) and (2) using a ranking loss function rather than a classification function. In addition to being conceptually simple, these modifications of the standard single-model, classification-based approach also deliver significant performance improvements. Specifically, we show that on the ACE corpus both strategies produce f -score gains of more than 3% across the three coreference evaluation metrics (MUC, B 3 , and CEAF).
Entity-Centric Coreference Resolution with Model Stacking
Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2015
Mention pair models that predict whether or not two mentions are coreferent have historically been very effective for coreference resolution, but do not make use of entity-level information. However, we show that the scores produced by such models can be aggregated to define powerful entity-level features between clusters of mentions. Using these features, we train an entity-centric coreference system that learns an effective policy for building up coreference chains incrementally. The mention pair scores are also used to prune the search space the system works in, allowing for efficient training with an exact loss function. We evaluate our system on the English portion of the 2012 CoNLL Shared Task dataset and show that it improves over the current state of the art.