Enhancing Unsupervised Semantic Parsing with Distributed Contextual Representations (original) (raw)
Related papers
Deep Contextual Disambiguation of Homonyms and Polysemants (preprint version)
Digital Scholarship in the Humanities, 2022
A new metric method for word sense disambiguation is put forward, being formulated within a phenomenological analogy to the wave function of an observable quantity in quantum mechanics where the actual meaning of a multivalued word (a homonym or a polysemant) is determined by its context. The choice of the actualized sense is determined by the minimal semantic distance between the semantics of possible word senses and that of the context, where the meanings of the word in question and the context follow from their semantic fields based on lexicographic hyperchains. In the contrast to the common models, our method accounts for the semantic polarity. The formulated method showed good results in disentangling polysemy, which was not possible to achieve within the BERT-based contextualized embedding approach.
Proceedings of the 46th Annual Meeting …, 2008
This paper introduces a machine learning method based on bayesian networks which is applied to the mapping between deep semantic representations and lexical semantic resources. A probabilistic model comprising Minimal Recursion Semantics (MRS) structures and lexicalist oriented semantic features is acquired. Lexical semantic roles enriching the MRS structures are inferred, which are useful to improve the accuracy of deep semantic parsing. Verb classes inference was also investigated, which, together with lexical semantic information provided by VerbNet and PropBank resources, can be substantially beneficial to the parse disambiguation task.
Grammarless Parsing for Joint Inference
24th International Conference on Computational Linguistics (COLING), 2012
Many NLP tasks interact with syntax. The presence of a named entity span, for example, is often a clear indicator of a noun phrase in the parse tree, while a span in the syntax can help indicate the lack of a named entity in the spans that cross it. For these types of problems joint inference offers a better solution than a pipelined approach, and yet large joint models are rarely pursued. In this paper we argue this is due in part to the absence of a general framework for joint inference which can efficiently represent syntactic structure. We propose an alternative and novel method in which constituency parse constraints are imposed on the model via combinatorial factors in a Markov random field, guaranteeing that a variable configuration forms a valid tree. We apply this approach to jointly predicting parse and named entity structure, for which we introduce a zero-order semi-CRF named entity recognizer which also relies on a combinatorial factor. At the junction between these two models, soft constraints coordinate between syntactic constituents and named entity spans, providing an additional layer of flexibility on how these models interact. With this architecture we achieve the best-reported results on both CRF-based parsing and named entity recognition on sections of the OntoNotes corpus, and outperform state-of-the-art parsers on an NP-identification task, while remaining asymptotically faster than traditional grammar-based parsers.
Stochastic HPSG Parse Disambiguation using the Redwoods Corpus
Research on Language and Computation, 2005
This article details our experiments on HPSG parse disambiguation, based on the Redwoods treebank. Using existing and novel stochastic models, we evaluate the usefulness of different information sources for disambiguation – lexical, syntactic, and semantic. We perform careful comparisons of generative and discriminative models using equivalent features and show the consistent advantage of discriminatively trained models. Our best system performs at over 76% sentence exact match accuracy.
Semantic Parsing using Distributional Semantics and Probabilistic Logic
Proceedings of the ACL 2014 Workshop on Semantic Parsing, 2014
We propose a new approach to semantic parsing that is not constrained by a fixed formal ontology and purely logical inference. Instead, we use distributional semantics to generate only the relevant part of an on-the-fly ontology. Sentences and the on-the-fly ontology are represented in probabilistic logic. For inference, we use probabilistic logic frameworks like Markov Logic Networks (MLN) and Probabilistic Soft Logic (PSL). This semantic parsing approach is evaluated on two tasks, Textual Entitlement (RTE) and Textual Similarity (STS), both accomplished using inference in probabilistic logic. Experiments show the potential of the approach.
Montague Meets Markov: Deep Semantics with Probabilistic Logical Form
We combine logical and distributional representations of natural language meaning by transforming distributional similarity judgments into weighted inference rules using Markov Logic Networks (MLNs). We show that this framework supports both judging sentence similarity and recognizing textual entailment by appropriately adapting the MLN implementation of logical connectives. We also show that distributional phrase similarity, used as textual inference rules created on the fly, improves its performance.
Using semantic resources to improve a syntactic dependency parser
2012
Probabilistic syntactic parsing has made rapid progress, but is reaching a performance ceiling. More semantic resources need to be included. We exploit a number of semantic resources to improve parsing accuracy of a dependency parser. We compare semantic lexica on this task, then we extend the back-off chain by punishing underspecified decisions. Further, a simple distributional semantics approach is tested. Selectional restrictions are employed to boost interpretations that are semantically plausible. We also show that self-training can improve parsing even without needing a re-ranker, as we can rely on a sufficiently good estimation of parsing accuracy. Parsing large amounts of data and using it in self-training allows us to learn world knowledge from the distribution of syntactic relation. We show that the performance of the parser considerably improves due to our extensions.
Verbal Polysemy Resolution through Contextualized Clustering of Arguments
Natural language is characterized by a high degree of polysemy, and the majority of content words accept multiple interpretations. However, this does not significantly complicate natural language understanding. Native speakers rely on context to assign the correct sense to each word in an utterance. NLP applications, such as automated word sense disambiguation, require the ability to identify correctly context elements that activate each sense. Our goal in this work is to address the problem of contrasting semantics of the arguments as the source of meaning differentiation for the predicate. We investigate different factors that influence the way sense differentiation for predicates is accomplished in composition and develop a method for identifying semantically diverse arguments that activate the same sense of a polysemous predicate. The method targets specifically polysemous verbs, with an easy extension to other polysemous words. The proposed unsupervised learning method is completely automatic and relies exclusively on distributional information, intentionally eschewing the use of human-constructed knowledge sources and annotated data. We develop the notion of selectional equivalence for polysemous predicates and propose a method for contextualizing the representation of a lexical item with respect to the particular context vi provided by the predicate. We also present the first attempt at developing a sense-annotated data set that targets sense distinctions dependent predominantly on semantics of a single argument as the source of disambiguation for the predicate. We analyze the difficulties involved in doing semantic annotation for such task. We examine different types of relations within sense inventories and give a qualitative analysis of the effects they have on decisions made by the annotators, as well as annotator error. The developed data set is used to evaluate the quality of the proposed clustering method. The output is adapted for evaluation within a standard sense induction paradigm. We use several evaluation measures to assess different aspects of the algorithm's performance. Relative to the baselines, we outperform the best systems in the recent SEMEVAL sense induction task (Agirre et al., 2007) on two out of three measures. We also discuss further extensions and possible uses for the proposed automatic algorithm, including the identification of selectional behavior of complex nominals (Pustejovsky, 1995) and the disambiguation of noun phrases with semantically weak head nouns. vii
Proceedings of the 49th Annual Meeting of …, 2011
We combine multiple word representations based on semantic clusters extracted from the algorithm and syntactic clusters obtained from the Berkeley parser in order to improve discriminative dependency parsing in the MST-Parser framework . We also provide an ensemble method for combining diverse cluster-based models. The two contributions together significantly improves unlabeled dependency accuracy from 90.82% to 92.13%.