A Framework for Ontology Evaluation (original) (raw)
Related papers
A snapshot of ontology evaluation criteria and strategies
Ontologies are key to information retrieval, semantic integration of datasets, and semantic similarity analyses. Evaluating ontologies (especially defining what constitutes a “good” or “better” ontology) is therefore of central importance for the Semantic Web community. Various criteria have been introduced in the literature to evaluate ontologies, and this article classifies them according to their relevance to the design or the implementation phase of ontology development. In addition, the article compiles strategies for ontology evaluation based on ontologies published until 2017 in two outlets: the Semantic Web Journal, and the Journal of Web Semantics. Gaps and opportunities for future research on ontology evaluation are exposed towards the end of the paper.
Approaches , methods , metrics , measures , and subjectivity in ontology evaluation : A survey
2014
Ontology evaluation is concerned with ascertain two important aspects of ontologies: quality and correctness. The distinction between the two is attempted in this survey as a way to better approach ontology evaluation. The role that ontologies play on the semantic web at large has been has been argued to have catalyzed the proliferation of ontologies in existence. This has also presented the challenge of deciding one the suitability of an given ontology to one’s purposes as compared with another ontology in a similar domain. This survey intends to analyze the state of the art in ontology evaluate spanning such topics as the approaches to ontology evaluation, the metrics and measures used. Particular interest is given to Data-driven ontology evaluation with special emphasis on the notion of bias and it’s relevance to evaluation results. Chief among the outputs of this survey is the gap analysis on the topic of ontology evaluation.
Ontology evaluation and validation
2005
The need for evaluation methodologies in the field of ontology development and reuse showed up as soon as 1994 and it has been growing ever since [Sure 2004]. Yet, no comprehensive and global approach to this problem has been proposed to date. What we present here is a novel perspective that tries to integrate the varied (and differently aimed) ontology evaluation methods proposed so far. Particular efforts are devoted to considering as many as possible measures that can affect the quality of an ontology, and whose ...
Semantic Web, 2007
Recent years have seen rapid progress in the development of ontologies as semantic models intended to capture and represent aspects of the real world. There is, however, great variation in the quality of ontologies. If ontologies are to become progressively better in the future, more rigorously developed, and more appropriately compared, then a systematic discipline of ontology evaluation must be created to ensure quality of content and methodology. Systematic methods for ontology evaluation will take into account representation of individual ontologies, performance (in terms of accuracy, domain coverage and the efficiency and quality of automated reasoning using the ontologies) on tasks for which the ontology is designed and used, degree of alignment with other ontologies and their compatibility with automated reasoning. A sound and systematic approach to ontology evaluation is required to transform ontology engineering into a true scientific and engineering discipline. This chapter discusses issues and problems in ontology evaluation, describes some current strategies, and suggests some approaches that might be useful in the future.
Prospects and Possibilities for Ontology Evaluation: The View from NCOR
In this position paper, we briefly describe the perspective of the US National Center for Ontological Research (NCOR, http://ncor.us) on ontology evaluation. NCOR's inauguration was recently held (October 2005), and at that time goals were identified and committees formed to pursue those goals, including the Ontology Evaluation Committee. This committee is charged with developing a plan for the evaluation of ontologies that is designed to transform ontological engineering into a true scientific and engineering discipline. This paper discusses some issues on ontology evaluation, including the relevant questions to ask, and suggests some approaches.
A SURVEY OF ONTOLOGY EVALUATION TECHNIQUES
An ontology is an explicit formal conceptualization of some domain of interest. Ontologies are increasingly used in various fields such as knowledge management, information extraction, and the semantic web. Ontology evaluation is the problem of assessing a given ontology from the point of view of a particular criterion of application, typically in order to determine which of several ontologies would best suit a particular purpose. This paper presents a survey of the state of the art in ontology evaluation.
Meeting Report : “ NIST Workshop on Ontology Evaluation ”
2012
The National Institute for Standards and Technology sponsored a workshop in October, 2007, on the subject of ontology evaluation. An international group of invited experts met for two days to discuss problems in measuring ontology quality. The workshop highlighted several divisions among ontology developers regarding approaches to ontology evaluation. These divisions were generally reflective of the opinions of the participants. However, the workshop documented a paucity of empirical evidence in support of any particular position. Given the importance of ontologies to every knowledge-intensive human activity, there is an urgent need for research to develop an empirically derived knowledge base of best practices in ontology engineering and methods for assuring ontology quality over time. This is a report of the workshop discussion and brainstorming by the participants about what such a research program might look like.
Methods for ontology evaluation
2005
A classification of methods and tools for the evaluation of ontologies for industrial practice is provided. It concerns methods and tools to:
Toward Ontology Evaluation across the lifecycle
Applied ontology
Currently, there is no agreed on methodology for development of ontologies, and there is no consensus on how ontologies should be evaluated. Consequently, evaluation techniques and tools are not widely utilized in the development of ontologies. This can lead to ontologies of poor quality and is an obstacle to the successful deployment of ontologies as a technology. The goal of the Ontology Summit 2013 was to create guidance for ontology developers and users on how to evaluate ontologies. Over a period of four months a variety of approaches were discussed by participants, who represented a broad spectrum of ontology, software, and system developers and users. We explored how established best practices in systems engineering and in software engineering can be utilized in ontology development. This document focuses on the evaluation of five aspects of the quality of ontologies: intelligibility, fidelity, craftsmanship, fitness, and deployability. A model for the ontology life cycle is pre...
A Survey on Ontology Evaluation Methods
Proceedings of the 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management, 2015
Ontologies nowadays have become widely used for knowledge representation, and are considered as foundation for Semantic Web. However with their wide spread usage, a question of their evaluation increased even more. This paper addresses the issue of finding an efficient ontology evaluation method by presenting the existing ontology evaluation techniques, while discussing their advantages and drawbacks. The presented ontology evaluation techniques can be grouped into four categories: gold standard-based, corpus-based, task-based and criteria based approaches.