General Actions:
Ontology Quality Assurance
Last modified by Linda van den Brink on 2014/10/02 11:16
====THIS DOCUMENT IS UNDER CONSTRUCTION
Introduction
The development process of the CB-NL ontology is guided by requirements, modeling principles and rules, design patterns, software tooling and human interaction.
CB-NL requirements come first. They determine the CB-NL capabilities: for what automated task can you use CB-NL? The requirements are documented on this page. Modeling Principles are next. They describe how CB-NL is developed. The modeling principles are described here. Design patterns help modelers to decide which OWL code represents which natural language statement. Software tools support the ontology development and QA process. And in the end a human always needs to test and verify if this all leads to the required endproduct. In this sense the development of CB-NL can be compared to any software development life cycle. There are however particular differences between developing an ontology such as CB-NL and developing any other kind of software. This page describes just that difference. It describes the way CB-NL has implemented the QA process.
The CB-NL ontology QA Process
The CB-NL is a software representation (in the form of a file) of statements and axioms that record the meaning of concepts used in the construction sector. For QA, both the representation and the meaning of the concepts need to be correct. The representation of the concepts is checked via a syntax check. Correct syntax refers to proper use of the W3C RDF, SKOS and OWL2 standards.This is an automated task. The meaning of the concepts is verified via a semantic check. This can be done automatically. Correct semantics means that the ontology is consistent (the ontology axioms do not contain contradictions). Correct semantics also means that concepts are coded as intended. This can be checked automatically when certain rules are implemented (such as design patterns) but they must always be verified via human intervention.
The quality of the CB-NL content is guaranteed by following these processes and principles:
- The concepts are modelled according to the Modeling guide
- For the coding of the concepts ontology editors are used to ensure use of proper syntax. When issues arise online RDF and OWL validators are used.
- Only modeling constructions that are described in Design Patterns are used
- Every ontology release is checked for consistent semantics in an automated process
- the ontology is verified with a reasoner
- reverse engineering of design patterns
- Human eyes have checked the correct meaning of every concept
Every release of CB-NL is accompanied by a report with the outcome of the QA test that were executed. An example of such a report can be found .
Tools used in the CB-NL QA process
The QA process is supported by a number of free tools that are available for download on the web.
Tools for syntax validation
The tools that the CB-NL project uses to verify the correct use of syntax are:
Ontology editor
Ontology editors display and log syntax errors via the user interface. CB-NL is validated with Topbraid Composer (use this link to free version) and Protege (version 4) .
Topbraid composer displays a syntax error like this:
And Protege like this:
RDF validator
purpose: check if code is valid RDF
source:http://www.w3.org/RDF/Validator/ (works only for RDF/XML)
OWL validator
purpose: check if code is valid OWL and determine which OWL dialect is used
source:http://mowl-power.cs.man.ac.uk:8080/validator/
Tools for validation of semantics
Reasoner
A reasoner or inference engine can check whether an ontology is logically consistent. It can also infer logical entailments.
The command line reasoner Pellet is used to verify the semantic quality of CB-NL. Pellet supports multiple reasoning tasks. For optimal semantic quality checking they must be executed in a particular order:
- pellet consistency (if not consistent - use pellet explain and fix issues first),
- pellet classify,
- pellet unsat.