Explanation Faithfulness Evaluation Measures Ontology
An ontology for organizing and recommending measures for evaluating the faithfulness of AI explanations
An ontology for organizing and recommending measures for evaluating the faithfulness of AI explanations
Design
Resources
The ontology is centered around the Faithfulness Measure concept. Measures use particular methods (the techniques used to determine the faithfulness score) which evaluate a given proxy. The proxy the faithfulness measure actually measures can then be inferred. Additionally, proxies can make assumptions about how faithfulness works. Faithfulness measures as may also make these assumptions. Many of these assumptions are linked to the proxies and faithfulness measures that make them via SWRL rules in the ontology.
Faithfulness measures evaluate collections of explanations (though those collections may only contain one explanation). These explanations have a particular explanation modality, the form it’s presented in. This allows the ontology to infer the explanation modality(s) that the faithfulness measure can evaluate.
There are several additional properties associated with faithfulness measures. Measures must be attributed to the document describing it, and must have an associated identifier. Faithfulness measures also have an associated granularity level and model specificity. The amount of model access a faithfulness measure requires is also included.
| EFEMO base ontology | efemo.rdf |
| EFEMO individuals KG | efemo_indv.rdf |
The explanation faithfulness evaluation measures ontology presented is given the prefix EFEMO. The ontologies reused, and the prefixes used, are shown below:
| Prefix | Links |
|---|---|
| EO | Explanation Ontology |
| EP | Explanation Patterns Ontology |
| PROV | Provenance Ontology |
| CC | Commons Collections |
| CI | Commons Identifiers |
| CD | Commons Documents |
| AV | Annotation Vocabulary |