Explanation Faithfulness Evaluation Measures Ontology
An ontology for organizing and recommending measures for evaluating the faithfulness of AI explanations
An ontology for organizing and recommending measures for evaluating the faithfulness of AI explanations
Design
Resources
As artificial intelligence systems, especially large language models, gain popularity there is an increased desire to trust these systems. One common method to increase trust is to provide explanations. These explanations provide information about the system’s workings and the knowledge used in its general reasoning processes or the processes behind a specific decision [1]. However, ensuring that these explanations are faithful, that they accurately represent that reasoning process, is a difficult and ongoing task [2]. Over 60 measures have been proposed in the last 5 years to quantify how faithful an explanation is, but it is difficult to compare them to decide which one is most appropriate for a given use case.
An ontology would provide a structured representation of faithfulness measures that would make comparisons easier. It would also document the inferences that researchers use to evaluate and categorize these measures. Researchers proposing new measures would be able to communicate the benefits of their idea with a shared vocabulary. Ideally it would also serve as the schema for a comprehensive knowledge graph of measures which would power a recommendation system for AI explainability researchers.
Therefore this project will develop such as ontology. For this project, the measures will be limited to those used to evaluate language-task explanations. This includes both natural language and saliency explanations. Image and graph-based tasks and their explanations will not be included due to the variety in their explanations and faithfulness measures, but the ontology will be developed such that future work could expand the ontology to include these measures. The development process will follow the ontology engineer methods presented in [3] and success will be determined by whether we can answer the competency questions developed in the project’s use case.
[1] Chari, S., Gruen, D. M., Seneviratne, O., & McGuinness, D. L. (2020). Foundations of explainable knowledge-enabled systems. In Knowledge Graphs for eXplainable Artificial Intelligence: Foundations, Applications and Challenges (pp. 23-48). IOS Press.
[2] Jacovi, A., & Goldberg, Y. (2020). Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness?. arXiv preprint arXiv:2004.03685.
[3] Kendall, E. F., & McGuinness, D. L. (2019). Ontology engineering. Morgan & Claypool Publishers.
| Resources | Links |
|---|---|
| Use Case | Use Case Page |
| Term List | Terminology Page |
| Ontology Overview | Ontology Page |
| Usage Scenarios, Competency Questions, Queries | Example Page |
Danielle Villa*1, Maria Chang2 , Deborah L. McGuinness1
None so far, but stay tuned!