Uncovering semantic bias in neural network models using a knowledge graph
Nikolov, Andriy ; d'Aquin, Mathieu
Nikolov, Andriy
d'Aquin, Mathieu
Loading...
Repository DOI
Publication Date
2020-10-19
Type
Conference Paper
Downloads
Citation
Nikolov, Andriy, & d'Aquin, Mathieu. (2020). Uncovering semantic bias in neural network models using a knowledge graph. Paper presented at the 29th ACM International Conference on Information and Knowledge Management, Online, 19-23 October.
Abstract
While neural networks models have shown impressive performance in many NLP tasks, lack of interpretability is often seen as a disadvantage. Individual relevance scores assigned by post-hoc explanation methods are not sufficient to show deeper systematic preferences and potential biases of the model that apply consistently across examples. In this paper we apply rule mining using knowledge graphs in combination with neural network explanation methods to uncover such systematic preferences of trained neural models and capture them in the form of conjunctive rules. We test our approach in the context of text classification tasks and show that such rules are able to explain a substantial part of the model behaviour as well as indicate potential causes of misclassifications when the model is applied outside of the initial training context.
Funder
Publisher
ACM
Publisher DOI
10.1145/3340531.3412009
Rights
Attribution-NonCommercial-NoDerivs 3.0 Ireland