FCE: Feedback Based Counterfactual Explanations for Explainable AI

Abstract

Artificial Intelligence can provide quite accurate predictions for critical applications (e.g., healthcare), but lacks the ability to explain its internal mechanism in most applications which require high interaction with humans. Even if many studies analyze machine learning models and their learning behavior and eventually provide an interpretation of the inner mechanics of these models, these studies often entail a simpler surrogate model, which generates explanations by producing a piece of interpretable information such as feature scores. The crucial caveat against these studies is the lack of human involvement in the design and evaluation of explanations, consequently giving rise to trust issues and lack of acceptance and understanding. To this end, we address this limitation by involving humans in the counterfactual explanation generation process which is enriched with user feedback, thus enhancing the automated explanations which are better aligned with user expectations. In this paper, we propose a user feedback based counterfactual explanation approach (FCE) for explainable Artificial Intelligence. In our work, we utilize feedback in two ways: first, to customize the explanations by providing the acceptable ranges in the feature space where to look for feasible counterfactuals, and second, to evaluate the generated explanations.

Keywords

• Counterfactual explanations
• explainable AI
• human-in-the-loop
• interactive machine learning
• user feedback