Explainable and Fair AI: Balancing Performance in Financial and Real Estate Machine Learning Models

Abstract

This paper introduces a framework that integrates fairness and transparency into advanced machine learning models, specifically LightGBM and XGBoost, applied to loan approval and house price prediction datasets. The key contribution is using fairness-focused techniques, such as Calibrated Equalized Odds and Intersectional Fairness, which are not widely studied in financial and real estate contexts. To improve model transparency, SHAP (SHapley Additive exPlanations) is utilized along with a novel fairness-based interpretability method to measure both model fairness and the importance of individual features. Through comprehensive experiments, we show that LightGBM delivers high accuracy while balancing fairness and performance effectively. The broader relevance of this work is discussed in the context of governance and regulatory requirements, highlighting the importance of responsible practices in high-stakes financial decision-making processes. This research highlights the importance of fairness and transparency in real-world applications, promoting equity, trust, and adherence to evolving legal standards, and provides practical insights for data scientists, machine learning researchers, and professionals in the real estate and financial sectors.

Keywords

• Fairness in AI
• explainable AI
• SHAP
• loan approval prediction
• equalized odds
• intersectional fairness