Explaining decision structures and data value for neural networks in crop yield prediction

Abstract

Neural networks are powerful machine learning models, but their reliability and trust are often criticized due to the unclear nature of their internal learned relationships. We explored neural network learning behavior in wheat yield prediction using game theory-based methods (SHapley Additive exPlanations, Shapley-like, cohort Owen), examined data impact on model performance and show optimization approaches. By identifying and removing harmful data samples, we demonstrated a significant improvement in prediction accuracy. We show that neural networks can learn decision patterns that often align with agronomic causal relationships. Building on these insights we propose a novel approach using an autoencoder to detect statistical implausible decisions, allowing us to flag and correct potential ‘misjudgements’, resulting in an 11% reduction in global model error. The proposed explainability methods can optimize the neural network training process through improved data acquisition and revising the internal learning process. This enhancement presents neural networks as trustworthy simulation agents for agricultural research, capable of supporting new scientific discoveries and assisting in real-world applications.

Keywords

• Shapley
• cohort Owen
• explainability
• data valuation
• autoencoder
• machine learning