Predicting the Rectal Temperature of Dairy Cows Using Infrared Thermography and Multimodal Machine Learning

Abstract

The paper proposes an approach for estimating the rectal temperature of dairy cows based on the non-invasive real-time monitoring of their respiration rates and the temperature-humidity index (THI) of the environment, combined with the analysis of infrared images. We use multimodal machine learning for the joint processing (fusion) of these different types of data. The implementation is performed using a new open source AutoML Python module named AutoGluon. After training and optimizing three different regression models (a neural network and two powerful boosting algorithms), it reduces the variance of the result using level 2 stacking. The evaluation metrics we work with are the mean absolute error, MAE, and the coefficient of determination, R2. For a sample of 295 studied animals, a weighted ensemble provides quite decent results: R2=0.73 and MAE ≈0.1 °C. For another sample of 118 cows, we additionally use the pulse rate as a predictor and we achieve R2=0.65, MAE ≈0.2 °C. The maximal error is almost 1 °C due to outliers, but the median absolute error in both cases is significantly lower: MedAE 0.1 °C, with the standard deviations respectively being 0.118° and 0.137°. These encouraging results give us confidence that tabular and visual data fusion in ML models has great potential for the advancement of non-invasive real-time monitoring and early diagnostics methods.

Keywords

• infrared thermography
• multimodal machine learning
• heat stress
• smart farming