Real-Time Team Performance and Workload Prediction From Voice Communications

Abstract

Automatic prediction of team performance and workload plays a crucial role in team selection, training, evaluation, and re-training processes. This study investigated the potential of using voice analysis of team-based communication for predicting team workload (TW) and team performance (TP). Both the TW and TP categories were labeled objectively. Ten teams of three participants were tasked with completing a computer-based command-and-control simulation that required communication of task-specific information to each team member. Recordings of each participant’s voice communications were used to train Convolution Neural Network (CNN) models for each team separately. It was hypothesized that integrating TW and TP information into the prediction process would support the prediction of both TW and TP categories. Two experiments were conducted. In the first experiment, the TP prediction networks were fine-tuned to predict TW, and conversely, the TW prediction networks were fine-tuned to predict TP. In the second experiment, the TP or TW prediction based on the assembly of interconnected TP and TW classifiers was tested. Both experiments confirmed the hypothesis. It was shown that task-related pre-requisite knowledge embedded into the neural network reduced neural network model training time and improved performance without increasing the training data size. Predictions based on combined TW and TP classification outcomes (using either separate or interconnected TW or TP classifiers) outperformed the baseline method using a single CNN model trained to predict either TW or TP alone. The classification accuracy was consistent with previously reported cognitive load prediction based on objective measures.

Keywords

• Speech classification
• voice classification
• cognitive load prediction
• workload prediction
• performance prediction
• team performance monitoring