NotiMind: Utilizing Responses to Smart Phone Notifications as Affective Sensors

Abstract

Today's mobile phone users are faced with large numbers of notifications on social media, ranging from new followers on Twitter and emails to messages received from WhatsApp and Facebook. These digital alerts continuously disrupt activities through instant calls for attention. This paper examines closely the way everyday users interact with notifications and their impact on users' emotion. Fifty users were recruited to download our application NotiMind and use it over a five-week period. Users' phones collected thousands of social and system notifications along with affect data collected via self-reported Positive and Negative Affect Schedule tests three times a day. Results showed a noticeable correlation between positive affective measures and keyboard activities. When large numbers of post and remove notifications occur, a corresponding increase in negative affective measures is detected. Our predictive model has achieved a good accuracy level using three different "in the wild" classifiers (F-measure 74%-78% within-subject model, 72%-76% global model). Our findings show that it is possible to automatically predict when people are experiencing positive, neutral, or negative affective states based on interactions with notifications. We also show how our findings open the door to a wide range of applications in relation to emotion awareness on social and mobile communication.

Keywords

• Mobile sensing
• affective computing
• mobile computing
• mobile social media
• machine learning