Abnormal Activity Detection and Classification of Bus Passengers With In-Vehicle Image Sensing

Abstract

As the self-driving technology is getting mature for public transportation applications, the safety concern of onboard passengers has become an important issue. It is essential to identify inappropriate or hazardous behaviors of passengers for the vehicles without human operators. In this work, we propose a technique to detect and classify the abnormal activities of passengers in a bus environment. Different from the existing human activity classification algorithms, our approach reduces the occlusion and increases the recognition rate by acquiring images from an overhead vision system. To overcome the increased complexity on feature extraction and classification, an action recognition network for top-view images are proposed by incorporating both spatial and temporal information. An image dataset, BUS-HAR, is generated for practical application scenarios with bus passengers. Experiments using real-world scene images have demonstrated the feasibility of our technique compared to existing approaches. The codes and image dataset are made available publicly at https://github.com/richardkuo1999/passenger-action-recognition.

Keywords

• Self-driving vehicle
• action recognition
• abnormal behavior detection
• deep neural network
• computer vision system