Making TSM better: Preserving foundational philosophy for efficient action recognition

Abstract

In this study, we present the Discriminative Temporal Shift Module (D-TSM), an enhancement of the Temporal Shift Module (TSM) for action recognition. TSM has limitations in capturing intricate temporal dynamics due to its simplistic feature shifting. D-TSM addresses this by introducing a subtraction operation before the shifting. This enables the extraction of discriminative features between adjacent frames, thereby allowing for effective action recognition where subtle motions serve as crucial cues. It preserves TSM’s foundational philosophy, prioritizing minimal computational overhead and no additional parameters. Our experiments demonstrate that D-TSM significantly improves performance of TSM and outperforms other leading 2D CNN-based methods.

Keywords

• Action recognition
• Gesture recognition
• Temporal modeling
• Temporal shift module