An Improved Point-Level Supervision Method for Temporal Action Localization

Abstract

Recently, with the expansion of the video platform market, research has been actively conducted on temporal action localization (TAL) for detecting actions in atypical videos. Most learning methods for TAL include full and weak supervision (weak supervision with only action classes) approaches. Full supervision requires considerable time for labeling and weak supervision exhibits low localization performance owing to the lack of informative annotations. To solve this problem, point-level weak supervision using single-point timestamps within the temporal interval of action instances has been proposed, which demonstrates superior performance to weakly-supervised methods using only action classes of action instances. In this study, we proposed an improved point-level supervision mechanism that provides point-level annotations for each action and background instance. In addition, a widely used multiple instance learning (MIL)-based framework was used to verify the proposed method, and pseudo-labels were used for action instance boundary learning. Also, the background point loss was designed to leverage the added point-level annotations. The datasets used in the experiment were THUMOS14, GTEA, BEOID, and ActivityNet1.2, and improved results were obtained compared to existing point-level supervision. The code is available from https://github.com/sang9390/An-Improved-Point-Level-Supervision-Method-for-TAL.

Keywords

• Temporal action localization
• multiple instance learning
• fully-supervised learning
• weakly-supervised learning