Zhejiang Daxue xuebao. Lixue ban (Jan 2025)
Stage actor tracking method based on infrared ink marking(基于红外油墨标记的舞台演员跟踪算法)
Abstract
Great progress has been made in multi-object tracking. When well illuminated, the state-of-the-art multi-object tracking algorithms can reach real-time and stable tracking results. However, in the case of stage performance where extreme illumination and severe occlusion always happen, it is a great challenge to achieve comparable performance. Due to the characteristics of similar appearance of actors, drastic changes in illumination, and frequent occlusions that always happen on the stage performance, direct use of existing tracking algorithms will cause a problem of frequent actor identities switching, making the related downstream tasks such as actor action recognition and virtual reality based projection difficult to be carried out. To solve this problem, we propose a stage actor tracking method based on near-infrared ink. As the energy spectrum of stage illuminations mainly concentrate on visible light bands, the infrared illumination environment in stage performance is relatively clean. Therefore, we use near-infrared flash light that is invisible to human eyes to illuminate the stage, and capture images with an infrared camera. Besides, anti-occlusion, invisible (under visible lights) infrared ink marks are designed to boost the actors' resolution in appearance. More specifically, the infrared ink marks are painted on actors' performance costumes in the form of binary ring codes. Then class labels of the ring codes are designated to associated actors as their identities for stable tracking. Experimental results on simulation dataset and real dataset show that the proposed binary ring codes painted in infrared ink preserve a recognition accuracy of over 90% with respect to occlusion rate at 60%, demonstrating its excellent anti-occlusion performance. Comparison result of actor tracking show that the proposed intra-frame data association method based on the intersection over maximal intersection index can correctly connect the detected marks and actors. The recall rate and accuracy of ID are greatly improved by integrating our method into state-of-the-art tracking models. While less affecting the actor performance and audience experience, the proposed method solves the problem of unstable tracking caused by extreme dim illumination and similar actors' appearance with lower cost and higher feasibility and is promising to be applied to the field of stage performance.(随着深度学习技术的不断成熟，基于深度学习的多目标跟踪研究取得了巨大进展。在良好光照条件下，现有基于深度学习的多目标跟踪算法能实现实时、稳定跟踪。然而，在极限光照与遮挡严重的舞台演出环境中，对舞台演员的稳定跟踪仍然面临巨大挑战。舞台演出存在演员表观相似、光照变化剧烈、遮挡频繁等问题，直接使用现有跟踪算法因演员身份切换频繁，导致基于演员跟踪的下游工作，如演员动作识别、虚实投影等无法有效开展。为此，提出了一种基于近红外油墨的舞台演员跟踪算法。由于舞台光源能量谱集中在可见光波段，红外光环境较干净，因此，在红外波段设计了一种抗遮挡、隐形的（可见光下不可见）油墨标记，以增强演员表观辨识度，即将红外油墨以二值环形码方式添加在演员服饰上，将环形码对应的类别标签作为关联演员ID，实现对舞台演员的稳定跟踪。在仿真与真实的标记数据集上的实验结果表明，基于红外油墨的二值环形码在60%的遮挡率下仍能达到90%以上的识别准确率，具有很好的抗遮挡性能。演员跟踪实验结果表明，基于交集比指标的帧内数据关联算法能提升ID跟踪轨迹的稳定性，将其应用于现有跟踪模型，ID的召回率和准确率均得到了较大提升。在不影响舞台演出与观演体验的前提下，解决了舞台暗光、演员表观相似导致的跟踪不稳定问题，且算法成本低、可行性高，在舞台演艺行业具有广泛的应用前景。)
Keywords
