Motion Enhanced Multi‐Level Tracker (MEMTrack): A Deep Learning‐Based Approach to Microrobot Tracking in Dense and Low‐Contrast Environments

Abstract

Tracking microrobots is challenging due to their minute size and high speed. In biomedical applications, this challenge is exacerbated by the dense surrounding environments with feature sizes and shapes comparable to microrobots. Herein, Motion Enhanced Multi‐level Tracker (MEMTrack) is introduced for detecting and tracking microrobots in dense and low‐contrast environments. Informed by the physics of microrobot motion, synthetic motion features for deep learning‐based object detection and a modified Simple Online and Real‐time Tracking (SORT)algorithm with interpolation are used for tracking. MEMTrack is trained and tested using bacterial micromotors in collagen (tissue phantom), achieving precision and recall of 76% and 51%, respectively. Compared to the state‐of‐the‐art baseline models, MEMTrack provides a minimum of 2.6‐fold higher precision with a reasonably high recall. MEMTrack's generalizability to unseen (aqueous) media and its versatility in tracking microrobots of different shapes, sizes, and motion characteristics are shown. Finally, it is shown that MEMTrack localizes objects with a root‐mean‐square error of less than 1.84 μm and quantifies the average speed of all tested systems with no statistically significant difference from the laboriously produced manual tracking data. MEMTrack significantly advances microrobot localization and tracking in dense and low‐contrast settings and can impact fundamental and translational microrobotic research.

Keywords

• bacteria
• biohybrid microrobotics
• collagen
• computer vision
• machine learning
• multiobject tracking