Departments of Convergence Medicine, Asan Medical Center, University of Ulsan College of Medicine, Seoul, South Korea
Kyunghwan Kim
Medipixel, Inc., Seoul, South Korea
Chaehyuk Lee
Medipixel, Inc., Seoul, South Korea
Hwi Kwon
Medipixel, Inc., Seoul, South Korea
Jinwoo Park
Medipixel, Inc., Seoul, South Korea
Kyoseok Song
Medipixel, Inc., Seoul, South Korea
Young In Kim
Department of Medical Science, Asan Medical Institute of Convergence Science and Technology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, South Korea
Jeeone Park
Department of Medical Science, Asan Medical Institute of Convergence Science and Technology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, South Korea
Inwook Back
Department of Medical Science, Asan Medical Institute of Convergence Science and Technology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, South Korea
Jae-Hyung Roh
Department of Cardiology in Internal Medicine, Chungnam National University Sejong Hospital, Chungnam National University School of Medicine, Daejeon, South Korea
In percutaneous intervention for treatment of coronary plaques, guidewire navigation is a primary procedure for stent delivery. Steering a flexible guidewire within coronary arteries requires considerable training, and the non-linearity between the control operation and the movement of the guidewire makes precise manipulation difficult. Here, we introduce a deep reinforcement learning (RL) framework for autonomous guidewire navigation in a robot-assisted coronary intervention. Using Rainbow, a segment-wise learning approach is applied to determine how best to accelerate training using human demonstrations, transfer learning, and weight initialization. ‘State’ for RL is customized as a focus window near the guidewire tip, and subgoals are placed to mitigate a sparse reward problem. The RL agent improves performance, eventually enabling the guidewire to reach all valid targets in ‘stable’ phase. For the last 300 out of 1000 episodes, the success rates of the guidewire navigation to the distal-main and side targets were 98% and 99% in 2D and 3D phantoms, respectively. Our framework opens a new direction in the automation of robot-assisted intervention, providing guidance on RL in physical spaces involving mechanical fatigue.
