Construction of an Interactive Platform for Piano Teaching in Virtual Reality Environment

Abstract

Under the impact of the wave of new technology, the form of piano teaching is no longer limited to the classroom and paper textbooks, and modern information technology, such as the Internet and virtual reality, is constantly integrated into traditional piano teaching. This paper utilizes the Unity3D engine to create a virtual reality environment for piano teaching interaction, based on VR technology, and to construct an interactive platform for piano teaching. Firstly, based on binocular stereo vision for the three-dimensional reconstruction of the teaching interactive scene, based on the improvement of the LeNet5 network for the user’s head gesture tracking, the constructed virtual scene realizes the real-time movement with the movement of the human head. The test set images using CPU-i76700HQ and GTX960 graphics cards consume 78ms and 25ms per frame, respectively, which meets the real-time system requirements. In this paper, the improved LeNet5 algorithm for head pose estimation without face detection is applicable and has a high detection accuracy of 99.31%, which is better than the algorithms such as Adaboost+Hough, LBF+BP, CNN+Heatmaps, DCNN, and FSA-Net, and is able to satisfy the real-time detection requirements of the virtual scene. The virtual piano teaching interactive platform has an average rating of more than 4 in attractiveness, subjectivity, interactivity, and convenience, but the running speed is only 3.46, so it is necessary to strengthen the technical support and improve the software and hardware level to break through the development bottleneck. Traditional piano teaching is given new vitality and kinetic energy by the interactive platform constructed in this paper for piano teaching in a virtual reality environment.

Keywords

• binocular stereo vision
• 3d reconstruction
• improved lenet5
• head pose estimation
• piano teaching interactive platform
• 97c70