Application Of Machine Learning Methods To Compare Disciplines Content Using Text Data

Abstract

The paper investigates one of the approaches based on machine learning methods aimed at finding and identifying similar disciplines. In the research we used two most popular methods of machine learning to process text data BERT and Doc2Vec. Machine learning was conducted using the datasets of various disciplines with the total of 2,5 million entries. To assess the quality of the developed models, 30 experts from different scientific fields were engaged in the study to evaluate the level of similarity between the disciplines defined by the trained models. Based on the results of the research, both methods trained using identical datasets generated similar outputs. Another algorithm Doc2Vec, trained on a relatively small data sample with 15 000 entries of the target discipline database that included disciplines descriptions and curriculums, showed better results which justifies the need for developing specific solutions for particular tasks. Further development of machine learning methods and models design to solve specific tasks in the educational field will promote digitalization of education within the area of university operations management.

Keywords

• text mining
• educational data mining
• education
• text's similarity