Evolution of Methodological Approaches to Evaluating the Efficiency of the Activities of Research Organizations in Ukraine

Abstract

The evolution of methodological approaches for evaluation of efficiency of different scientific institutions used in Ukraine is considered. The analysis of current regulatory acts and the methodological approach to state certification and evaluation of the activities of scientific institutions, which has been used after gaining the independence, indicates their orientation on a system of quantitative parameters. Evaluation of the effectiveness of a scientific institution’s activities by a single indicator based on complex and ambiguously defined calculations does not meet international standards, which are implemented in the EU countries. In order to implement measures, mentioned in the Concept of Development of the National Academy of Sciences of Ukraine for 2014-2023, different approach to evaluation of activities of scientific institutions, was applied in the National Academy of Sciences of Ukraine (NASAU). The introduced Methodology for evaluating the effectiveness of scientific institutions of the National Academy of Sciences of Ukraine is based on the methods, principles and approaches to assessment, defined in the Leiden manifesto of the leading specialists on scientometrics. According to this methodology, procedure of assessment could not be reduced to the utilization of selected quantitative indicators only, despite their importance. In the NASAU, the adoption of evaluation results is carried out on the basis of a qualitative analysis, which covers various aspects of the activities of the institution and its divisions. It is executed by the independent experts, who are specialists in scientific areas of the institution’s activities and who have high reputation among their colleagues. The main differences between two approaches to the evaluation methods, which are used today in Ukraine to assess the activities of scientific institutions of the National Academy of Sciences of Ukraine, are considered. The results obtained are analyzed and compared. The positive aspects of evaluating scientific institutions of NASU through expert judgment are identified, and the problems that arise during the assessment procedure are assessed. As a conclusion, shift from determining the results of the activities of scientific institutions by calculating a single indicator provides an opportunity for experts to evaluate institutions as a whole and their divisions, taking into account the uniqueness and specific features, which depend on the scientific disciplines and research areas. This also opens the way to determine the development prospects and place of the research institutions in the national and international ‘scientific map’.

Keywords

• research organization, evaluation methodology, evaluation criteria, peer review, comprehensive indicator, performance based research funding.