IEEE Access (Jan 2023)
Evaluating the Effectiveness of Author-Count Based Metrics in Measuring Scientific Contributions
Abstract
The assessment and evaluation of the academic influence of the researcher is a challenging task. This task allows the scientific community to make valuable decisions, such as identifying leading experts in a specific field, nominating candidates for scientific awards, awarding scholarships/grants, promoting researchers, and selecting tenure positions. Scientists have proposed various varied and multifaceted metrics to determine the most influential researchers. These metrics are the citation count, total number of publications, hybrid approaches, h-index, and variants of the h-index. Contemporary research in this domain shows that there is no universally accepted yardstick available for determining the finest parameter to recognize the most influential researcher within a particular domain. Moreover, to recognize the potential metric, some researchers have conducted evaluation surveys. In these evaluation surveys, the researchers utilized a limited number of indices on a small and imbalanced dataset as well as on fictional case scenarios, which makes it challenging to determine the significance and influence of each metric over the others. The present study computed fourteen distinct metrics based on the author-count. Our aim is to determine the potential metrics. For experimental purposes, we collected 1050 researchers’ data from the mathematics domain. For the benchmark dataset, we have collected the awardee’s data of the last three decades of four different societies of mathematics domain. To evaluate these metrics, we first computed the Spearman correlations among the obtained values of these metrics to assess their similarities and differences. The results showed a high degree of correlation among these metrics. However, some metrics represent weak correlations, signifying that their rankings are highly dissimilar to those of the other metrics. Furthermore, the position of award winners is checked in the top 10, 50, and 100 return records based on a ranked list of each metric. The potential value of each metric such as the hf metric, indicates that 60% of the awardees in the top 10% of the ranked list are associated with this, whereas the potential value of fractional g metric is linked with 49% of the awardees in the top 100% of the ranked list. In addition, it is further scrutinized that most of the award winners lie in a top position belonging to IMS, LMS, and AMS society return by fractional g-index, gf index, and gm index, which indicates that there is some relationship between these societies and metrics.
Keywords
