Computational Experiments in Computer Science Research: A Literature Survey

Abstract

Computational experiments have been important since the 70s to find solutions to problems using computers, and even more now with the advancement of technology, especially artificial intelligence. The aim of this study is to know the level of interest on the part of researchers toward computational experiments in Computer Science. To find out this information, a Systematic Mapping Study has been carried out in IEEE, DBLP, Scopus, ACM and Web of Science databases. A total of 226 papers were selected as primary studies and 22 types of measurements were found, it was determined that the three most used measurements in computational experiments are performance at 51.33%, efficiency at 15.49%, and effectiveness at 15.04%. A total of 25 frameworks used for the computational experiments were determined, however 74.3% of the reports did not specify any framework used, 11.5% used a custom framework and the rest used a specific framework. It is concluded that performance is the most important measurement in computational experiments and that it is necessary for researchers to rely on a framework so that these experiments can be replicated or refined.

Keywords

• Algorithms
• performance
• computational experiments
• computer science
• guidelines
• systematic mapping study