Social Technologies (Dec 2011)
Automated Evaluation of Computer Programs at Undergraduate Level: Suitability Study for Competitive Events
Abstract
Purpose—The paper is aimed towards the problem of automated evaluation of computer programs designed by students during competitive events where the participants have to design an algorithm and to implement it as a working program, which has to be evaluated. A similar evaluation has to be performed at universities by the lecturers teaching basics of programming. The purpose of this paper is to investigate the automated evaluation applied to computer programs designed by the university students, and to analyze the suitability of such methods for evaluation of computer programs designed by high school students in competitive events.Design/methodology/approach—Comparative analysis is the main approach used in this paper.Findings—The educational and technical goals that seek evaluation of computer programs at the university level are different from those in informatics contests. Therefore the majority of approaches applied at universities have led to development of automated evaluation systems with functionalities different than the ones required by informatics contests, and in the opinion of the author cannot be easily transferred to informatics contests. We identified one area where such transfer might be possible: automated evaluation of programming style.Research limitations/implications—The experience of automated evaluation of programming style applied at the universities has a potential to be transferred to informatics contests; however the universities may ask their students to apply specific programming style, while informatics contests should accept a very broad range of reasonable programming styles. Additional investigation is required to answer the question in which way and to which extent the experience gained at the universities can be applied in informatics contests and the maturity exam.Practical implications—This paper is a step towards developing a fair and motivated evaluation scheme in informatics contests. The paper would be useful for the researchers as it gives guidelines for future research.Originality/Value—It is very important to apply fair and motivated evaluation in informatics contests as they involve the majority of high-school students interested in computer science in Lithuania and some other countries as well and may motivate them to study computer science at the university. To the knowledge of the author, this is the first paper analyzing the suitability of automated evaluation methods developed at the university level, to be applied for the evaluation at high school level in particular in informatics contests. On the other hand, the evaluation in informatics contests is an issue that attracts the interest of international community of informatics contests.
