Intelligent Systems with Applications (Mar 2024)
Claude 2.0 large language model: Tackling a real-world classification problem with a new iterative prompt engineering approach
Abstract
In the last year, Large Language Models (LLMs) have transformed the way of tackling problems, opening up new perspectives in various works and research fields, due to their ability to generate and understand human languages. In this regard, the recent release of Claude 2.0 has contributed to the processing of more complex prompts. In this scenario, the goal of this paper is to evaluate the effectiveness of Claude 2.0 in a specific classification task. In particular, we considered the Forest cover-type problem, concerning the prediction of a cover-type value according to the geospatial characterization of target worldwide areas. To this end, we propose a novel iterative prompt template engineering approach, which integrates files by exploiting prompts and evaluates the quality of responses provided by the LLM. Moreover, we conducted several comparative analyses to evaluate the effectiveness of Claude 2.0 with respect to online and batch learning models. The results demonstrated that, although some online and batch models performed better than Claude 2.0, the new iterative prompt engineering approach improved the quality of responses, leading to better performance with increases ranging from 14% to 32% in terms of accuracy, precision, recall, and F1-score.
