Chat GPT Impact Analysis on API Testing: A Controlled Experiment

Abstract

This research examines the impact of ChatGPT as a learning aid for students in API testing. A controlled experiment compared two groups: one utilizing ChatGPT and the other relying on traditional documentation. The findings indicate that participants using ChatGPT scored significantly higher in both exam tests compared to the documentation group, despite taking longer to complete tasks. Statistical analysis using t-tests confirmed these differences as significant. Post-test surveys revealed an increase in participants confidence and effectiveness in understanding and using APIs after interacting with ChatGPT. However, potential downsides, such as over-reliance on ChatGPT and insufficient deep conceptual understanding, were also observed. The results suggest that while ChatGPT can greatly enhance the quality of learning and productivity in API-related tasks, users must balance AI assistance with independent problem-solving skills. This study underscores the potential of ChatGPT as a valuable educational tool, provided it is integrated thoughtfully into the learning process.

Keywords

• api development
• api platform
• chatgpt
• controlled experiment
• software testing