Exploring the potential of AI for pragmatics instruction

Abstract

Large language models (LLM) (e.g., ChatGPT) have garnered massive attention due to their potential to serve as interactive partners for a wide range of teaching and learning activities. In instructed second language acquisition, one potential use of LLMs is for pragmatics instruction. The current study sought to assess ChatGPT’s suitability for this task by analyzing its output in response to 144 DCTs (discourse completion tasks), which test a language learner’s ability to create apologies, requests, and refusals over differential levels of social and power distance, and degrees of imposition. Quantitatively, ChatGPT produced significantly shorter apologies than the other two speech acts (p < .001, R2 = .49), utilizing fewer pragmatic strategies (p < .001, R2 = .38). Closer social distance and greater imposition also resulted in longer responses, with higher imposition also eliciting ChatGPT to use more pragmatic strategies. Qualitatively, human raters scored ChatGPT’s apologies significantly lower in terms of their appropriateness, politeness, and language use, despite there being no statistical differences in ratings for requests and refusals. The main pedagogical implications are that, while ChatGPT offers an attractive option for students looking for more language exposure and interaction outside of the classroom, some degree of skepticism is still required due to its quantitative and qualitative inconsistencies.

Keywords

• ChatGPT
• large language models
• pragmatics
• pragmatic instruction
• discourse completion tasks