Speaker
Abstract section 2: Contribution/research questions
Did the LLM always offer appropriate grammatical corrections where needed?
Did the LLM always recognise grammatically acceptable questions?
Did the LLM always offer advice on appropriacy where needed?
Did the LLM always recognise the student’s intended meaning in offering feedback?
What kind of questions proved difficult for LLMs to assess properly?
Abstract section 4: Outcomes/results
While the clear majority of feedback in each round was assessed to be both correct and appropriate (whether leaving the question as is or offering appropriate correction), a concerning minority of feedback offered either overcorrection (correction where none was needed), or failed to recognise the student’s likely intention and thus failed to offer appropriate feedback that a teacher familiar with Japanese EFL learners would offer. The former is a concern for student motivation, while the latter for the efficacy of the approach. The paper qualitatively analyses the kinds of questions where appropriate feedback failed to develop questions for further research.
Abstract section 5: References
Dillon, T. (2025) Automated Analysis of ESL Interaction Tasks Using ChatGPT. Korean Journal of English Language and Linguistics, Vol 25, July 2025, pp. 955-981
DOI: 10.15738/kjell.25..202507.955
Nufus, Z., & Warrah, S. M. (2025). ChatGPT's Handling of L2 Learners’ Fossilized Errors: A Linguistic Evaluation. DUTIES: Education and Humanities International Journal, 1(2), 21-41. DOI:10.70152/duties.v1i2.219
Abstract section 1: Relevance
Using LLMs to provide corrective feedback to students gives the opportunity for students to study autonomously as well as receive feedback more quickly. However, most research into such use looks at feedback on student writing, where a longer text provides greater clues for the LLM to use to provide feedback. Nufus and Warrah (2025) and Dillon (2025) show that LLMs may struggle with sentences lacking sufficient context where a human assessor would not. This situation may occur where students seek feedback for individual phrases in preparing for conversation practice. Research is needed into the quality of such feedback.
Abstract section 3: Content/method
A class of 25 students over three rounds were asked to write a series of questions unaided on a basic conversational topic (self-introduction, families, and hometowns). They were then directed to ask a free version of ChatGPT to assess these questions with identical wording, specifying that they were Japanese learners of English. They transcribed the response, which was taken in by the teacher and assessed according to the research questions above.
| Title | Student use of LLMs to correct low context phrases |
|---|---|
| Teaching Context | College and university education |