Speaker
Description
Generative AI has recently saturated various language learning platforms and applications, such as Duolingo, English Central, Elsa. Integration of features powered by generative AI conveys potential advantage for language learners, suggesting individualized feedback and human-like interaction. However, the exact benefits of generative AI features for language learners are yet to be confirmed.
The following study examines the use of a generative AI chatbot, Replika, in text-based English language conversation practice, and analyzes whether teacher’s corrective feedback for said conversations with the chatbot affects writing skill of English language learners (ELL). The participants are comprised of 34 University-level Japanese ELL, with the experimental group (n = 17) having conversations with the chatbot Replika and receiving corrective feedback from the teacher, and control group (n = 17) practicing English conversation with the chatbot Replika only. Writing skill is evaluated through select CAF indices (average sentence length, measures of lexical diversity voc-D and MTLD, spelling errors per 100 words, number of tokens), which are measured in pre- and post-test writing tasks with Text Inspector, a tool for text analysis. Participants were also asked to provide feedback and impressions at the end of the experiment.
These results show a significant improvement in all six CAF indices regardless of corrective feedback from the teacher. However, teacher’s feedback seems to be particularly effective in the development of lexical diversity measured by indices voc-D and MTLD. The results suggest that text-based conversations with generative AI chatbot, Replika, potentially allow adult ELL to improve some aspects of their writing (number of words in a sentence, diversity of vocabulary, spelling errors), even without corrective feedback from the teacher. However, lexical diversity is improved more in the presence of corrective feedback. The study offers a case in favor of the use of generative AI for writing.
| Keywords | generative AI, CAF, writing |
|---|
