Speaker
Abstract section 5: References
Birdsell, B. J. (2022). Student writings with DeepL: Teacher evaluations and implications for teaching. In P. Ferguson, & R. Derrah (Eds.), Reflections and New Perspectives. JALT. https://doi.org/10.37546/JALTPCP2021-14
Raine, P. (2023). ChatGPT: Initial implications for language teaching and learning. The Language Teacher, 47(2), 38-41. https://doi.org/10.37546/JALTTLT47.2
Tseng, Y, & Lin, Y. (2024). Enhancing English as a foreign language (EFL) learners’ writing with ChatGPT: A university-level course design. The Electronic Journal of e-Learning, 22(2), 78-97. https://doi.org/10.34190/ejel.21.5.3329
Abstract section 3: Content/method
This research is a longitudinal study using the traditional scientific experimentation model of control and variable groups. Approximately 100 first-year university students completed ten English writing tasks over the course of one year. Half-way through the research, the control and variable group conditions were adjusted to account for potential priming effects from English reading. Writing samples were assessed on word count length and type/token ratio using ditigal analysis, and relevance and accuracy through researcher evaluation.
Abstract section 4: Outcomes/results
Although there were several developments in student writing throughout the year between writing tasks, there were few practical differences between the control and variable groups. Results were mixed as students in the control group (self-revision) consistantly wrote with more lexical diversity (M =.628, SD= <.001) than students in the experimental group (LLM-assisted revision)(M = .615, SC = .001) (t(9)=3.517, p .007) but students in the experimental group regularly produced longer essays (M=.552, sd=.005) than the control group (M=.511 SD=.003) (t(9)=-4.665, p=.001).
Abstract section 2: Contribution/research questions
Did students using a large language model to revise their essays develop writing skills of quantity, lexical diversity, relevancy and accuracy at a different rate than students using traditional self-evaluation to revise their essays?
Abstract section 1: Relevance
No technology has penetrated the classroom quite like the introduction of large language models (LLMs) in recent years. However, the research on student and teacher perceptions (Raine, 2023), LLM prompt manipulation techniques (Tseng & Lin, 2024), and suggested activities that utilize the programs far outweighs the amount of research being done on the assessment of the affect of AI on language learning production, of which results are mixed if not downright degenerative (Birdsell, 2022). As new technology emerges, rather than looking merely at the possibilities of the future, we must also be aware of the realities of the present.
| Title | A Luddite’s Assessment of AI-Assisted Writing |
|---|---|
| Teaching Context | College and university education |