Speaker
Abstract section 1: Relevance
As large language models become increasingly accessible to students, understanding differences between human and AI-generated creative writing has important implications for English education. Research in this field is still emerging, with some studies reporting no significant differences in creativity between human- and AI-generated stories (Orwig et al., 2024), and other studies finding that human participants could not distinguish between human- and AI-generated haiku (Hituswari et al., 2023). The present study extends this line of research by comparing texts written by EFL students and AI-generated texts produced in response to the same creative writing prompt, analyzing thematic differences and creativity.
Abstract section 4: Outcomes/results
Using reflective thematic analysis key distinguishing features related to the situation framing and writer perspective in the “message in the bottle” writings were identified. First, human writings tended to present more varied, random, and unpredictable situations, whereas genAI responses followed more patterned and overlapping scenarios. Second, regarding requests for the reader, human texts emphasized relationship-building and overcoming difficulties, while genAI texts were more abstract and focused on the environment (e.g., the beech). Creativity ratings, conducted by human and LLM judges blind to the study, indicated that human texts were statistically more creative.
Abstract section 2: Contribution/research questions
This study has two main research questions.
RQ1: What thematic patterns emerge from analyzing the human- and AI-generated responses to the creative prompt, and how do these patterns differ between groups?
RQ2: How do human judges and LLMs rate these writings for creativity?
Abstract section 5: References
Birdsell, B. J. (2025). Basic psychological needs and creativity in an EFL context. In B. Lacy, R. P. Lege, & M. Swanson (Eds.), Moving JALT Into the Future: Opportunity, Diversity, and Excellence. JALT. https://doi.org/10.37546/JALTPCP2024-35
Hitsuwari, J., Ueda, Y., Yun, W., & Nomura, M. (2023). Does human–AI collaboration lead to more creative art? Aesthetic evaluation of human-made and AI-generated haiku poetry. Computers in Human Behavior, 139, 107502.
Orwig, W., Edenbaum, E. R., Greene, J. D., & Schacter, D. L. (2024). The language of creativity: Evidence from humans and large language models. The Journal of Creative Behavior, 58(1), 128-136.
Abstract section 3: Content/method
Eighty-one EFL students first wrote a response to the prompt, “Write a message in a bottle,” and then used an LLM to generate a response to the same prompt. To address RQ1, reflective thematic analysis was conducted using NVivo to examine themes and identify key differences between conditions. To address RQ2, creativity was evaluated using the LLM-Consensual Assessment Technique (Birdsell, 2025). Because each participant produced both texts, a paired-samples t-test was conducted.
| Title | A Comparative Analysis of Student and LLM Creative Writing |
|---|---|
| Teaching Context | College and university education |