Speaker
Abstract section 2: Contribution/research questions
- Are there observable differences in atomic event data (e.g., input frequency) between independent and AI-assisted writing sessions?
- To what extent can higher-order drafting processes, such as planning and revision, be modeled from raw event logs?
Abstract section 5: References
Gánem-Gutiérrez, G. A., & Gilmore, A. (2018). Tracking the real-time evolution of a writing event: Second language writers at different proficiency levels. Language Learning, 68(2), 469–506. https://doi.org/10.1111/lang.12280
Kaliterna, M., Žuljević, M. F., Ursić, L., Jerković, I., & Tokalić, R. (2024). Testing the capacity of Bard and ChatGPT for writing essays on ethical dilemmas: A cross-sectional study. Scientific Reports, 14, Article 26046. https://doi.org/10.1038/s41598-024-77576-3
Abstract section 4: Outcomes/results
Preliminary analysis indicates that the framework successfully distinguishes between independent and AI-integrated workflows by mapping atomic event data to specific cognitive phases. Notable differences were observed in the distribution of temporal planning and the frequency of revision cycles. Specifically, event logs reveal distinct shifts in interactional latency and input burst during AI-mediated tasks, suggesting that micro-level interactions such as the timing between AI prompts and subsequent edits can effectively model macro-level procedural changes. These findings demonstrate the framework's capacity to categorize diverse learner engagement patterns and identify unique behavioral signatures in AI-assisted composition.
Abstract section 1: Relevance
While frameworks for tracking digital composition exist (Gánem-Gutiérrez & Gilmore, 2018), empirical data regarding AI-assisted drafting processes remains scarce. Moreover, current research on AI-assisted writing focuses on qualitative product differences rather than drafting mechanics (Kaliterna et al., 2024). This presentation introduces a study designed to address these gaps by utilizing a customized Moodle activity to capture real-time event data during independent and AI-assisted writing tasks. By providing a framework to interpret the "black box" of learner behavior, educators can better identify how students negotiate with generative tools when composing texts, ultimately informing the design of effective, integrity-focused writing interventions.
Abstract section 3: Content/method
Engineering students at a Japanese university completed two IELTS-modeled writing tasks on Moodle 3: one independent and one AI-assisted. As they worked, a customized JavaScript template captured real-time user inputs such as keystrokes and mouse clicks while restricting external copy-pasting and autocompletion. Sessions were monitored in person (3-hour limit; 300-word minimum), and participants completed a pre- and post-activity survey regarding AI awareness and perceived utility. Data was then collected and normalized for analysis.
| Title | Cracking the Black Box: Tracking AI-Assisted Writing |
|---|---|
| Teaching Context | College and university education |