Speaker
Description
Sponsored by Tsukuba University Graduate Student Showcase: This study evaluates the prompt design and AI-generated feedback used in an AI-mediated role-play system developed to support younger learners’ English speaking development. The system integrates role-play prompting, CEFR-aligned task design, and focused explicit feedback strategies to generate interactive speaking practice and individualized feedback. Experts and teachers evaluate the pedagogical quality, consistency, and appropriateness of both the prompt design and AI-generated feedback through questionnaires and interviews.
Abstract section 3: Content/method
An AI-mediated role-play system was developed using a Prompt Design Framework that integrates different prompting techniques, CEFR-aligned tasks, and explicit feedback strategies. Experts and teachers evaluate the prompt design and AI-generated feedback through questionnaires and interviews.
Abstract section 2: Contribution/research questions
This study aims to evaluate the usefulness, pedagogical appropriateness, and consistency of the Prompt Design Framework and AI-generated feedback used in an AI-mediated role-play system for younger learners' speaking development. The findings are expected to inform the refinement of the system prior to classroom implementation and contribute to the development of effective AI-assisted speaking support for younger language learners.
Abstract section 4: Outcomes/results
Initial evaluations indicate that the Prompt Design Framework shows promise in generating focused and explicit feedback that aligns with the intended speaking objectives. Early feedback from experts and teachers suggests generally positive perceptions regarding its usefulness, pedagogical appropriateness, and ability to generate consistent feedback. At the same time, several areas for refinement have been identified, including the need to balance the naturalness of AI-mediated interactions with the practical requirements of data collection and classroom implementation. These insights will inform ongoing revisions to the system prior to large-scale classroom implementation.
Abstract section 1: Relevance
Recent advances in generative AI have enabled the provision of automated speaking feedback. However, existing AI-generated feedback, often designed for adult learners, tends to be excessive in quantity, lacks pedagogical structure, and may not provide sufficiently actionable guidance for younger learners. This highlights the need for carefully designed prompting and feedback strategies that can generate focused, meaningful, and age-appropriate support for speaking development.
| Title | Evaluating the Prompt Design and AI-Generated Feedback |
|---|---|
| Teaching Context | Junior/senior high-school |