Speakers
KEYWORDS
computer assisted language learning
artificial intelligence
language assessment
ABSTRACT
The rise of generative artificial intelligence (GenAI) has enabled educators and researchers to efficiently create materials for language assessment (Xi, 2023). Initial research in this area has revealed some of the benefits and limitations of utilizing GenAI for language assessment development (Aryadoust et al., 2024; Shin & Lee, 2023). However, these early studies have focused on the assessment of second language (L2) English. This presentation reports on a study that seeks to address this gap in the literature by comparing Japanese Language Proficiency Test (JLPT) reading comprehension items created by human experts with those created by GenAI. To this end, two reading comprehension passages were selected from sample JLPT materials developed by the Japan Foundation and Japan Educational Exchanges and Services, the organizations responsible for the development and administration of the JLPT. Next, two reading passages and corresponding test items were created with ChatGPT 4.0 using a procedure outlined by Shin and Lee. Following this, a survey based on Shin and Lee was administered, which asked Japanese speakers to blindly evaluate the quality of the reading passages and test items. While data collection is ongoing, full results of the research and the study’s implications will be discussed during the presentation.
| TITLE | Comparing JLPT and ChatGPT-generated reading comprehension items |
|---|---|
| RELEVANT SIG | Computer Assisted Language Learning (CALL) |
| FORMAT | Research-oriented Oral Face-to-face presentation (25 minutes, including Q&A) |
