Speaker
Description
Sponsored by Duolingo
Conversational AI offers new ways to practice speaking, yet reliable assessment of learner progress in real-time dialogue remains a challenge. To address this, we developed an automated grader for Video Call with Lily, Duolingo’s conversation partner. Four expert raters scored over 800 learner responses using a CEFR-aligned rubric, achieving high agreement (r=.95). Human–machine scores closely aligned (r=.85), validating the system’s reliability and establishing a foundation for scalable, data-driven feedback in self-directed language learning.
Summary
Conversational AI offers new ways to practice speaking, yet reliable assessment of learner progress in real-time dialogue remains a challenge. To address this, we developed an automated grader for Video Call with Lily, Duolingo’s conversation partner. Four expert raters scored over 800 learner responses using a CEFR-aligned rubric, achieving high agreement (r=.95). Human–machine scores closely aligned (r=.85), validating the system’s reliability and establishing a foundation for scalable, data-driven feedback in self-directed language learning.
| Teaching Context | General |
|---|
