Speaker
Abstract section 4: Outcomes/results
All four detectors consistently identified AI generated texts with high accuracy; however, the free versions showed significant fairness issues when assessing human generated writing samples. These tools exhibited elevated false positive rates for ESL writing, illustrating inconsistent threshold behavior and algorithmic sensitivity to linguistic variation, a pattern also noted in recent comparative studies (Chaka, 2024; Liang et al., 2023). While misclassification patterns varied by device, the free versions clearly indicated inconsistency in assessments. These findings urge caution, as such tools can flag original ESL writing as AI generated, potentially influencing high stakes educational decisions (Woelfel, 2023).
Abstract section 3: Content/method
The research analyzed over 900 texts (400–500 words each) in three categories: AI generated, academic published, and ESL graduate student writing. The study assessed the texts using four commonly known AI detectors. Each tool’s binary threshold classifications were recorded and compared. Performance metrics and inferential statistics, including Chi square tests, were used to evaluate inter detector consistency, algorithmic behavior, and bias patterns, with particular attention to human-written text being misclassified (Weber Wulff et al., 2023).
Abstract section 2: Contribution/research questions
This study provides empirical evidence of threshold and algorithmic bias within four commercially available detectors commonly used by educators. The research question is: To what extent threshold settings and algorithmic features in AI-detection devices influence misclassification of ESL college student writing with false positives? The findings highlight risks in using such tools to assess ESL college writing, confirming previous research showing false positives and inequitable outcomes for multilingual writers (Dalalah & Dalalah, 2023; Giray, 2024).
Abstract section 1: Relevance
This study addresses present concerns in current theory and practice related to sociolinguistic justice frameworks and how educational institutes use technology for assessment. The study examines thresholds, algorithmic fairness, linguistic bias, and the reliability of AI text detection in education. Recent research shows that detectors often misinterpret linguistic variation, disproportionately affecting multilingual writers and reinforcing structural inequities (Liang et al., 2023). By empirically analyzing threshold behavior and algorithmic tendencies across major detectors, this study contributes to debates about responsible AI use, assessment integrity, and the need for transparent institutional policies (Dwyer & Laird, 2023).
Abstract section 5: References
References:
Chaka, C. (2024). Accuracy pecking order: How 30 AI detectors stack up in detecting generative artificial intelligence content in university English L1 and English L2 student essays. Journal of Applied Learning and Teaching, 7(1). https://doi.org/10.37074/jalt.2024.7.1.33
Dalalah, D. & Dalalah, OMA. (2023, July).The false positives and false negatives of generative AI detection tools in education and academic research: The case of ChatGPT, The International Journal of Management Education, 21(2). https://doi.org/10.1016/j.ijme.2023.100822
Dwyer, M, and Laird, E. (2023). Up in the air: Educators juggling the potential of generative AI with detection, discipline, and distrust. Center for Democracy and Technology. (2023, March).https://cdt.org/wp-content/uploads/2024/03/2024-03-21-CDT-Civic-Tech-Generative-AI-Survey-Research-final.pdf (Accessed 6 July 2025).
Giray, L. (2024). The problem with false positives: AI detection unfairly accuses scholars of AI plagiarism. Journal of Academic Integrity and Technology Ethics, 9(2), 134–147. https://www.researchgate.net/publication/386998010
Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023). GPT detectors are biased against non-native English writers. Patterns, 4(8). https://doi.org/10.1016/j.patter.2023.100779
Nordstokke, D. W., & Zumbo, B. D. (2007). A cautionary tale about Levene’s tests for equal variances. Journal of Educational Research and Policy Studies, 7(1): 1–14. https://files.eric.ed.gov/fulltext/EJ809430.pdf (Accessed 1 July 2025).
Walters, W. (2023). The Effectiveness of Software Designed to Detect AI-Generated Writing: A Comparison of 16 AI Text Detectors. Open Information Science, 7(1). https://doi.org/10.1515/opis-2022-0158
Woelfel, K. (2023, December 18). Late applications: Disproportionate effects of generative AI detectors on English learners [Policy brief]. Center for Democracy & Technology. https://cdt.org/insights/brief-late-applications-disproportionate-effects-of-generative-ai-detectors-on-english-learners/ (Accessed 16 July 2025).
| Title | AI‑Text Detection Fairness in ESL College Student Writing |
|---|---|
| Teaching Context | College and university education |