17–19 May 2024
Meijo University Nagoya Dome Campus
Asia/Tokyo timezone

Applying Error Analysis Results of Sarcasm Detection: Humans vs NLP Models

18 May 2024, 13:00
1h 20m
Learning Commons (1st Floor)

Learning Commons (1st Floor)

Poster Research and Development Posters Session

Speaker

Oliver Cakebread-Andrews (Kwansei Gakuin University)

Description

Figurative language in general, and sarcasm in particular, is a difficult concept for learners of English to fully grasp and successfully use (Wallace, 2014; Kim and Lantolf, 2016). While correct understanding of sarcasm may not have significant application in day-to-day business, for example, it is nonetheless a regular part of many native speaker’s language usage (Angraini, 2017). Some of the many reasons that people use sarcasm include criticising, in a less-threatening manner (Claridge, 2011, Tracy, 2002), for stress-relief (Ducharme, 1994), in order to persuade others (Sopory and Dillary, 2002) and to improve relationships with others (Haiman, 1993). For non-native speakers (NNS) who intend to ingratiate themselves in the culture, as well as the language, of first-language-English communities, there are many benefits to understanding it (Genc and Bada, 2005; Kuo and Lai 2006). Despite maybe being consider the gold standard, native speakers also can struggle with accurate sarcasm detection (Abercrombie, 2016.

As it turns out, natural language processing (NLP) models also struggle to always detect whether utterances are sarcastic or not (Wallace, 2014). This study aimed to answer the following research questions:

What patterns can be found through error analysis of sarcasm detection between NLP models and native and non-native speakers of English?
What applications of these patterns are there for language education?

Three models (RoBERTa, DeBERTa and a logistic regression model) were used to analyse 300 comments from the FigLang 2020 Reddit Dataset, with and without context. The same 300 comments were also given to 39 NNS and 30 native speakers, and the results were compared. The key findings will be shown in this poster presentation.

Keywords Natural Language Processing, Error Analysis, Sentiment Analysis, Data-Driven Learning

Primary author

Oliver Cakebread-Andrews (Kwansei Gakuin University)

Presentation materials

There are no materials yet.