Does BERT pay attention to cyberbullying?

Elsafoury, Fatma; Katsigiannis, Stamos; Wilson, Steven; Ramzan, Naeem

Abstract

Social media have brought threats like cyberbullying, which can lead to stress, anxiety, depression and in some severe cases, suicide attempts. Detecting cyberbullying can help to warn/ block bullies and provide support to victims. However, very few studies have used self-attention-based language models like BERT for cyberbullying detection and they typically only report BERT’s performance without examining in depth the reasons for its performance. In this work, we examine the use of BERT for cyberbullying detection on various datasets and attempt to explain its performance by analysing its attention weights and gradient-based feature importance scores for textual and linguistic features. Our results show that attention weights do not correlate with feature importance scores and thus do not explain the model’s performance. Additionally, they suggest that BERT relies on syntactical biases in the datasets to assign feature importance scores to class-related words rather than cyberbullying-related linguistic features.

Citation

Elsafoury, F., Katsigiannis, S., Wilson, S., & Ramzan, N. (2021, July). Does BERT pay attention to cyberbullying?. Presented at 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Online

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 44th International ACM SIGIR Conference on Research and Development in Information Retrieval
• Start Date: Jul 11, 2021
• End Date: Jul 15, 2021
• Acceptance Date: Apr 15, 2021
• Online Publication Date: Jul 11, 2021
• Publication Date: 2021-07
• Deposit Date: Apr 15, 2021
• Publicly Available Date: Jul 16, 2021
• Pages: 1900-1904
• DOI: https://doi.org/10.1145/3404835.3463029
• Public URL: https://durham-repository.worktribe.com/output/1139585