Skip to main content

Research Repository

Advanced Search

Does BERT pay attention to cyberbullying?

Elsafoury, Fatma; Katsigiannis, Stamos; Wilson, Steven; Ramzan, Naeem

Does BERT pay attention to cyberbullying? Thumbnail


Fatma Elsafoury

Steven Wilson

Naeem Ramzan


Social media have brought threats like cyberbullying, which can lead to stress, anxiety, depression and in some severe cases, suicide attempts. Detecting cyberbullying can help to warn/ block bullies and provide support to victims. However, very few studies have used self-attention-based language models like BERT for cyberbullying detection and they typically only report BERT’s performance without examining in depth the reasons for its performance. In this work, we examine the use of BERT for cyberbullying detection on various datasets and attempt to explain its performance by analysing its attention weights and gradient-based feature importance scores for textual and linguistic features. Our results show that attention weights do not correlate with feature importance scores and thus do not explain the model’s performance. Additionally, they suggest that BERT relies on syntactical biases in the datasets to assign feature importance scores to class-related words rather than cyberbullying-related linguistic features.


Elsafoury, F., Katsigiannis, S., Wilson, S., & Ramzan, N. (2021). Does BERT pay attention to cyberbullying?. .

Conference Name 44th International ACM SIGIR Conference on Research and Development in Information Retrieval
Conference Location Online
Start Date Jul 11, 2021
End Date Jul 15, 2021
Acceptance Date Apr 15, 2021
Online Publication Date Jul 11, 2021
Publication Date 2021-07
Deposit Date Apr 15, 2021
Publicly Available Date Jul 16, 2021
Pages 1900-1904


You might also like

Downloadable Citations