Rank over Class: The Untapped Potential of Ranking in Natural Language Processing

Atapour-Abarghouei, Amir; Bonner, Stephen; McGough, Andrew Stephen

Abstract

Text classification has long been a staple within Natural Language Processing (NLP) with applications spanning across diverse areas such as sentiment analysis, recommender systems and spam detection. With such a powerful solution, it is often tempting to use it as the go-to tool for all NLP problems since when you are holding a hammer, everything looks like a nail. However, we argue here that many tasks which are currently addressed using classification are in fact being shoehorned into a classification mould and that if we instead address them as a ranking problem, we not only improve the model, but we achieve better performance. We propose a novel end-to-end ranking approach consisting of a Transformer network responsible for producing representations for a pair of text sequences, which are in turn passed into a context aggregating network outputting ranking scores used to determine an ordering to the sequences based on some notion of relevance. We perform numerous experiments on publicly-available datasets and investigate the applications of ranking in problems often solved using classification. In an experiment on a heavily- skewed sentiment analysis dataset, converting ranking results to classification labels yields an approximately 22% improvement over state-of-the-art text classification, demonstrating the efficacy of text ranking over text classification in certain scenarios.

Citation

Atapour-Abarghouei, A., Bonner, S., & McGough, A. S. (2021, December). Rank over Class: The Untapped Potential of Ranking in Natural Language Processing. Presented at 2021 IEEE International Conference on Big Data (IEEE BigData 2021), Orlando, FL, USA

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 2021 IEEE International Conference on Big Data (IEEE BigData 2021)
• Start Date: Dec 15, 2021
• End Date: Dec 18, 2021
• Acceptance Date: Nov 14, 2021
• Publication Date: Dec 15, 2021
• Deposit Date: Dec 3, 2021
• Publicly Available Date: Dec 6, 2021
• Publisher: Institute of Electrical and Electronics Engineers
• ISBN: 9781665445993
• DOI: https://doi.org/10.1109/bigdata52589.2021.9671386
• Public URL: https://durham-repository.worktribe.com/output/1138825
• Related Public URLs: https://arxiv.org/pdf/2009.05160.pdf