Skip to main content

Research Repository

Advanced Search

MuLD: The Multitask Long Document Benchmark

Hudson, G Thomas; Al Moubayed, Noura

MuLD: The Multitask Long Document Benchmark Thumbnail


Authors

George Hudson g.t.hudson@durham.ac.uk
Post Doctoral Research Associate



Contributors

Nicoletta Calzolari
Editor

Frederic Bechet
Editor

Philippe Blache
Editor

Khalid Choukri
Editor

Christopher Cieri
Editor

Thierry Declerck
Editor

Sara Goggi
Editor

Hitoshi Isahara
Editor

Bente Maegaard
Editor

Joseph Mariani
Editor

Helene Mazo
Editor

Jan Odijk
Editor

Stelios Piperidis
Editor

Abstract

The impressive progress in NLP techniques has been driven by the development of multi-task benchmarks such as GLUE and SuperGLUE. While these benchmarks focus on tasks for one or two input sentences, there has been exciting work in designing efficient techniques for processing much longer inputs. In this paper, we present MuLD: a new long document benchmark consisting of only documents over 10,000 tokens. By modifying existing NLP tasks, we create a diverse benchmark which requires models to successfully model long-term dependencies in the text. We evaluate how existing models perform, and find that our benchmark is much more challenging than their ‘short document’ equivalents. Furthermore, by evaluating both regular and efficient transformers, we show that models with increased context length are better able to solve the tasks presented, suggesting that future improvements in these models are vital for solving similar long document problems. We release the data and code for baselines to encourage further research on efficient NLP models.

Citation

Hudson, G. T., & Al Moubayed, N. (2022). MuLD: The Multitask Long Document Benchmark. In N. Calzolari, F. Bechet, P. Blache, K. Choukri, C. Cieri, T. Declerck, …S. Piperidis (Eds.),

Conference Name 13th Conference on Language Resources and Evaluation (LREC 2022)
Conference Location Marseille, France
Start Date Jun 20, 2022
End Date Jun 25, 2022
Acceptance Date Apr 4, 2022
Online Publication Date Jun 20, 2022
Publication Date 2022
Deposit Date Apr 26, 2022
Publicly Available Date Jun 26, 2022
Pages 3675-3685
Public URL https://durham-repository.worktribe.com/output/1138178
Publisher URL http://www.lrec-conf.org/proceedings/lrec2022/index.html
Related Public URLs https://arxiv.org/pdf/2202.07362.pdf

Files






You might also like



Downloadable Citations