SK_DU Team: Cross-Encoder based Evidence Retrieval and Question Generation with Improved Prompt for the AVeriTeC Shared Task

Malviya, Shrikant; Katsigiannis, Stamos

Abstract

As part of the AVeriTeC shared task, we developed a pipelined system comprising robust and finely tuned models. Our system integrates advanced techniques for evidence retrieval and question generation, leveraging cross-encoders and large language models (LLMs) for optimal performance. With multi-stage processing, the pipeline demonstrates improvements over baseline models, particularly in handling complex claims that require nuanced reasoning, by improved evidence extraction, question generation and veracity prediction. Through detailed experiments and ablation studies, we provide insights into the strengths and weaknesses of our approach, highlighting the critical role of evidence sufficiency and context dependency in automated fact-checking systems. Our system secured a competitive rank, 7th on the development and 12th on the test data, in the shared task, underscoring the effectiveness of our methods in addressing the challenges of real-world claim verification.

Citation

Malviya, S., & Katsigiannis, S. (2024, November). SK_DU Team: Cross-Encoder based Evidence Retrieval and Question Generation with Improved Prompt for the AVeriTeC Shared Task. Presented at 7th Fact Extraction and VERification Workshop (FEVER), Miami, Florida, USA

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 7th Fact Extraction and VERification Workshop (FEVER)
• Start Date: Nov 15, 2024
• Acceptance Date: Sep 30, 2024
• Deposit Date: Oct 7, 2024
• Peer Reviewed: Peer Reviewed
• Public URL: https://durham-repository.worktribe.com/output/2949286
• Publisher URL: https://fever.ai/workshop.html
• Additional Information: Workshop part of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP)