Skip to main content

Research Repository

Advanced Search

Language as a latent sequence: Deep latent variable models for semi-supervised paraphrase generation

Yu, Jialin; Cristea, Alexandra I.; Harit, Anoushka; Sun, Zhongtian; Aduragba, Olanrewaju Tahir; Shi, Lei; Al Moubayed, Noura

Language as a latent sequence: Deep latent variable models for semi-supervised paraphrase generation Thumbnail


Authors

Jialin Yu jialin.yu@durham.ac.uk
Academic Visitor

Zhongtian Sun zhongtian.sun@durham.ac.uk
PGR Student Doctor of Philosophy

Lei Shi



Abstract

This paper explores deep latent variable models for semi-supervised paraphrase generation, where the missing target pair for unlabelled data is modelled as a latent paraphrase sequence. We present a novel unsupervised model named variational sequence auto-encoding reconstruction (VSAR), which performs latent sequence inference given an observed text. To leverage information from text pairs, we additionally introduce a novel supervised model we call dual directional learning (DDL), which is designed to integrate with our proposed VSAR model. Combining VSAR with DDL (DDL+VSAR) enables us to conduct semi-supervised learning. Still, the combined model suffers from a cold-start problem. To further combat this issue, we propose an improved weight initialisation solution, leading to a novel two-stage training scheme we call knowledge-reinforced-learning (KRL). Our empirical evaluations suggest that the combined model yields competitive performance against the state-ofthe-art supervised baselines on complete data. Furthermore, in scenarios where only a fraction of the labelled pairs are available, our combined model consistently outperforms the strong supervised model baseline (DDL) by a significant margin (𝑝 < .05; Wilcoxon test).

Citation

Yu, J., Cristea, A. I., Harit, A., Sun, Z., Aduragba, O. T., Shi, L., & Al Moubayed, N. (2023). Language as a latent sequence: Deep latent variable models for semi-supervised paraphrase generation. AI open, 4, 19-32. https://doi.org/10.1016/j.aiopen.2023.05.001

Journal Article Type Article
Acceptance Date May 18, 2023
Online Publication Date May 26, 2023
Publication Date 2023
Deposit Date May 30, 2023
Publicly Available Date May 30, 2023
Journal AI Open
Peer Reviewed Peer Reviewed
Volume 4
Pages 19-32
DOI https://doi.org/10.1016/j.aiopen.2023.05.001
Public URL https://durham-repository.worktribe.com/output/1172628

Files







You might also like



Downloadable Citations