Integrating Speech Input in Educational Immersive Virtual Reality Applications: A Systematic Review

Alghamdi, Nuha; Cristea, Alexandra I.

Abstract

The topic of immersive virtual reality (IVR) in education has gained increasing attention in recent years, due to its potential to enhance learner outcomes and to mitigate learning costs. As we can capture a multitude of information from speech and generate valuable information from it, there has been an interest in exploring this source of data in such environments. Additionally, speech is being used in new and different ways in such environments. However, its specific usage in IVR-based education has not been reviewed yet. Thus, this systematic review seeks to examine, for the first time, the current state of research, specifically on using speech input - and, related to this, Natural Language Processing (NLP) - in educational IVR environments. We conducted a comprehensive search of the popular Web of Science and Scopus databases, to identify relevant papers. To properly reflect the state-of-the-art, English peer-reviewed articles published in the last 5 years (between 2020 - 2024), were included in the review, based on keywords search. 595 articles were identified and processed via the established PRISMA procedure, to exclude all duplicate or irrelevant papers, rendering 23 articles as relevant. For these, we identified the target educational subjects and the purpose of using speech as a data source. We also investigated speech recognition models and NLP models used. This systematic review provides evidence supporting the use of speech input as a valuable data source in educational IVR applications. We also propose the first, to the best of our knowledge, taxonomy for speech and NLP for IVR in education, as well as identify potential further research directions, all of which can help researchers and educators, when considering incorporating speech and NLP into educational IVR applications, to enrich the teaching and learning experience.

Citation

Alghamdi, N., & Cristea, A. I. (2024, August). Integrating Speech Input in Educational Immersive Virtual Reality Applications: A Systematic Review. Presented at 2024 IEEE 12th International Conference on Intelligent Systems (IS), Varna, Bulgaria

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 2024 IEEE 12th International Conference on Intelligent Systems (IS)
• Start Date: Aug 29, 2024
• End Date: Aug 31, 2024
• Acceptance Date: Jun 30, 2024
• Publication Date: Oct 9, 2024
• Deposit Date: Oct 16, 2024
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• Volume: 22
• Pages: 1-8
• Series ISSN: 2832-4145
• Book Title: 2024 IEEE 12th International Conference on Intelligent Systems (IS)
• DOI: https://doi.org/10.1109/is61756.2024.10705165
• Public URL: https://durham-repository.worktribe.com/output/2960802