R. M. Sales
Enhancing lecture capture with deep learning
Sales, R. M.; Giani, S.
Abstract
This paper provides an insight into the development of a state-of-the-art video processing system to address limitations within Durham University’s ‘Encore’ lecture capture solution. The aim of the research described in this paper is to digitally remove the persons presenting from the view of a whiteboard to provide students with a more effective online learning experience. This work enlists a ‘human entity detection module’, which uses a remodelled version of the Fast Segmentation Neural Network to perform efficient binary image segmentation, and a ‘background restoration module’, which introduces a novel procedure to retain only background pixels in consecutive video frames. The segmentation network is trained from the outset with a Tversky loss function on a dataset of images extracted from various Tik-Tok dance videos. The most effective training techniques are described in detail, and it is found that these produce asymptotic convergence to within 5% of the final loss in only 40 training epochs. A cross-validation study then concludes that a Tversky parameter of 0.9 is optimal for balancing recall and precision in the context of this work. Finally, it is demonstrated that the system successfully removes the human form from the view of the whiteboard in a real lecture video. Whilst the system is believed to have the potential for real-time usage, it is not possible to prove this owing to hardware limitations. In the conclusions, wider application of this work is also suggested.
Citation
Sales, R. M., & Giani, S. (2024). Enhancing lecture capture with deep learning. Advances in Engineering Software, 196, Article 103732. https://doi.org/10.1016/j.advengsoft.2024.103732
Journal Article Type | Article |
---|---|
Acceptance Date | Jul 8, 2024 |
Online Publication Date | Jul 29, 2024 |
Publication Date | 2024-10 |
Deposit Date | Sep 13, 2022 |
Publicly Available Date | Jul 31, 2024 |
Journal | Advances in Engineering Software |
Print ISSN | 0965-9978 |
Publisher | Elsevier |
Peer Reviewed | Peer Reviewed |
Volume | 196 |
Article Number | 103732 |
DOI | https://doi.org/10.1016/j.advengsoft.2024.103732 |
Public URL | https://durham-repository.worktribe.com/output/1191638 |
Files
Published Journal Article
(2.5 Mb)
PDF
Publisher Licence URL
http://creativecommons.org/licenses/by/4.0/
You might also like
An hp-adaptive discontinuous Galerkin method for phase field fracture
(2023)
Journal Article
Convolutional neural network framework for wind turbine electromechanical fault detection
(2023)
Journal Article
On Effects of Concentrated Loads on Perforated Sensitive Shells of Revolution
(2023)
Journal Article
Downloadable Citations
About Durham Research Online (DRO)
Administrator e-mail: dro.admin@durham.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2025
Advanced Search