R. M. Sales
Enhancing lecture capture with deep learning
Sales, R. M.; Giani, S.
Abstract
This paper provides an insight into the development of a state-of-the-art video processing system to address limitations within Durham University’s ‘Encore’ lecture capture solution. The aim of the research described in this paper is to digitally remove the persons presenting from the view of a whiteboard to provide students with a more effective online learning experience. This work enlists a ‘human entity detection module’, which uses a remodelled version of the Fast Segmentation Neural Network to perform efficient binary image segmentation, and a ‘background restoration module’, which introduces a novel procedure to retain only background pixels in consecutive video frames. The segmentation network is trained from the outset with a Tversky loss function on a dataset of images extracted from various Tik-Tok dance videos. The most effective training techniques are described in detail, and it is found that these produce asymptotic convergence to within 5% of the final loss in only 40 training epochs. A cross-validation study then concludes that a Tversky parameter of 0.9 is optimal for balancing recall and precision in the context of this work. Finally, it is demonstrated that the system successfully removes the human form from the view of the whiteboard in a real lecture video. Whilst the system is believed to have the potential for real-time usage, it is not possible to prove this owing to hardware limitations. In the conclusions, wider application of this work is also suggested.
Citation
Sales, R. M., & Giani, S. (2024). Enhancing lecture capture with deep learning. Advances in Engineering Software, 196, Article 103732. https://doi.org/10.1016/j.advengsoft.2024.103732
Journal Article Type | Article |
---|---|
Acceptance Date | Jul 8, 2024 |
Online Publication Date | Jul 29, 2024 |
Publication Date | 2024-10 |
Deposit Date | Sep 13, 2022 |
Publicly Available Date | Jul 31, 2024 |
Journal | Advances in Engineering Software |
Print ISSN | 0965-9978 |
Publisher | Elsevier |
Peer Reviewed | Peer Reviewed |
Volume | 196 |
Article Number | 103732 |
DOI | https://doi.org/10.1016/j.advengsoft.2024.103732 |
Public URL | https://durham-repository.worktribe.com/output/1191638 |
Files
Published Journal Article
(2.5 Mb)
PDF
Publisher Licence URL
http://creativecommons.org/licenses/by/4.0/
You might also like
UKACM Proceedings 2024
(2024)
Presentation / Conference Contribution
Modelling Fracture Behaviour in Fibre-Hybrid 3D Woven Composites
(2024)
Presentation / Conference Contribution
Immersed traction boundary conditions in phase field fracture modelling
(2024)
Presentation / Conference Contribution
Recursive autoencoder network for prediction of CAD model parameters from STEP files
(2024)
Presentation / Conference Contribution
An investigation into the methods for modelling pre-existing cracks in phase field problems
(2023)
Presentation / Conference Contribution
Downloadable Citations
About Durham Research Online (DRO)
Administrator e-mail: dro.admin@durham.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search