MAGR: Manifold-Aligned Graph Regularization for Continual Action Quality Assessment

Zhou, Kanglei; Wang, Liyuan; Zhang, Xingxing; Shum, Hubert P.H.; Li, Frederick W. B.; Li, Jianguo; Liang, Xiaohui

Abstract

Action Quality Assessment (AQA) evaluates diverse skills but models struggle with non-stationary data. We propose Continual AQA (CAQA) to refine models using sparse new data. Feature replay preserves memory without storing raw inputs. However, the misalignment between static old features and the dynamically changing feature manifold causes severe catastrophic forgetting. To address this novel problem, we propose Manifold-Aligned Graph Regularization (MAGR), which first aligns deviated old features to the current feature manifold, ensuring representation consistency. It then constructs a graph jointly arranging old and new features aligned with quality scores. Experiments show MAGR outperforms recent strong baselines with up to 6.56%, 5.66%, 15.64%, and 9.05% correlation gains on the MTL-AQA, FineDiving, UNLV-Dive, and JDM-MSA split datasets, respectively. This validates MAGR for continual assessment challenges arising from non-stationary skill variations. Code is available at https://github.com/ZhouKanglei/MAGR_CAQA.

Citation

Zhou, K., Wang, L., Zhang, X., Shum, H. P., Li, F. W. B., Li, J., & Liang, X. (2024, September). MAGR: Manifold-Aligned Graph Regularization for Continual Action Quality Assessment. Presented at Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Milan, Italy

• Presentation Conference Type: Conference Paper (published)
• Conference Name: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
• Start Date: Sep 29, 2024
• End Date: Oct 4, 2024
• Acceptance Date: Jul 6, 2024
• Online Publication Date: Nov 1, 2024
• Publication Date: Jan 1, 2025
• Deposit Date: Aug 5, 2024
• Publicly Available Date: Nov 1, 2024
• Print ISSN: 0302-9743
• Publisher: Springer
• Peer Reviewed: Peer Reviewed
• Volume: 15069 LNCS
• Pages: 375-392
• Series Title: Lecture Notes in Computer Science
• Series Number: 15069
• Series ISSN: 0302-9743
• Book Title: Computer Vision – ECCV 2024
• DOI: https://doi.org/10.1007/978-3-031-73247-8_22
• Public URL: https://durham-repository.worktribe.com/output/2740857