PHI: Bridging Domain Shift in Long-Term Action Quality Assessment via Progressive Hierarchical Instruction

Zhou, Kanglei; Shum, Hubert P. H.; Li, Frederick W. B.; Zhang, Xingxing; Liang, Xiaohui

Abstract

Long-term Action Quality Assessment (AQA) aims to evaluate the quantitative performance of actions in long videos. However, existing methods face challenges due to domain shifts between the pre-trained large-scale action recognition backbones and the specific AQA task, thereby hindering their performance. This arises since fine-tuning resource-intensive backbones on small AQA datasets is impractical. We address this by identifying two levels of domain shift: task-level, regarding differences in task objectives, and feature-level, regarding differences in important features. For feature-level shifts, which are more detrimental, we propose Progressive Hierarchical Instruction (PHI) with two strategies. First, Gap Minimization Flow (GMF) leverages flow matching to progressively learn a fast flow path that reduces the domain gap between initial and desired features across shallow to deep layers. Additionally, a temporally-enhanced attention module captures long-range dependencies essential for AQA. Second, List-wise Contrastive Regularization (LCR) facilitates coarse-to-fine alignment by comprehensively comparing batch pairs to learn fine-grained cues while mitigating domain shift. Integrating these modules, PHI offers an effective solution. Experiments demonstrate that PHI achieves state-of-the-art performance on three representative long-term AQA datasets, proving its superiority in addressing the domain shift for long-term AQA.

Citation

Zhou, K., Shum, H. P. H., Li, F. W. B., Zhang, X., & Liang, X. (2025). PHI: Bridging Domain Shift in Long-Term Action Quality Assessment via Progressive Hierarchical Instruction. IEEE Transactions on Image Processing, https://doi.org/10.1109/TIP.2025.3574938

• Journal Article Type: Article
• Acceptance Date: Mar 9, 2025
• Online Publication Date: Jun 4, 2025
• Publication Date: 2025
• Deposit Date: May 27, 2025
• Publicly Available Date: Jul 1, 2025
• Journal: IEEE Transactions on Image Processing
• Print ISSN: 1057-7149
• Electronic ISSN: 1941-0042
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• DOI: https://doi.org/10.1109/TIP.2025.3574938
• Public URL: https://durham-repository.worktribe.com/output/3964025
• Publisher URL: https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=83