Skip to main content

Research Repository

Advanced Search

Focalized Contrastive View-invariant Learning for Self-supervised Skeleton-based Action Recognition

Men, Qianhui; Ho, Edmond S.L.; Shum, Hubert P.H.; Leung, Howard

Focalized Contrastive View-invariant Learning for Self-supervised Skeleton-based Action Recognition Thumbnail


Authors

Qianhui Men

Edmond S.L. Ho

Howard Leung



Abstract

Learning view-invariant representation is a key to improving feature discrimination power for skeleton-based action recognition. Existing approaches cannot effectively remove the impact of viewpoint due to the implicit view-dependent representations. In this work, we propose a self-supervised framework called Focalized Contrastive View-invariant Learning (FoCoViL), which significantly suppresses the view-specific information on the representation space where the viewpoints are coarsely aligned. By maximizing mutual information with an effective contrastive loss between multi-view sample pairs, FoCoViL associates actions with common view-invariant properties and simultaneously separates the dissimilar ones. We further propose an adaptive focalization method based on pairwise similarity to enhance contrastive learning for a clearer cluster boundary in the learned space. Different from many existing self-supervised representation learning work that rely heavily on supervised classifiers, FoCoViL performs well on both unsupervised and supervised cla ssifiers with superior recognition performance. Extensive experiments also show that the proposed contrastive-based focalization generates a more discriminative latent representation.

Citation

Men, Q., Ho, E. S., Shum, H. P., & Leung, H. (2023). Focalized Contrastive View-invariant Learning for Self-supervised Skeleton-based Action Recognition. Neurocomputing, 537, 198-209. https://doi.org/10.1016/j.neucom.2023.03.070

Journal Article Type Article
Acceptance Date Mar 28, 2023
Online Publication Date Mar 31, 2023
Publication Date Jun 7, 2023
Deposit Date Apr 3, 2023
Publicly Available Date Apr 1, 2024
Journal Neurocomputing
Print ISSN 0925-2312
Electronic ISSN 1872-8286
Publisher Elsevier
Peer Reviewed Peer Reviewed
Volume 537
Pages 198-209
DOI https://doi.org/10.1016/j.neucom.2023.03.070
Public URL https://durham-repository.worktribe.com/output/1176534

Files






You might also like



Downloadable Citations