Hard No-Box Adversarial Attack on Skeleton-Based Human Action Recognition with Skeleton-Motion-Informed Gradient

Lu, Zhengzhi; Wang, He; Chang, Ziyi; Yang, Guoan; Shum, Hubert P.H.

Abstract

Recently, methods for skeleton-based human activity recognition have been shown to be vulnerable to adversarial attacks. However, these attack methods require either the full knowledge of the victim (i.e. white-box attacks), access to training data (i.e. transfer-based attacks) or frequent model queries (i.e. black-box attacks). All their requirements are highly restrictive, raising the question of how detrimental the vulnerability is. In this paper, we show that the vulnerability indeed exists. To this end, we consider a new attack task: the attacker has no access to the victim model or the training data or labels, where we coin the term hard no-box attack. Specifically, we first learn a motion manifold where we define an adversarial loss to compute a new gradient for the attack, named skeleton-motion informed (SMI) gradient. Our gradient contains information of the motion dynamics, which is different from existing gradient-based attack methods that compute the loss gradient assuming each dimension in the data is independent. The SMI gradient can augment many gradient-based attack methods, leading to a new family of no-box attack methods. Extensive evaluation and comparison show that our method imposes a real threat to existing classifiers. They also show that the SMI gradient improves the transferability and imperceptibility of adversarial samples in both no-box and transfer-based black-box settings.

Citation

Lu, Z., Wang, H., Chang, Z., Yang, G., & Shum, H. P. (2023, October). Hard No-Box Adversarial Attack on Skeleton-Based Human Action Recognition with Skeleton-Motion-Informed Gradient. Presented at ICCV 2023: 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France

• Presentation Conference Type: Conference Paper (published)
• Conference Name: ICCV 2023: 2023 IEEE/CVF International Conference on Computer Vision (ICCV)
• Start Date: Oct 4, 2023
• End Date: Oct 6, 2023
• Acceptance Date: Aug 10, 2023
• Publication Date: 2023
• Deposit Date: Aug 10, 2023
• Publicly Available Date: Dec 31, 2023
• Publisher: Institute of Electrical and Electronics Engineers
• Pages: 4574-4583
• DOI: https://doi.org/10.1109/ICCV51070.2023.00424
• Public URL: https://durham-repository.worktribe.com/output/1715085