Re-ID-AR: Improved Person Re-identification in Video via Joint Weakly Supervised Action Recognition
Alsehaim, A.; Breckon, T.P.
We uniquely consider the task of joint person re-identification (Re-ID) and action recognition in video as a multi-task problem. In addition to the broader potential of joint Re-ID and action recognition within the context of automated multi-camera surveillance, we show that the consideration of action recognition in addition to Re-ID results in a model that learns discriminative feature representations that both improve Re-ID performance and are capable of providing viable per-view (clip-wise) action recognition. Our approach uses a single 2D Convolutional Neural Network (CNN) architecture comprising a common ResNet50-IBN backbone CNN architecture, to extract frame-level features with subsequent temporal attention for clip level feature extraction, followed by two sub-branches:- the IDentification (sub-)Network (IDN) for person Re-ID and the Action Recognition (sub-)Network for per-view action recognition. The IDN comprises a single fully connected layer while the ARN comprises multiple attention blocks on a one-to-one ratio with the number of actions to be recognised. This is subsequently trained as a joint Re-ID and action recognition task using a combination of two task-specific, multi-loss terms via weakly labelled actions obtained over two leading benchmark Re-ID datasets (MARS, LPW). Our consideration of Re-ID and action recognition as a multi-task problem results in a multi-branch 2D CNN architecture that outperforms prior work in the field (rank-1 (mAP) – MARS: 93.21%(87.23%), LPW: 79.60%) without any reliance 3D convolutions or multi-stream networks architectures as found in other contemporary work. Our work represents the first benchmark performance for such a joint Re-ID and action recognition video understanding task, hitherto unapproached in the literature, and is accompanied by a new public dataset of supplementary action labels for the seminal MARS and LPW Re-ID datasets.
Alsehaim, A., & Breckon, T. (2021). Re-ID-AR: Improved Person Re-identification in Video via Joint Weakly Supervised Action Recognition.
|Conference Name||BMVC 2021|
|Start Date||Nov 22, 2021|
|End Date||Nov 25, 2021|
|Deposit Date||Oct 26, 2021|
|Publicly Available Date||Oct 26, 2021|
|Series Title||The British Machine Vision Conference (BMVC)|
Accepted Conference Proceeding
You might also like
Robust Semi-Supervised Anomaly Detection via Adversarially Learned Continuous Noise Corruption
U3DS3 : Unsupervised 3D Semantic Scene Segmentation
ACR: Attention Collaboration-based Regressor for Arbitrary Two-Hand Reconstruction
Less is More: Reducing Task and Model Complexity for 3D Point Cloud Semantic Segmentation