Professor Hubert Shum's Outputs (95)

A Virtual Reality Framework for Human-Driver Interaction Research: Safe and Cost-Effective Data Collection (2024)
Presentation / Conference Contribution
Crosato, L., Wei, C., Ho, E. S. L., Shum, H. P. H., & Sun, Y. (2024, March). A Virtual Reality Framework for Human-Driver Interaction Research: Safe and Cost-Effective Data Collection. Presented at 2024 ACM/IEEE International Conference on Human Robot Interaction (HRI '24), Boulder, CO, USA

The advancement of automated driving technology has led to new challenges in the interaction between automated vehicles and human road users. However, there is currently no complete theory that explains how human road users interact with vehicles, an... Read More about A Virtual Reality Framework for Human-Driver Interaction Research: Safe and Cost-Effective Data Collection.

HINT: High-quality INpainting Transformer with Mask-Aware Encoding and Enhanced Attention (2024)
Journal Article
Chen, S., Atapour-Abarghouei, A., & Shum, H. P. H. (2024). HINT: High-quality INpainting Transformer with Mask-Aware Encoding and Enhanced Attention. IEEE Transactions on Multimedia, 26, 7649-7660. https://doi.org/10.1109/TMM.2024.3369897

Existing image inpainting methods leverage convolution-based downsampling approaches to reduce spatial dimensions. This may result in information loss from corrupted images where the available information is inherently sparse, especially for the scen... Read More about HINT: High-quality INpainting Transformer with Mask-Aware Encoding and Enhanced Attention.

Enhancing surgical performance in cardiothoracic surgery with innovations from computer vision and artificial intelligence: a narrative review (2024)
Journal Article
Constable, M. D., Shum, H. P. H., & Clark, S. (2024). Enhancing surgical performance in cardiothoracic surgery with innovations from computer vision and artificial intelligence: a narrative review. Journal of Cardiothoracic Surgery, 19(1), Article 94. https://doi.org/10.1186/s13019-024-02558-5

When technical requirements are high, and patient outcomes are critical, opportunities for monitoring and improving surgical skills via objective motion analysis feedback may be particularly beneficial. This narrative review synthesises work on techn... Read More about Enhancing surgical performance in cardiothoracic surgery with innovations from computer vision and artificial intelligence: a narrative review.

Pose-based tremor type and level analysis for Parkinson’s disease from video (2024)
Journal Article
Zhang, H., Ho, E. S. L., Zhang, X., Del Din, S., & Shum, H. P. H. (2024). Pose-based tremor type and level analysis for Parkinson’s disease from video. International Journal of Computer Assisted Radiology and Surgery, 19(5), 831-840. https://doi.org/10.1007/s11548-023-03052-4

Current methods for diagnosis of PD rely on clinical examination. The accuracy of diagnosis ranges between 73 and 84%, and is influenced by the experience of the clinical assessor. Hence, an automatic, effective and interpretable supporting system fo... Read More about Pose-based tremor type and level analysis for Parkinson’s disease from video.

Hard No-Box Adversarial Attack on Skeleton-Based Human Action Recognition with Skeleton-Motion-Informed Gradient (2023)
Presentation / Conference Contribution
Lu, Z., Wang, H., Chang, Z., Yang, G., & Shum, H. P. (2023, October). Hard No-Box Adversarial Attack on Skeleton-Based Human Action Recognition with Skeleton-Motion-Informed Gradient. Presented at ICCV 2023: 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France

Recently, methods for skeleton-based human activity recognition have been shown to be vulnerable to adversarial attacks. However, these attack methods require either the full knowledge of the victim (i.e. white-box attacks), access to training data (... Read More about Hard No-Box Adversarial Attack on Skeleton-Based Human Action Recognition with Skeleton-Motion-Informed Gradient.

Tackling Data Bias in Painting Classification with Style Transfer (2023)
Presentation / Conference Contribution
Vijendran, M., Li, F. W., & Shum, H. P. (2023, February). Tackling Data Bias in Painting Classification with Style Transfer. Presented at VISAPP '23: 2023 International Conference on Computer Vision Theory and Applications, Lisbon, Portugal

It is difficult to train classifiers on paintings collections due to model bias from domain gaps and data bias from the uneven distribution of artistic styles. Previous techniques like data distillation, traditional data augmentation and style transf... Read More about Tackling Data Bias in Painting Classification with Style Transfer.

Unaligned 2D to 3D Translation with Conditional Vector-Quantized Code Diffusion using Transformers (2023)
Presentation / Conference Contribution
Corona-Figueroa, A., Bond-Taylor, S., Bhowmik, N., Gaus, Y. F. A., Breckon, T. P., Shum, H. P., & Willcocks, C. G. (2023, October). Unaligned 2D to 3D Translation with Conditional Vector-Quantized Code Diffusion using Transformers. Presented at ICCV23: 2023 IEEE/CVF International Conference on Computer Vision, Paris, France

Generating 3D images of complex objects conditionally from a few 2D views is a difficult synthesis problem, compounded by issues such as domain gap and geometric misalignment. For instance, a unified framework such as Generative Adversarial Networks... Read More about Unaligned 2D to 3D Translation with Conditional Vector-Quantized Code Diffusion using Transformers.

Unifying Human Motion Synthesis and Style Transfer with Denoising Diffusion Probabilistic Models (2023)
Presentation / Conference Contribution
Chang, Z., Findlay, E. J., Zhang, H., & Shum, H. P. (2023, February). Unifying Human Motion Synthesis and Style Transfer with Denoising Diffusion Probabilistic Models. Presented at GRAPP 2023: 2023 International Conference on Computer Graphics Theory and Applications, Lisbon, Portugal

Generating realistic motions for digital humans is a core but challenging part of computer animations and games, as human motions are both diverse in content and rich in styles. While the latest deep learning approaches have made significant advancem... Read More about Unifying Human Motion Synthesis and Style Transfer with Denoising Diffusion Probabilistic Models.

A Mixed Reality Training System for Hand-Object Interaction in Simulated Microgravity Environments (2023)
Presentation / Conference Contribution
Zhou, K., Chen, C., Ma, Y., Leng, Z., Shum, H. P., Li, F. W., & Liang, X. (2023, October). A Mixed Reality Training System for Hand-Object Interaction in Simulated Microgravity Environments. Presented at ISMAR 23: International Symposium on Mixed and Augmented Reality, Sydney, Australia

As human exploration of space continues to progress, the use of Mixed Reality (MR) for simulating microgravity environments and facilitating training in hand-object interaction holds immense practical significance. However, hand-object interaction in... Read More about A Mixed Reality Training System for Hand-Object Interaction in Simulated Microgravity Environments.

Enhancing Perception and Immersion in Pre-Captured Environments through Learning-Based Eye Height Adaptation (2023)
Presentation / Conference Contribution
Feng, Q., Shum, H. P., & Morishima, S. (2023, October). Enhancing Perception and Immersion in Pre-Captured Environments through Learning-Based Eye Height Adaptation. Presented at ISMAR 23: International Symposium on Mixed and Augmented Reality, Sydney, Australia

Pre-captured immersive environments using omnidirectional cameras provide a wide range of virtual reality applications. Previous research has shown that manipulating the eye height in egocentric virtual environments can significantly affect distance... Read More about Enhancing Perception and Immersion in Pre-Captured Environments through Learning-Based Eye Height Adaptation.

Social Interaction‐Aware Dynamical Models and Decision‐Making for Autonomous Vehicles (2023)
Journal Article
Crosato, L., Tian, K., Shum, H. P., Ho, E. S., Wang, Y., & Wei, C. (2023). Social Interaction‐Aware Dynamical Models and Decision‐Making for Autonomous Vehicles. Advanced Intelligent Systems, 6(3), Article 2300575. https://doi.org/10.1002/aisy.202300575

Interaction‐aware autonomous driving (IAAD) is a rapidly growing field of research that focuses on the development of autonomous vehicles (AVs) that are capable of interacting safely and efficiently with human road users. This is a challenging task,... Read More about Social Interaction‐Aware Dynamical Models and Decision‐Making for Autonomous Vehicles.

Multi-Task Spatial-Temporal Graph Auto-Encoder for Hand Motion Denoising (2023)
Journal Article
Zhou, K., Shum, H. P., Li, F. W., & Liang, X. (2023). Multi-Task Spatial-Temporal Graph Auto-Encoder for Hand Motion Denoising. IEEE Transactions on Visualization and Computer Graphics, https://doi.org/10.1109/TVCG.2023.3337868

In many human-computer interaction applications, fast and accurate hand tracking is necessary for an immersive experience. However, raw hand motion data can be flawed due to issues such as joint occlusions and high-frequency noise, hindering the inte... Read More about Multi-Task Spatial-Temporal Graph Auto-Encoder for Hand Motion Denoising.

Correlation-Distance Graph Learning for Treatment Response Prediction from rs-fMRI (2023)
Presentation / Conference Contribution
Zhang, X., Zheng, S., Shum, H. P., Zhang, H., Song, N., Song, M., & Jia, H. (2023, November). Correlation-Distance Graph Learning for Treatment Response Prediction from rs-fMRI. Presented at ICONIP 2023: 2023 International Conference on Neural Information Processing, Changsha, China

Resting-state fMRI (rs-fMRI) functional connectivity (FC)
analysis provides valuable insights into the relationships between different brain regions and their potential implications for neurological or psychiatric disorders. However, specific design... Read More about Correlation-Distance Graph Learning for Treatment Response Prediction from rs-fMRI.

Less is More: Reducing Task and Model Complexity for 3D Point Cloud Semantic Segmentation (2023)
Presentation / Conference Contribution
Li, L., Shum, H. P., & Breckon, T. P. (2023, June). Less is More: Reducing Task and Model Complexity for 3D Point Cloud Semantic Segmentation. Presented at 2023 IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR), Vancouver, BC

Whilst the availability of 3D LiDAR point cloud data has significantly grown in recent years, annotation remains expensive and time-consuming, leading to a demand for semisupervised semantic segmentation methods with application domains such as auton... Read More about Less is More: Reducing Task and Model Complexity for 3D Point Cloud Semantic Segmentation.

Region-based Appearance and Flow Characteristics for Anomaly Detection in Infrared Surveillance Imagery (2023)
Presentation / Conference Contribution
Gaus, Y., Bhowmik, N., Issac-Medina, B., Atapour-Abarghouei, A., Shum, H., & Breckon, T. (2023, June). Region-based Appearance and Flow Characteristics for Anomaly Detection in Infrared Surveillance Imagery. Presented at IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023, Vancouver, BC

Anomaly detection is a classical problem within automated visual surveillance, namely the determination of the normal from the abnormal when operational data availability is highly biased towards one class (normal) due to both insufficient sample siz... Read More about Region-based Appearance and Flow Characteristics for Anomaly Detection in Infrared Surveillance Imagery.

Hierarchical Graph Convolutional Networks for Action Quality Assessment (2023)
Journal Article
Zhou, K., Ma, Y., Shum, H. P., & Liang, X. (online). Hierarchical Graph Convolutional Networks for Action Quality Assessment. IEEE Transactions on Circuits and Systems for Video Technology, 33(12), 7749 - 7763. https://doi.org/10.1109/TCSVT.2023.3281413

Action quality assessment (AQA) automatically evaluates how well humans perform actions in a given video, a technique widely used in fields such as rehabilitation medicine, athletic competitions, and specific skills assessment. However, existing work... Read More about Hierarchical Graph Convolutional Networks for Action Quality Assessment.

INCLG: Inpainting for Non-Cleft Lip Generation with a Multi-Task Image Processing Network (2023)
Journal Article
Chen, S., Atapour-Abarghouei, A., Ho, E. S., & Shum, H. P. (2023). INCLG: Inpainting for Non-Cleft Lip Generation with a Multi-Task Image Processing Network. Software impacts, 17, Article 100517. https://doi.org/10.1016/j.simpa.2023.100517

We present a software that predicts non-cleft facial images for patients with cleft lip, thereby facilitating the understanding, awareness and discussion of cleft lip surgeries. To protect patients’ privacy, we design a software framework using image... Read More about INCLG: Inpainting for Non-Cleft Lip Generation with a Multi-Task Image Processing Network.

Focalized Contrastive View-invariant Learning for Self-supervised Skeleton-based Action Recognition (2023)
Journal Article
Men, Q., Ho, E. S., Shum, H. P., & Leung, H. (2023). Focalized Contrastive View-invariant Learning for Self-supervised Skeleton-based Action Recognition. Neurocomputing, 537, 198-209. https://doi.org/10.1016/j.neucom.2023.03.070

Learning view-invariant representation is a key to improving feature discrimination power for skeleton-based action recognition. Existing approaches cannot effectively remove the impact of viewpoint due to the implicit view-dependent representations.... Read More about Focalized Contrastive View-invariant Learning for Self-supervised Skeleton-based Action Recognition.

A Video-Based Augmented Reality System for Human-in-the-Loop Muscle Strength Assessment of Juvenile Dermatomyositis (2023)
Journal Article
Zhou, K., Cai, R., Ma, Y., Tan, Q., Wang, X., Li, J., Shum, H. P., Li, F. W., Jin, S., & Liang, X. (2023). A Video-Based Augmented Reality System for Human-in-the-Loop Muscle Strength Assessment of Juvenile Dermatomyositis. IEEE Transactions on Visualization and Computer Graphics, 29(5), 2456-2466. https://doi.org/10.1109/tvcg.2023.3247092

As the most common idiopathic inflammatory myopathy in children, juvenile dermatomyositis (JDM) is characterized by skin rashes and muscle weakness. The childhood myositis assessment scale (CMAS) is commonly used to measure the degree of muscle invol... Read More about A Video-Based Augmented Reality System for Human-in-the-Loop Muscle Strength Assessment of Juvenile Dermatomyositis.

Denoising Diffusion Probabilistic Models for Styled Walking Synthesis (2022)
Presentation / Conference Contribution
Findlay, E., Zhang, H., Chang, Z., & Shum, H. P. (2022, November). Denoising Diffusion Probabilistic Models for Styled Walking Synthesis. Presented at MIG 2022: The 15th Annual ACM SIGGRAPH Conference on Motion, Interaction and Games, Guanajuato, Mexico

Generating realistic motions for digital humans is time-consuming for many graphics applications. Data-driven motion synthesis approaches have seen solid progress in recent years through deep generative models. These results offer high-quality motion... Read More about Denoising Diffusion Probabilistic Models for Styled Walking Synthesis.