Hand Gesture Recognition (HGR) is a form of perceptual computing that allows artificial systems to capture and interpret human gestures. HGR has applications in human-machine interaction, virtual reality, augmented reality, and human behavior analysis. The human hand can assume a near-infinite number of poses and orientations to form myriad gestures, thus increasing the difficulty of the HGR task.
The hand skeleton of connected joints effectively describes the hand’s geometric shape and thus contains richer semantic gesture information while eliminating noise from individual differences in physical hand characteristics. The efficacy and computational efficiency of skeleton-based HGR frameworks can be significantly enhanced by transforming the hand gesture recognition task into an image classification task, taking advantage of the success of ML algorithms/architectures in the image classification domain.
This research thesis document presents a skeleton-based HGR framework that implements temporal information condensation via data-level fusion to encode spatiotemporal gesture information into RGB images. The spatiotemporal images are processed through an end-to-end ensemble multi-stream CNN architecture that generates the class probabilities.
The framework was extensively evaluated on the CNR, FPHA, LMDHG, SHREC2017, and DHG1428 HGR benchmark datasets. The framework’s performance was competitive with the SOTA for the evaluation datasets, within -4.10% and +6.86% of reported classification accuracies. The experimental results confidently demonstrated the viability of the temporal information condensation and data-level fusion techniques within the HGR domain.
School of Sciences and Engineering
Robotics, Control & Smart Systems Program
MS in Robotics, Control and Smart Systems
Dr. Maki Habib
Dr. Mohamed Moustafa
Committee Member 1
Dr. Amr Goneid
Committee Member 2
Dr. Hazem Abbas
Committee Member 3
Dr. Mohamed El-Morsi
Institutional Review Board (IRB) Approval
Not necessary for this item
(2023).Skeleton-Based Hand Gesture Recognition Using Data-Level Fusion [Master's Thesis, the American University in Cairo]. AUC Knowledge Fountain.
Yusuf, Oluwaleke. Skeleton-Based Hand Gesture Recognition Using Data-Level Fusion. 2023. American University in Cairo, Master's Thesis. AUC Knowledge Fountain.
Available for download on Thursday, December 19, 2024
Artificial Intelligence and Robotics Commons, Data Science Commons, Graphics and Human Computer Interfaces Commons, Robotics Commons